Rule-Based, Supervised, or Unsupervised: Choosing a Visual Inspection Method by Defect
Rule-based checks, supervised AI, and anomaly detection answer different inspection questions. Here is how to match each method to the defect you need to catch, how to combine them on one station, and how to set the error budget first.
Watch one station work
Three methods. One part. One decision.
What the station knows
Nothing decided yet
Each method gets its turn before the part is routed.
- Caught by
- Pending
- Destination
- Not routed yet
The part arrives under controlled light and one frame is captured.
Choose a part or a step to explore it.
We recently spent a working session with a machine vision engineering team, walking through how they set up camera inspection on production lines. We expected to talk about cameras and AI models. Most of the conversation was about something simpler: which question each inspection tool is built to answer.
Their point was that inspection stations rarely fail because the technology is weak. They fail because one method is asked to answer every question. A fixed threshold is asked to judge a textured surface. A defect model is asked to find a defect nobody showed it. An anomaly model is asked to count parts. Each of those is the wrong tool for that question, and the result is either scrapped good parts or shipped bad ones.
This article turns that session into a practical guide. It covers the three families of visual inspection method (rule-based checks, supervised AI, and unsupervised anomaly detection), what the published evidence says about each, how to combine them on one station, and the decisions to write down before anyone trains a model. It is the first in a series. Later articles cover defect size and camera resolution, lighting, indoor and outdoor conditions, training data and thresholds, and keeping a station accurate after go-live.
Three methods, three different questions
Picture a machined housing on a conveyor. The quality engineer has two complaints. The camera rejects good parts whenever the coolant film looks different, and operators have started overriding it. Last month a housing shipped with a chipped flange, because nobody had written a check for chips. Both complaints come from asking one method to answer several questions.
Rule-based checks answer "does this feature meet a specification I can write down?" They measure an edge-to-edge distance, count holes, confirm a label is present in a region, read a code, or compare a color against a reference. The engineer sets thresholds directly, and when a result is wrong you can see which parameter caused it.
Supervised AI answers "is this named defect present, and where?" You label examples of the defect, such as a scratch, void, or burr, and the model learns to find that class. The output type matters. A peer-reviewed survey of deep learning for defect detection describes detectors that return a location and class probability, and segmentation networks that make a prediction for every pixel [1]. An image-level label supports only pass or fail. A box supports counting and locating. A pixel outline lets a rule measure the defect's length or area against a tolerance.
Unsupervised anomaly detection answers "does this part look different from the good parts I have seen?" The model trains on defect-free images only and flags anything that departs from that learned normal [4]. It is the only one of the three that can catch a defect nobody has seen before.
Write the question before you pick the tool
For each defect, finish the sentence "a part fails if...". If the sentence contains a number, a count, or a code, start with a rule. If it names a defect you can collect examples of, consider supervised AI. If the honest answer is "anything that looks wrong," that is an anomaly detection question.
What the evidence says about each method
Supervised AI needs fewer defect images than most plants assume
The common objection to supervised inspection is that defects are rare, so you can never collect enough. Research on surface crack detection showed a segmentation-based network learning from roughly 25 to 30 defective samples rather than hundreds or thousands [2]. That result comes from one dataset and one defect type, so treat it as a reason to try, not a planning number. Follow-up work found that mixing many cheap good-or-bad image labels with a small number of detailed defect outlines approaches the performance of fully outlined training data [3].
The limitation is just as important. Models trained on a few known defect types can become biased toward those types and miss defects that look different [6]. A supervised model will not find the chipped flange unless chips were in its training set.
Anomaly detection looks perfect on clean benchmarks and much weaker on hard ones
Anomaly detection methods trained only on good images have reported image-level detection scores up to 99.6% on a widely used industrial benchmark [4]. That number is real, and it is also misleading if read alone. A 2025 follow-up benchmark added what production lines actually have: lighting changes between training and testing, high variation among good parts, transparent and overlapping objects, and very small defects. On that benchmark, state-of-the-art methods scored below 60% on its defect-localization metric [5].
There is a second blind spot. Research on what its authors call logical anomalies found that earlier benchmarks were biased toward local damage such as scratches and dents [7]. A logical anomaly is a part where every component looks normal but the arrangement is wrong: two fasteners where three belong, the right label on the wrong variant, connectors in swapped positions. Methods that judge small image patches tend to miss those errors [7].
The practical reading: anomaly detection is a valuable safety net, but it needs stable lighting, good-part training images that cover normal variation, and a human review step. Count, sequence, and variant checks belong with rules or classification.
Hybrids are what work in production
The strongest published production evidence we found is a hybrid. An automotive supplier's spring inspection system combines conventional image processing, a cascade of neural networks, a rule-based reject decision that requires a minimum number of flagged regions, and human review of uncertain parts [8]. In a long-term test on 9,020 parts, the error rate fell from 16.83% with the previous system to 2.80% with the new one, and neither system passed a defective part [8]. Continued learning later brought the error rate from 4.42% to 2.58% over 16,153 parts [8].
That is one plant and one part family, so the percentages will not transfer. The design goal transfers: reduce false rejects while holding escapes at zero, using each method where it is strongest.
How to match methods to defects
The table below compares the three methods on the factors that decide daily performance.
| Factor | Rule-based checks | Supervised AI | Anomaly detection |
|---|---|---|---|
| Best question | Dimension, count, presence, position, code read | Named defect classes with visual variety | Unexpected or rare defects |
| Training data | None; engineer sets parameters | Labeled good and defective images | Good parts only |
| Output | Measured value and pass or fail | Class, location, often an outline | Anomaly score and heat map |
| Typical failure | False rejects when normal variation exceeds a fixed threshold | Misses defect types absent from training | Flags harmless variation and lighting changes; misses wrong-combination errors |
| How to fix it | Adjust thresholds and regions | Add or relabel examples | Add good images that cover normal variation |
| Explaining a reject | Parameter and measured value | Labeled class and outline | Region that looked different |
Three patterns come up repeatedly:
- Dimensional and assembly checks stay rule-based. Hole diameter, connector seating, label position, and code grading have written specifications. A rule produces a traceable number a quality engineer can defend in a customer audit.
- Cosmetic defects on variable surfaces move to supervised AI. Cast, machined, and textured surfaces vary from part to part. A contrast threshold that catches a real scratch also catches tool marks and coolant streaks. A model trained on labeled scratches and on labeled harmless variation can separate them.
- Rare and unforeseen damage gets an anomaly layer. A chip or foreign object that has never occurred cannot be labeled in advance. Route these flags to review until the team understands what the model is catching.
Build the station in six steps
Watch the animated walkthrough at the top of this article: one part passes through all six steps, and each example part is caught by a different method. In order:
1. Capture. The part arrives under controlled light and one frame is captured. Every later step depends on this image, so fix lighting and fixturing before tuning any model. 2. Align. A rule-based locator finds the part and corrects its position, so every later check inspects the same region. 3. Measure. Rules check what has a specification: dimensions, counts, presence, and codes. Confirm the product variant here so the right tolerances apply. 4. Find named defects. A supervised model outlines the defect classes in your defect catalog. A rule then measures each outline, because a short scratch and a long one may have different dispositions. 5. Flag the unexpected. An anomaly model runs last and sends anything unfamiliar to human review rather than automatic rejection. 6. Route. Each finding goes to a defined destination: ship, reject, or review, with the reason recorded.
Step 4 is where most teams gain the most. A model alone says "scratch here." A rule applied to the outline says, for example, "scratch 1.4 mm long in the sealing zone, limit 1.0 mm." The second sentence is what a quality engineer can act on and what belongs in the inspection record.
Back on the housing line, this sequence resolves both complaints. A supervised model trained on real scratches and on coolant film stops rejecting good parts for surface variation. The anomaly layer, routed to review, catches the next chipped flange even though nobody ever labeled a chip.
Write the error budget before choosing a tool
Every method trades two errors against each other: false rejects (good parts scrapped or reworked) and escapes (defective parts shipped). The right balance differs by defect. A blemish on a hidden face may tolerate some escapes to protect yield. A missing safety clip tolerates none.
The NIST AI Risk Management Framework calls for documenting test sets and metrics, evaluating under conditions similar to deployment, and monitoring performance in production [9]. For an inspection station, that becomes a short record per defect:
| Defect | Method | Escape tolerance | False reject tolerance | Validation evidence |
|---|---|---|---|---|
| Connector not seated | Rule-based position | None accepted | Low | Seated and unseated samples at line speed |
| Scratch in sealing zone | Supervised outline plus length rule | None above size limit | Moderate | Labeled scratch set and coolant film set |
| Blemish, hidden face | Supervised outline | Some accepted | Low | Blemish set reviewed by quality |
| Unknown damage | Anomaly detection to review | Reviewed, not auto-rejected | Limited by review capacity | Good-part set covering shifts, lots, and lighting |
Google's machine learning guidance makes a related point: you do not need machine learning to launch, and simple rules are a valid first version [10]. The same guidance suggests moving to a learned model when the rules become too complex to maintain [10]. In inspection terms, if a rule-based scratch check has grown into a stack of exceptions for coolant, lighting, and supplier lots, that is the signal to try supervised AI on that defect, not on the whole station.
Where maintenance teams fit in
Inspection results are also equipment signals. A rising rate of one defect class can point to tool wear, a drifting fixture, or a degrading light on the inspection station itself. When a supervised model reports scratch length per part, the trend of that value tells you something about the machine upstream.
Route those trends to people who can act on them. A step change in burr detections after a tool change belongs to the machining cell lead. A slow rise in anomaly flags across all parts can mean the station's own lighting or lens needs attention. Monitory connects inspection events to asset records and work orders, so a defect trend becomes a planned task with an owner instead of a chart someone reads after a customer complaint. If you want a second opinion on one of your own stations, the free inspection method assessment at the end of this article maps each of your defects to a method. Our guide on tuning condition alerts with technician feedback covers the same feedback loop for sensor alerts.
Summary
Rule-based checks, supervised AI, and anomaly detection are not competing products. They answer different questions. Use rules for anything with a written specification, supervised AI for named defects on variable surfaces, and anomaly detection as a reviewed safety net for what nobody anticipated.
Next steps for one station this month:
1. List every defect the station must catch and write a "fails if" sentence for each. 2. Assign a method to each defect using the comparison table. 3. Write the escape and false reject tolerance for each defect, with a named quality owner. 4. Collect validation images under real production conditions, including harmless variation that should pass. 5. Route anomaly flags to review and defect trends to the asset owner.
For the full path from detection to disposition, read computer vision quality inspection: from defect detection to corrective action. For other uses of the same cameras, see five computer vision uses that are not quality inspection, or review computer vision on the Monitory platform.
Frequently asked questions
Is AI inspection always better than rule-based inspection?
No. When the pass criterion is a dimension, count, position, or code, a rule is usually simpler to validate and explain. AI earns its place where normal part-to-part variation makes fixed thresholds reject good parts, or where defects vary too much to describe with parameters.
How many images does supervised inspection need?
It depends on defect variety, surface variation, and image consistency. Published research has shown a network learning a surface crack from a few dozen defective samples [2], and has shown that a few detailed defect outlines add a lot of value [3]. Your requirement comes from validation on your own parts. Collect harmless variation as carefully as defects, because that is what prevents false rejects.
Can anomaly detection replace a defect catalog?
No. Anomaly detection tells you a part looks different, not which defect it has or whether it matters. Benchmark results drop sharply when lighting changes and good parts vary [5], and patch-based methods tend to miss wrong-count or wrong-combination errors [7]. Use it alongside a defect catalog and send its flags to review.
Should anomaly detection reject parts automatically?
Start with review routing. Once the team has reviewed enough flags to know which are real defects and which are harmless variation, add the harmless cases to the good-part training set and decide per defect whether automatic rejection is justified.
When should a model be retrained?
When production conditions change in ways the training data did not cover: a new supplier lot, a surface finish change, a lighting replacement, or a new product variant. Keep a fixed validation set and rerun it after every model or setting change, so you can see whether fixing one defect made another worse.
References
[1] J. Yang et al., "Using Deep Learning to Detect Defects in Manufacturing: A Comprehensive Survey and Current Challenges", Materials, 2020. https://pmc.ncbi.nlm.nih.gov/articles/PMC7766692/
[2] D. Tabernik, S. Šela, J. Skvarč, D. Skočaj, "Segmentation-Based Deep-Learning Approach for Surface-Defect Detection", Journal of Intelligent Manufacturing, 2020. https://arxiv.org/abs/1903.08536
[3] J. Božič, D. Tabernik, D. Skočaj, "Mixed Supervision for Surface-Defect Detection: From Weakly to Fully Supervised Learning", Computers in Industry, 2021. https://arxiv.org/abs/2104.06064
[4] K. Roth et al., "Towards Total Recall in Industrial Anomaly Detection", CVPR 2022. https://arxiv.org/abs/2106.08265
[5] L. Heckler-Kram et al., "The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection", 2025. https://arxiv.org/abs/2503.21622
[6] X. Yao et al., "Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detection", CVPR 2023. https://arxiv.org/abs/2207.01463
[7] P. Bergmann et al., "Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization", International Journal of Computer Vision, 2022. https://link.springer.com/article/10.1007/s11263-022-01578-9
[8] P. Trampert et al., "Production-Ready End-to-End Visual Quality Inspection for Defect Detection on Surfaces Based on a Multi-Stage AI System", IMPROVE 2024. https://scitepress.org/publishedPapers/2024/125365/pdf/index.html
[9] NIST, "Artificial Intelligence Risk Management Framework (AI RMF 1.0)", 2023. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
[10] Google, "Rules of Machine Learning", Google for Developers. https://developers.google.com/machine-learning/guides/rules-of-ml
Free for manufacturing and maintenance teams
Free inspection method assessment
Get a second opinion on which method should catch each defect on one of your stations.
You send
- Your defect list with pass and fail rules
- Sample images of good and defective parts
You get back
- A defect-by-defect method map: rules, supervised AI, or anomaly detection
- A draft escape and false reject budget to validate against
Not sure where camera inspection fits yet? A discovery call walks through your lines and assets to find the inspection and monitoring problems worth solving first. Prefer someone on site? See the free half-day on-site assessment.
Related articles
Ready to put this into practice?
See how Monitory helps manufacturing teams implement these strategies.