Write the claim before the test
State the product families, visible defect categories, inspection views, line speeds and environmental conditions in scope. Specify the required physical action and what happens to uncertain or uninspectable items. A claim about detecting surface breaks on one product is not evidence for every recipe, package or hidden defect.
Agree acceptance criteria before seeing the final results. Quality, engineering and production should understand the consequences of missed defects and unnecessary rejection. If the checkpoint supports a hazard-related control, involve the competent food safety team and establish the additional evidence required by the site's hazard analysis.
Keep development separate from evaluation
Training data teaches the model; development data helps select settings; final test data evaluates the frozen approach. Repeatedly tuning against the final test set turns it into another development set. Record exactly which model, thresholds, camera configuration and software build produced the reported result.
Split by physical product, batch or production interval where needed to avoid leakage. Adjacent frames of the same item must not quietly appear on both sides of the evaluation. Similar images can create an impressive score without demonstrating generalisation to future production.
Include independent reference decisions and preserve disagreement records. Reviewers should use the approved criteria rather than a model's predictions as their source of truth. Deliberately prepared challenges can cover rare conditions, but should be distinguished from naturally occurring defects and checked for realism.
Report the errors separately
Overall accuracy can hide poor defect detection when most production is acceptable. Report results by defect category and product family, with counts and denominators. Use a confusion matrix, explain the decision unit and distinguish image classification from successful physical removal.
| Measure | Definition | Interpretation |
|---|---|---|
| Defect recall | Detected defective items ÷ reference-defective items | How often the target defect was found in the test |
| False-reject rate | Rejected acceptable items ÷ reference-acceptable items | Unnecessary rejection under the tested conditions |
| Reject precision | Reference-defective items among predicted rejects ÷ predicted rejects | Depends on defect prevalence in the evaluation |
| Uninspectable rate | Items without a usable inspection ÷ presented items | Coverage lost to capture or process limitations |
| Removal success | Confirmed removals ÷ items commanded for removal | Actuator and tracking performance, not model accuracy |
Express uncertainty honestly
Zero observed misses does not establish a zero miss rate. For independent, representative defective trials with no misses, the “rule of three” gives an approximate one-sided 95% upper bound of 3 divided by the number of trials for the miss probability. It is a planning approximation, not a substitute for a justified sampling design.
Correlated examples, selective sampling and unrealistic challenge items weaken the interpretation. A small set repeated many times may exercise the mechanism while saying little about defect diversity. Report those repetitions separately and seek statistical advice when the required assurance or sampling structure is complex.
Zero observed misses does not establish a zero miss rate.
Challenge the complete checkpoint
Run at the intended speed range with normal spacing variation. Test start-up, changeover, cleaning recovery, camera obstruction, missing triggers, network interruption and reject failures. A deliberately introduced challenge must follow an approved procedure that prevents the test material entering released production unintentionally.
Evaluate whether the correct physical item was removed, whether uncertain states followed the authorised route and whether records support batch review. A high offline score cannot compensate for late results, tracking loss or a reject device that fails without an alarm.
Maintain the evidence after release
Set review intervals and change triggers. New ingredients, changed film, lighting replacement, camera movement and software updates can all alter performance. Drift monitoring should prompt investigation; it should not silently retrain or approve a model without controlled evaluation.
ISO 9001 provides a quality-management context for documented processes. ISO/IEC 25012 describes data-quality concepts, and ISO/IEC 23894 addresses AI risk management. The NIST AI Risk Management Framework offers another useful structure for managing AI-related risk. None supplies a universal food-inspection acceptance percentage.
Keep the approved protocol, raw counts, exceptions, corrective actions and release decision together. The strongest conclusion is a bounded one: this version met these criteria on this evidence, within these operating limits. That statement is more useful than a broad promise of intelligent quality control.