ADOPD: Reference-Privileged On-Policy Distillation for MLLM-Based Industrial Anomaly Detection

Technical Summary: ADOPD for Industrial Anomaly Detection

Problem Statement
Industrial Anomaly Detection (IAD) requires identifying fine-grained deviations from normal visual patterns. While Multimodal Large Language Models (MLLMs) offer promising interfaces for open-set recognition and explainable reasoning, their open-domain representations often struggle to capture product-specific anomaly patterns. Current approaches leverage reference comparison at inference time (e.g., one-shot learning) to improve accuracy by providing product-specific visual context. However, this explicit reference-conditioned inference introduces computational overhead due to the need for image retrieval and processing for every query. The core challenge addressed by this work is whether the benefits of reference comparison can be internalized into model parameters during training, allowing a query-only model to perform effectively at inference without access to reference images.

Methodology: ADOPD Framework
The authors propose ADOPD (Reference-Privileged On-Policy Distillation), a framework designed to distill the benefits of reference comparison into a query-only student model. The approach operates under a "privileged information" setting where a frozen teacher model has access to both the query image and reference evidence during training, while the student only sees the query.

The methodology consists of two primary mechanisms:

1. Reference-Privileged On-Policy Distillation (OPD):
   Unlike standard Supervised Fine-Tuning (SFT) which uses fixed teacher targets (off-policy), ADOPD evaluates the teacher on prefixes generated by the student itself (on-policy). This mitigates exposure bias.

   • The student samples rollouts from a query-only prompt.
   • A frozen teacher evaluates these student-generated tokens under a matched reference (semantically consistent with the query anomaly) and a mismatched reference (deliberately breaks the valid query-reference relation).
   • The token-level learning direction is defined by the log-probability ratio between the matched-reference teacher and the student. This specifies what the student should learn (reinforcing tokens preferred by the teacher under valid reference conditions).

2. Contrastive Reference Weighting:
   To determine how strongly to learn, ADOPD introduces a mechanism to calibrate the update magnitude based on the specificity of the reference.

   • The teacher's likelihood gap between the matched and mismatched views defines a contrastive reference support score. This score estimates whether the teacher's preference is genuinely grounded in the reference comparison rather than language priors or query cues.
   • This score is aggregated over the sequence and normalized against the group mean and standard deviation to produce a group-relative reference weight.
   • The final training objective combines the token-level advantage (direction) with the sequence-level weight (magnitude), creating a calibrated advantage for policy-gradient updates.

Key Contributions

• ADOPD Framework: A novel reference-privileged on-policy distillation framework that internalizes inference-time reference comparison benefits into model parameters, enabling high-performance zero-shot inference.
• Asymmetric Reference Intervention: The introduction of a matched/mismatched reference control mechanism. The matched view defines the token-level OPD direction, while the mismatched view controls the sequence-level update strength, ensuring the model learns fine-grained anomaly inspection strategies rather than generic visual supervision.
• Performance Gains: The method demonstrates that separating "what to learn" from "how strongly to learn" allows the model to retain the benefits of reference comparison without requiring reference images at inference.

Experimental Results
Experiments were conducted on the MMAD benchmark, which aggregates MVTec-AD, VisA, MVTec-LOCO, and GoodsAD.

• Zero-Shot Performance: Using only 6,000 constructed training records, ADOPD achieved an average accuracy of 77.31% on the MMAD benchmark under zero-shot inference.
• Backbone Improvement: This represents a 6.14 percentage point improvement over the Qwen3-VL-4B backbone.
• Comparison to Baselines:
  • ADOPD outperformed the reference-conditioned one-shot setting of the same backbone by 2.64 points.
  • It surpassed the strongest listed IAD-specific model (Reason-IAD, 8B) by 0.60 points, despite using a smaller 4B backbone.
  • It improved upon SFT-based distillation by 1.61 points.
• Granularity Analysis: Experiments showed that using regional crops (matched/mismatched) for privileged supervision yielded better results than full-image references, indicating the model learns local anomaly patterns rather than global product features.
• Qualitative Analysis: Attribution maps indicated that ADOPD produces stronger responses near annotated defects (e.g., thin scratches, isolated spots) compared to baselines, demonstrating improved sensitivity to defect-relevant visual evidence.

Significance and Claims
The paper claims that ADOPD successfully internalizes the benefits of reference comparison, allowing MLLMs to perform fine-grained anomaly inspection without the computational overhead of retrieving reference images at inference. The authors emphasize that the method learns a specific "anomaly inspection strategy shaped by reference comparison" rather than merely adding generic visual supervision. The work highlights that contrastive reference supervision is crucial for retaining these benefits, as evidenced by the performance drop when using uncalibrated OPD or SFT. The authors note that the current approach relies on paired images and defect annotations, suggesting future work could explore weaker or automatically constructed privileged references.