Accelerating New Product Introduction for Visual Quality Inspection via Few-Shot Diffusion-Based Defect Synthesis

Technical Summary: Accelerating NPI via Few-Shot Diffusion-Based Defect Synthesis

1. Problem Statement

In industrial automated visual inspection, the most critical defects (e.g., hairline scratches, punctures, folds) are often "rare events." This creates a severe data scarcity problem during the New Product Introduction (NPI) phase. Traditional supervised learning models require large, annotated datasets to be effective, but during the early stages of a new production line, manufacturers lack sufficient real-world defect examples. This creates a gap between the immediate need for automated quality control and the delayed availability of representative training data. Standard data augmentation (rotation, scaling) fails to capture the complex structural and textural variability required for high-fidelity defect modeling.

2. Methodology

The authors propose an end-to-end generative pipeline based on Latent Diffusion Models (LDMs) designed to disentangle defect morphology from its background. The framework consists of three modular stages:

• Stage 1: Defect Embedding Learning (Masked Textual Inversion): To prevent "background leakage" (where the model accidentally learns the texture of the original surface along with the defect), the authors use masked textual inversion. By applying a binary mask during the optimization of the new word embedding, the model learns a compact representation (only a few KBs) that captures only the visual essence of the defect.
• Stage 2: Conditioned Generation (Noise-Blended Diffusion): The learned embedding is used to synthesize the defect onto new, defect-free target surfaces. The process uses a noise-blending approach inspired by RePaint, where unmasked regions are anchored to the original background using a forward-diffused sample, while the reverse diffusion process freely generates the defect within the masked area. A dynamic Classifier-Free Guidance (CFG) schedule is used to ensure structural coherence before refining textural details.
• Stage 3: Seamless Integration (Post-Processing): To eliminate visual discontinuities, the pipeline employs statistical color/lighting matching and gradient-aware refinement. This stage treats integration as a local structural optimization problem, matching the morphological gradients of the synthetic patch to the target surface to ensure photorealistic adherence.

3. Key Contributions

• Unified Generative Pipeline: A specialized framework for industrial inspection that achieves high-fidelity, localized defect synthesis with minimal background contamination.
• Dual-Scenario Support: The framework is designed for both Few-Shot Augmentation (enriching a small set of real defects) and Zero-Shot Adaptation (transferring defect concepts from a source material to a completely new target material).
• Hardware Efficiency: The system is lightweight enough to run on standard industrial workstations (e.g., an NVIDIA RTX 5080) with a memory footprint under 6 GB, making it practical for factory deployment rather than just research.

4. Experimental Results

The framework was evaluated using a private industrial dataset and the MVTec AD dataset, using RF-DETR as the downstream detector. Performance was measured using mAP@0.01 (to account for the difficulty of detecting thin scratches).

• Few-Shot Data Augmentation (Experiment 1):
  • Starting with only 4 real scratch samples, the "Real Only" baseline achieved 78.8% mAP.
  • By adding 150 synthetic variations (Real+Synthetic), the mAP improved to 83.3%, with a significant boost in Recall (from 50.4% to 64.3%).
• Zero-Shot Domain Adaptation (Experiment 2):
  • A model trained on "Product A" (white metal) and tested on "Product B" (gray metal) achieved only 65.0% mAP.
  • By using the pipeline to synthesize defects directly onto "Product B" surfaces and fine-tuning the model, the mAP jumped to 85.1%.

5. Significance

The research demonstrates that generative AI can fundamentally shift industrial quality control from a reactive to a proactive paradigm. Instead of waiting weeks or months for defects to naturally occur on a new production line, manufacturers can use a handful of examples to generate high-fidelity training data. This significantly accelerates the NPI timeline, allowing robust, automated inspection systems to be operational from "Day 1" of production.