TinyGLASS: Real-Time Self-Supervised In-Sensor Anomaly Detection

1. Problem Statement

Industrial quality control relies heavily on visual inspection, but traditional deep learning approaches face two major hurdles in real-world manufacturing:

• Data Scarcity: High-quality labeled defect data is often unavailable, especially in "high-mix-low-volume" production environments. This necessitates self-supervised approaches that learn normal patterns using only defect-free data.
• Deployment Constraints: State-of-the-art self-supervised models (e.g., GLASS) are computationally heavy and require significant memory, making them unsuitable for resource-constrained edge devices or in-sensor computing platforms. There is a gap between high-performance models and their ability to run in real-time on low-power hardware like the Sony IMX500 intelligent vision sensor.

2. Methodology: TinyGLASS Architecture

The authors propose TinyGLASS, a lightweight adaptation of the GLASS framework optimized for in-sensor execution.

• Backbone Replacement: The original WideResNet-50 backbone (24.9M parameters) is replaced with a compact ResNet-18. This reduces the feature dimensionality from 1536 to 384 while maintaining multi-scale feature fusion (concatenating level-2 and level-3 features).
• Hardware-Aware Modifications: To ensure compatibility with Sony's Model Compression Toolkit and the IMX500's static graph tracing requirements:
  • Static Graph Tracing: Dynamic tensor reshaping and unsupported operations (like PyTorch hooks) are eliminated. The forward pass is modified to return only required layers directly.
  • Shape Consistency: The discriminator input path is redesigned to maintain 4D tensor representations throughout, avoiding dynamic shapes caused by flatten operations.
  • Quantization: The model is quantized to INT8 (both weights and activations) using Sony's toolkit.
• Training Objective: The model uses a self-supervised approach with only normal data during training. It employs two complementary loss functions:
  • Binary Cross-Entropy (BCE): For the discriminator.
  • Focal Loss: To handle class imbalance between normal and synthetic defect regions.
  • Global & Local Anomaly Scores (GAS & LAS): The architecture aggregates global contextual deviations and fine-grained spatial discrepancies to detect both large and subtle defects.
• Inference Flow: The sensor executes the network to generate patch-level anomaly heatmaps. Image-level decisions are aggregated on the host (e.g., Raspberry Pi) if needed, minimizing bandwidth usage.

3. Key Contributions

1. First Real-Time In-Sensor Anomaly Detection: Demonstrates the first successful deployment of a self-supervised anomaly detection model running at 20 FPS directly on the Sony IMX500 intelligent vision sensor.
2. Extreme Model Compression: Achieves an 8.7× reduction in active parameters (from 24.9M to ~2.85M) while maintaining competitive performance. The model fits within the sensor's strict 8 MB memory constraint.
3. Robustness Analysis: Thoroughly evaluates the model's resilience to contaminated training data (where normal training sets contain up to 30% defective samples), showing stable performance under moderate noise.
4. New Dataset (MMS Dataset): Introduces a custom industrial dataset captured with both a high-resolution microscope and the IMX500 sensor, enabling cross-device evaluation and domain transfer analysis.

4. Experimental Results

The model was evaluated on the MVTec-AD benchmark and the custom MMS Dataset.

• Performance on MVTec-AD:
  • Image-Level AUROC: 94.2% (INT8 quantized). This is a 7.7 percentage point drop from the original GLASS (99.1%) but remains highly competitive.
  • Pixel-Level AUROC: 92.9%.
• Performance on MMS Dataset:
  • Achieved 88.9% Image-Level AUROC when trained and tested on microscope images.
• Robustness to Contamination:
  • On the MVTec-AD "carpet" category, with 30% contamination in the training set, the Image-Level AUROC dropped from 95.6% (0% contamination) to 88.3%, demonstrating significant resilience.
• System Efficiency (IMX500):
  • Throughput: ~20 FPS.
  • Compute: 1.88 G MACs per inference (256×256 input).
  • Energy Efficiency: 470 GMAC/J.
  • Power Consumption: 4.0 mJ per inference.
  • Latency: Real-time end-to-end processing.

5. Significance

TinyGLASS bridges the critical gap between advanced AI algorithms and industrial edge deployment. By proving that high-accuracy, self-supervised anomaly detection can run entirely on the sensor (in-sensor computing), the work offers a solution for:

• Reduced Latency & Bandwidth: Eliminating the need to transmit raw video streams to a central server.
• Energy Efficiency: Drastically lowering power consumption compared to cloud-based or edge-compute solutions.
• Scalability: Enabling small and medium enterprises (SMEs) with limited infrastructure to implement real-time, high-quality visual inspection without expensive hardware or massive labeled datasets.

This work establishes a new baseline for TinyML in industrial vision, proving that aggressive model compression and quantization do not preclude real-time, high-performance anomaly detection.