Benchmarking Recurrent Event-Based Object Detection for Industrial Multi-Class Recognition on MTevent

Technical Summary: Benchmarking Recurrent Event-Based Object Detection for Industrial Multi-Class Recognition on MTevent

Problem Statement
Event cameras offer high temporal resolution, high dynamic range, and reduced motion blur, making them theoretically ideal for industrial robotics operating under fast motion, challenging illumination, and partial occlusion. However, existing event-based object detection research is heavily skewed toward autonomous driving scenarios (e.g., GEN1) or single-category robotics tasks (e.g., person detection in PEDRo). There is a significant gap in understanding how current detectors perform in industrial multi-class environments, which are characterized by diverse object categories, cluttered layouts, repeated structures, and frequent human-object interactions. Furthermore, it remains unclear whether recurrent temporal modeling—which accumulates sparse event evidence over time—provides a tangible benefit over non-recurrent approaches in these complex, long-tail industrial settings.

Methodology
The authors present a controlled benchmarking study using the MTevent dataset, an industrial event-camera benchmark recorded in warehouse and factory-like scenes. The study focuses on 17 detection classes (16 rigid objects plus humans) derived from the dataset annotations.

• Models: The study compares a non-recurrent YOLOv8s baseline against a recurrent variant, ReYOLOv8s, which augments the YOLOv8s backbone with ConvLSTM modules to aggregate temporal features.
• Input: Models utilize 5-channel temporally binned event-tensor inputs (5 temporal bins, 50 ms windows, 256×320 resolution).
• Variables: The evaluation analyzes three primary factors:
  1. Recurrence: Comparing non-recurrent (clip length 1) vs. recurrent models.
  2. Temporal Clip Length: Testing recurrent models with clip lengths of {3, 7, 11, 21} using stride-based temporal sampling.
  3. Pretraining/Transfer: Evaluating initialization from two external event domains: GEN1 (driving-oriented) and PEDRo (robotics-oriented person detection), alongside scratch training.
• Training Protocol: All experiments use consistent hyperparameters (AdamW, specific learning rate, batch size 16) and evaluation metrics (mAP50, mAP50–95) on a custom 60/13/2 train/validation/test scene split to ensure controlled comparison.

Key Results

• Recurrence vs. Non-Recurrence: On the MTevent validation split, the best scratch-trained recurrent model (clip length 21) achieved 0.285 mAP50, a 9.6% relative improvement over the non-recurrent YOLOv8s baseline (0.260 mAP50). This confirms that temporal context is beneficial for industrial event-based detection.
• Clip Length Dynamics: Performance for scratch-trained models did not increase monotonically with clip length; performance dipped at clip length 11 before rising at 21. In contrast, GEN1-pretrained models showed consistent improvement as clip length increased (from 0.293 at length 3 to 0.329 at length 21), suggesting pretraining stabilizes temporal optimization.
• Pretraining Impact:
  • GEN1 Initialization: Yielded the best overall result (0.329 mAP50), significantly outperforming scratch training.
  • PEDRo Initialization: Performed poorly (0.251 mAP50 at clip 11), falling below both the best recurrent scratch model and the non-recurrent baseline. This indicates that mismatched source-domain pretraining (e.g., person-only data applied to multi-class industrial scenes) can be detrimental.
• Zero-Shot Transfer: Before fine-tuning, GEN1-pretrained models failed to detect humans in MTevent (AP50 = 0.000), while PEDRo-pretrained models achieved limited success (AP50 = 0.136). This highlights that zero-shot transfer depends heavily on source-target similarity in scene structure and motion patterns, not just label overlap.
• Failure Modes: Persistent errors were dominated by class imbalance (rare classes) and human-object interactions, where carrying scenarios create merged event blobs that reduce class separability.

Contributions and Significance
The authors explicitly state that their contribution is not a new architecture or a new dataset. Instead, the work serves as a focused benchmarking and analysis study of recurrent event-based detection in industrial environments.

• Significance: The paper positions itself as a necessary stress test for current model families in complex, multi-class industrial settings, an area that remains comparatively underexplored.
• Key Takeaways:
  1. Recurrence improves performance, but event-domain pretraining (specifically from a well-aligned source like GEN1) has a stronger impact than recurrence alone.
  2. Source-domain alignment is critical; mismatched pretraining can degrade performance below scratch training levels.
  3. In the current setting, gains may be more achievable through improved data coverage and transfer protocols rather than increased architectural complexity.
  4. The study provides a reproducible reference point (code and results available) for future industrial event-camera research, highlighting specific bottlenecks like long-tail class scarcity and human-object interaction.