LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices

Technical Summary: LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices

Problem Statement
Event-based vision sensors offer a promising paradigm for energy-aware artificial intelligence by providing sparse, low-latency visual signals that eliminate redundant data processing common in frame-based imaging. However, the asynchronous, noise-prone, and sparse nature of event streams poses significant challenges for conventional deep learning models. Standard Convolutional Neural Networks (CNNs) are often computationally intensive and ill-suited for the low-power embedded platforms required for edge computing. Furthermore, existing solutions, such as Spiking Neural Networks (SNNs) or transformer-based architectures, introduce substantial computational overhead or complexity that hinders real-time deployment on resource-constrained devices. There is a critical need for specialized learning architectures that can efficiently encode the spatiotemporal structure of event data while maintaining high recognition fidelity with minimal energy consumption.

Methodology
The authors propose "LiteEvent-AE," a compact and configurable event-driven autoencoder designed to compress neuromorphic data while preserving essential spatiotemporal structures. The architecture consists of two primary phases:

1. Event Representation and Encoding: The system utilizes dense event representations, converting asynchronous event streams into structured, frame-like tensors via event accumulation. The core is a lightweight convolutional encoder-decoder structure. The encoder employs a stack of convolutional blocks with Rectified Linear Unit (ReLU) activation and spatial downsampling (Max pooling) to map high-dimensional event frames into a low-dimensional latent space. The use of ReLU is highlighted for its ability to introduce non-linearity, suppress noise inherent in sparse event data, and promote sparse activations that align with the data's nature.
2. Classification Adaptation: To transform the autoencoder into a classifier, the pretrained encoder is repurposed as a fixed feature extractor. Two fully connected layers are appended to the latent output to perform decision boundary learning. This design decouples representation learning from task-specific discrimination, minimizing parameter count.
3. Adaptive Thresholding: The framework supports adaptive event thresholding, allowing the system to dynamically adjust event detection sensitivity based on environmental conditions. This reduces redundant event generation without compromising essential visual information.

Key Contributions

• Architecture: Introduction of a configurable, lightweight autoencoder tailored specifically for event-based object classification, capable of operating on static event frames using standard convolutions.
• Scalability and Robustness: Demonstration of the architecture's robustness across multiple event activity thresholds, validating its adaptability to varying noise levels and scene dynamics.
• Hardware Efficiency: Comprehensive evaluation of the model on resource-constrained embedded platforms, specifically the Raspberry Pi 4B (CPU) and NVIDIA Jetson Nano (GPU).
• Performance Benchmarking: A rigorous comparison against State-of-the-Art (SOTA) CNNs, including the YOLO family (v4, v7, v9), EfficientDet-b0, MobileNet-v1, and YuNet.

Experimental Results
The model was evaluated on the Smart Event Face Dataset (SEFD) and the Event-Based Crossing Dataset (EBCD) across various temporal thresholds.

• Accuracy vs. Parameters: On the SEFD dataset, the proposed autoencoder classifier achieved competitive accuracy compared to YOLOv9 (within 4–6% difference) while utilizing 35.6× fewer parameters (1.7M parameters for the 100% model vs. 60.5M for YOLOv9). The reduced 50% filter configuration further lowers the parameter count to 458K. On the EBCD dataset, the model achieved an average accuracy of 93.33% across tested thresholds, outperforming YOLOv4, YOLOv7, and MobileNet-v1.
• Reconstruction Fidelity: The autoencoder demonstrated strong reconstruction capabilities. The full-capacity model achieved a Structural Similarity Index Measure (SSIM) of 0.939 on SEFD, while the reduced 50% filter configuration maintained a high SSIM of 0.915, indicating resilience under significant architectural compression.
• Latency and Throughput:
  • On the NVIDIA Jetson Nano, the 50% classifier model delivered a real-time throughput of 44.8 FPS.
  • On the Raspberry Pi 4B, the proposed model exhibited up to 414× more FPS compared to YOLO-family models, significantly outperforming other lightweight baselines like MobileNet-v1 and YuNet. Specifically, the autoencoder classifier achieved 22.1× the FPS of YOLOv9 on the Raspberry Pi 4B. On the Jetson Nano, the classifier demonstrated 21.4× higher FPS than YOLOv4.
• Energy Efficiency: The most significant finding relates to energy consumption. On the Raspberry Pi 4B, the 50% autoencoder classifier consumed only 16.19 Joules to process 100 images. This represents approximately 726.3× lower energy consumption compared to YOLOv9 under the same evaluation protocol.

Significance and Claims
The paper claims that LiteEvent-AE successfully addresses the trade-off between computational efficiency and recognition accuracy in event-based vision. By leveraging a compact autoencoder architecture, the authors demonstrate that it is possible to achieve high-speed perception on low-power edge devices without the prohibitive energy costs associated with heavy SOTA models. The results suggest that such compact, event-driven models are viable for advancing environmentally conscious, low-power AI systems in autonomous, mobile, and embedded computing environments. The work establishes a foundation for future extensions to more complex multi-object and multi-class event datasets, emphasizing the potential for real-time detection in power-constrained scenarios.