Hardware-Accelerated Event-Graph Neural Networks for Low-Latency Time-Series Classification on SoC FPGA

Technical Summary: Hardware-Accelerated Event-Graph Neural Networks for Low-Latency Time-Series Classification on SoC FPGA

Problem Statement
As the volume of data generated by embedded edge sensors increases, there is a critical need for intelligent local processing capable of handling time-series signals with low latency and low power consumption. While Artificial Intelligence (AI) offers solutions for real-time prediction, conventional hardware like GPUs consumes excessive power, and microprocessors often fail to meet strict latency requirements. Although Field-Programmable Gate Arrays (FPGAs) offer a path to optimized architectures, existing event-based AI approaches, particularly Spiking Neural Networks (SNNs), face challenges in fully exploiting data sparsity due to the non-deterministic patterns of synaptic weight-memory access. Furthermore, while event-graph neural networks have shown promise in software for time-series data, a dedicated hardware architecture for event-graph audio processing on System-on-Chip (SoC) FPGAs has not yet been realized.

Methodology
The authors propose a hardware-software co-design of an event-graph neural network (EGNN) implemented on a SoC FPGA. The system processes data from an artificial cochlea (AC) model, which converts continuous time-series audio into sparse event data. The methodology involves three primary stages:

1. Event-Graph Generation:

   • Input: Asynchronous events defined by channel index and timestamp.
   • Optimization: To address the high computational complexity of the baseline software model (which required fitting local surfaces to estimate normal vectors), the authors introduced a novel graph generation method.
   • Skip Step Connection: Instead of scanning all possible neighbors, the system employs a deterministic "skip step" pattern to form edges between vertices. This reduces memory overhead and latency.
   • Feature Simplification: Event-normal vectors are replaced with simpler features based on the average timestamp and channel coordinates of neighboring events.
   • Memory Architecture: Events are stored in 1D context memory (Block RAM) indexed by channel, continuously overwriting timestamps to maintain a sliding window of the most recent events.

2. Graph Convolution:

   • The system utilizes PointNetConv layers adapted for hardware.
   • Normalization: A Positional Normalization (PN) step is introduced to rescale position differences (channel and time) back to the (0, 1) range, addressing precision issues during quantization.
   • Processing: The convolution is executed in a fully asynchronous, event-by-event manner. The architecture uses dual-port BRAMs to store event features and weights, allowing for parallel vector multiplication.
   • Quantization: The model employs quantization-aware training. Feature maps and weights are stored as unsigned integers, with scaling performed via DSP multiplication and bit-shifting.

3. System Integration:

   • The design is partitioned between the Processing System (PS) and Programmable Logic (PL) of the SoC FPGA.
   • The PL handles the graph generation and convolution.
   • The PS (running on the ARM core) handles the final classification head (Multi-Layer Perceptron) and manages data flow, including reading test data from an SD card and simulating event timestamps.

Key Contributions

• First Embedded Event-Graph Audio System: The paper presents the first embedded system for event-graph-based audio processing on a SoC FPGA.
• Fully Asynchronous Processing: The implementation supports fully asynchronous, event-by-event processing, preserving the temporal sparsity of the data.
• Hardware-Aware Optimizations: The authors propose specific adaptations, including the "skip step" graph generation method and simplified feature extraction, to enable low-power, low-latency, and low-resource utilization on reconfigurable hardware.
• Benchmarking: The system is evaluated against the Spiking Heidelberg Digits (SHD) dataset, a standard benchmark for time-series event data.

Results
The proposed system was evaluated on the SHD dataset (8156 training, 2264 test samples) and compared against state-of-the-art software models and existing FPGA-based SNN implementations.

• Accuracy:
  • The base model achieved 92.74% floating-point accuracy and 92.30% quantized accuracy on the SHD test set.
  • This performance is within 2.4% and 2% of state-of-the-art models with over 10× and 67× more parameters, respectively.
  • The quantized model outperformed previous FPGA-based SNN implementations by 19.3% (compared to Carpegna et al.) and 4.5% (compared to Matinizadeh et al.).
• Efficiency and Resources:
  • The base model uses only 18.9k parameters, significantly fewer than competing models (e.g., 2.1M or 3.9M parameters in other works).
  • The "tiny" model variant achieved comparable accuracy to previous state-of-the-art while utilizing only 23% of LUTs and 37% of BRAMs compared to the Matinizadeh et al. implementation.
  • Latency: The per-event latency for the graph generation and convolution modules is 8 µs (at 200 MHz), with a total system latency (including PS classification) of 179 µs after the last event. This contrasts with SNN approaches that often require processing entire data samples after encoding.
  • Power: The total power consumption for the base architecture (including PS) is 3.94 W (1.20 W for PL, 2.74 W for PS).

Significance and Claims
The paper claims to demonstrate that event-graph neural networks are highly suitable for event-data processing, offering a viable alternative to SNNs for hardware implementation. The authors assert that their approach successfully exploits the inherent sparsity of event-data to reduce computational complexity and latency without sacrificing accuracy.

The significance of the work lies in:

1. Performance: Establishing a new state-of-the-art for FPGA solutions on the SHD benchmark, achieving higher accuracy with significantly fewer resources and lower latency than previous SNN-based approaches.
2. Scalability: Showing that the event-graph approach scales effectively, with larger models (272k parameters) achieving 94.45% accuracy, outperforming many larger SNN models.
3. Hardware Feasibility: Proving that the deterministic weight access patterns of event-graphs allow for efficient hardware acceleration that truly leverages data sparsity, overcoming the bottlenecks associated with SNN memory access.

The authors conclude that their hardware implementation is both highly efficient and capable of superior task accuracy compared to SNN alternatives, making it a promising candidate for near-sensor AI processing at the edge. Future work is identified as integrating real event sensors and extending the system to handle continuous streams of multiple words rather than isolated samples.