💻 computer science

Hardware-aware Graph Neural Networks prunning for embedded event-based vision

This paper proposes a hardware-aware pruning and quantization strategy for Graph Convolutional Neural Networks that optimizes model architecture for embedded FPGA platforms, achieving significant BRAM memory reductions with minimal accuracy loss across various event-based vision datasets.

Original authors: Piotr Wzorek, Kamil Jeziorek, Tomasz Kryjak

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Piotr Wzorek, Kamil Jeziorek, Tomasz Kryjak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Making "Event Cameras" Fit in a Backpack

Imagine you are trying to pack a giant, high-tech library of books (a complex AI brain) into a tiny backpack (an embedded computer on a drone or robot). The books are heavy, and the backpack has a strict weight limit. If you try to shove the whole library in, the backpack rips, or the robot can't move.

This paper solves that problem for a specific type of camera called an Event Camera.

The Camera: Unlike normal cameras that take a full photo every second (like a video camera), an event camera is like a room full of people who only speak up when something changes. If a bird flies across the room, they shout "Bird!" If a light flickers, they shout "Light!" If nothing moves, they stay silent. This creates a "sparse" stream of data—lots of silence, but very fast and efficient when things happen.
The Problem: To understand this stream of shouts, we use a special type of AI called a Graph Neural Network (GCNN). Think of this network as a massive web of connections. However, the "web" requires a lot of internal storage (memory) to keep track of the connections. On small, battery-powered robots, this storage is the biggest bottleneck. It's like trying to carry the whole library in your backpack when you only have room for a few pamphlets.

The Solution: A Smart Packing Strategy

The authors (Piotr, Kamil, and Tomasz) developed a new way to shrink this AI brain so it fits in the backpack without losing its ability to recognize objects. They call this "Hardware-Aware Pruning and Quantization."

Here is how they did it, broken down into three simple steps:

1. The "Pruning" (Cutting the Fat)

Imagine the AI network is a tree with thousands of branches. Some branches are thick and carry a lot of important information; others are tiny twigs that barely do anything.

What they did: They didn't just randomly chop off branches. They looked at the specific "backpack" (the FPGA chip) they were packing for. They realized that the backpack has specific-sized shelves (memory blocks).
The Trick: They cut the branches in a very specific pattern so that the remaining tree fits perfectly onto the shelves, leaving no empty space wasted. This is called Structured Pruning.

2. The "Quantization" (Switching to Smaller Boxes)

Usually, the AI stores information using very large, heavy boxes (high-precision numbers).

What they did: They decided to switch to smaller, lighter boxes (lower precision numbers, like 6-bit or 8-bit instead of 32-bit).
The Result: You can fit way more information into the same amount of space, but the boxes are slightly less precise. It's like switching from storing items in heavy wooden crates to lightweight cardboard boxes. You save space, but you have to be careful not to drop them.

3. The "Search" (Finding the Perfect Fit)

The hardest part is figuring out which branches to cut and how small the boxes should be. There are millions of possible combinations.

The Strategy: They used two clever search methods:
- Fine Grid Search (FG): Like looking at a map and checking every major intersection to find the best starting route.
- Greedy Layer-wise Iterative Deepening (GLID): Like a hiker who takes a step down a path, checks if the view is still good, and if it is, takes another step. If the view gets too bad, they stop. This allows them to find the "sweet spot" where the backpack is as light as possible, but the AI is still smart enough to do its job.

The Results: Lighter Backpack, Still Smart

They tested this method on three different "libraries" of data (datasets):

MNIST-DVS: Recognizing handwritten digits.
CIFAR-10: Recognizing everyday objects like cars and airplanes.
N-Caltech101: Recognizing more complex objects.

The Outcome:

Memory Savings: They managed to reduce the memory usage (specifically the "BRAM" or Block RAM, which is the internal storage of the chip) by 26.5% to 31.4%.
Accuracy Cost: The AI got slightly "dumber," but only by a little bit. The accuracy dropped by between 1.65% and 5.18%.
Real-World Test: They actually built a physical version of this on a real chip (the ZCU104 platform). It worked perfectly, running at high speed with no errors, proving that the "lighter backpack" strategy works in real life.

The Takeaway

Think of this paper as a master packer who figured out how to fit a massive library into a small backpack by:

Cutting out unnecessary books (Pruning).
Replacing heavy hardcovers with paperbacks (Quantization).
Using a smart map to find the exact combination that fits the shelves perfectly (The Search Methods).

The result is a robot brain that is much lighter and faster, perfect for drones and mobile robots, without losing the ability to see and understand the world around it.

Technical Summary: Hardware-aware Graph Neural Networks Pruning for Embedded Event-based Vision

Problem Statement

Mobile robotics and autonomous systems require vision sensors capable of operating with high performance in dynamic environments while adhering to strict constraints on latency, throughput, and energy consumption. Event-based cameras (Dynamic Vision Sensors or DVS) address these needs by capturing scene changes asynchronously with microsecond resolution, low latency, and high dynamic range. However, processing the resulting sparse, time-series data efficiently on embedded platforms remains a significant challenge.

While Spiking Neural Networks (SNNs) preserve temporal sparsity, they often suffer from lower accuracy and pose challenges for hardware acceleration on non-neuromorphic platforms due to non-deterministic memory access patterns. Conversely, Graph Convolutional Neural Networks (GCNNs), which treat events as graph nodes, offer high accuracy and are well-suited for hardware acceleration. Yet, deploying GCNNs on resource-constrained embedded heterogeneous FPGA platforms is hindered by the high utilization of on-chip memory (BlockRAM and UltraRAM), particularly for storing feature maps between layers. When external DRAM is not feasible due to latency or energy constraints, the on-chip memory becomes the primary bottleneck, limiting the deployment of larger, more accurate models.

Methodology

The authors propose a hardware-aware optimization strategy specifically designed for the Event-based FPGA-accelerated Graph Convolutional Network (EFGCN) architecture. The core objective is to minimize the utilization of internal FPGA memory resources (specifically dual-port 36kb BlockRAM and 288kb UltraRAM) while maintaining high classification accuracy.

The methodology integrates two primary techniques:

Structured Pruning: Unlike unstructured pruning, this approach removes entire channels (output feature maps) based on their position in the weight matrix. The selection of channels is constrained by the mathematical properties of the target FPGA memory blocks. The authors derive specific families of valid channel counts ( $C$ ) that ensure full utilization of BRAM and URAM blocks given a specific quantization precision ( $q$ ). This avoids memory fragmentation and ensures efficient resource packing.
Quantization: The method reduces the precision of feature vectors and edge information stored in memory. By lowering the bit-width ( $q$ ), the memory width required per channel decreases, allowing for either a reduction in total memory usage or an increase in channel capacity within the same memory footprint.

To navigate the vast design space (exceeding $7.8 \times 10^6$ possible configurations), the authors employ a two-stage search procedure:

Fine Grid Search (FG): Starting from a baseline configuration (maximum channels, 8-bit quantization), the algorithm performs a downward search across layers and quantization levels. It evaluates configurations based on a Pareto frontier, balancing classification quality against resource utilization to identify a promising subset of solutions.
Greedy Layer-wise Iterative Deepening Search (GLID): This stage further refines the selected configurations. It iteratively reduces resource utilization layer-by-layer, selecting the modification (pruning or quantization step) that yields the best quality score for the least resource cost. The process terminates when the accuracy drop exceeds a predefined threshold.

Following the search, the selected models undergo a fine-tuning phase with a low learning rate to recover accuracy lost during compression.

Key Contributions

The paper claims the following primary contributions:

First Hardware-Aware Strategy for Event GCNNs: The authors present the first method combining hardware-aware pruning and quantization specifically for event-based vision and Graph Convolutional Neural Networks.
Flexible Architecture Adaptation: The proposed method enables the flexible selection of a final model configuration based on the specific resource constraints and accuracy requirements of the target embedded platform.
Validated Performance: The approach was evaluated across multiple datasets and network configurations, demonstrating significant reductions in BRAM usage (26.5%–31.4%) with controlled accuracy degradation (1.65%–5.18%).
Hardware Proof-of-Concept: A hardware module was developed and verified on a ZCU104 heterogeneous FPGA platform, confirming the effectiveness of the pruning strategy in a real-world implementation.

Experimental Results

The method was evaluated using three neuromorphic datasets: MNIST-DVS, CIFAR-10-DVS, and N-Caltech101. The experiments utilized a baseline EFGCN model with five convolutional layers.

Memory Reduction: The optimization strategy achieved BRAM reductions of 28.8% for CIFAR-10 (with a 1.65% accuracy drop), 31.4% for MNIST-DVS (3.55% accuracy drop), and 26.5% for N-Caltech101 (5.18% accuracy drop).
Hardware Implementation: On the ZCU104 platform, the final optimized model for MNIST-DVS demonstrated a 30.6% reduction in total BRAM usage compared to the baseline. Additionally, logic resource utilization (LUTs and Flip-Flops) decreased by approximately 34% and 29%, respectively. Notably, DSP utilization increased by 54% due to the synthesis tool mapping certain 6-bit multiplications to DSP blocks, though the overall memory savings were the primary target.
Search Efficiency: The two-stage search (FG followed by GLID) proved effective in finding high-quality configurations without the computational cost of exhaustively searching the entire design space or using complex evolutionary algorithms (which the authors noted failed to converge in their preliminary tests).

Significance and Claims

The paper positions this work as a critical step toward enabling efficient, autonomous mobile robotics. By addressing the memory bottleneck inherent in FPGA-based GCNN implementations, the authors enable the deployment of complex event-based vision models on embedded systems without relying on external memory, which would introduce unacceptable latency and power overhead.

The authors modestly frame their contribution as a "first exploration" of this specific intersection (hardware-aware pruning for event-based GCNNs). They emphasize that the method allows for a trade-off between accuracy and resource usage, providing a tool for system designers to tailor models to specific hardware constraints. The work does not claim to solve all challenges in event-based vision but specifically targets the optimization of the EFGCN architecture for embedded heterogeneous platforms, offering a practical pathway to deploy high-performance, low-latency vision systems on drones and other mobile robots.