FeNN-DMA: A RISC-V SoC for SNN acceleration

Technical Summary: FeNN-DMA – A RISC-V SoC for SNN Acceleration

Problem Statement
Spiking Neural Networks (SNNs) offer a promising, energy-efficient alternative to standard Artificial Neural Networks (ANNs), particularly for spatio-temporal tasks like keyword spotting and video classification. However, SNNs possess low arithmetic intensity due to their event-based communication, making them poorly matched to standard high-throughput accelerators like GPUs and TPUs, which are optimized for dense matrix multiplication. While dedicated hardware such as ASICs and FPGAs has been developed for SNNs, existing FPGA-based solutions often face significant trade-offs. Many are tailored to specific, simple neuron models (e.g., Leaky Integrate-and-Fire) and image-based datasets (e.g., MNIST) converted from ANNs, failing to support the complex, recurrent, and delay-rich architectures required for state-of-the-art SNN research. Furthermore, systems supporting external memory often suffer from latency bottlenecks, while those relying solely on on-chip memory cannot scale to the neuron and synapse counts required for modern models.

Methodology
The authors present FeNN-DMA, a fully programmable System-on-Chip (SoC) based on the RISC-V architecture, tailored for simulating SNNs on AMD UltraScale+ FPGAs (specifically the Kria KV260 development kit). The design integrates a custom vector processor with a dedicated Direct Memory Access (DMA) controller to balance programmability with memory-bound performance.

• Architecture: The SoC features a tightly coupled scalar core (CV32E40x) and a 32-lane, 512-bit wide vector co-processor (VEC). The VEC is a 3-stage pipelined processor with specialized instructions for SNN operations, including masked addition, stochastic rounding, and random number generation.
• Memory Hierarchy: To address the memory-bound nature of SNNs, FeNN-DMA utilizes a hybrid memory approach. It employs high-density UltraRAM (URAM) blocks for large vector data (weights and neuron states) and BlockRAM (BRAM) for instruction memory and "lane-local" memory. The lane-local memory supports indexed loads, enabling efficient handling of sparse connectivity and synaptic delays without serializing operations across cycles.
• DMA Controller: A bespoke DMA controller manages data streaming between 4 GB of off-chip DDR4 memory and on-chip buffers. It overlaps memory transfer latency with computation, allowing the system to simulate models larger than the on-chip memory capacity.
• Software Stack: The authors developed PyFeNN, a Python-based toolchain. It allows users to define neuron models in a C-like language and SNN topologies using PyTorch-like syntax or import models from the Neuromorphic Intermediate Representation (NIR). The toolchain compiles these descriptions into optimized RISC-V vector code.
• Algorithms: The system implements optimized algorithms for dense, sparse, and delayed connectivity. For sparse connectivity, it iterates over spike bitfields and processes connections in parallel across vector lanes, utilizing the lane-local memory to distribute target neuron inputs.

Key Contributions

1. Updated FeNN Core: An enhanced vector processor with lane-local memories to support synaptic delays and sparse connectivity, along with a custom instruction set (Table 1) optimized for SNN computations (e.g., VANDADD, VRNG, VMUL with rounding).
2. Bespoke DMA Controller: A custom controller that efficiently streams weights from off-chip memory to on-chip buffers, enabling the simulation of large-scale networks (up to 256 million synapses per core) while hiding memory latency.
3. PyFeNN Framework: A high-level programming environment that bridges the gap between SNN model definition and hardware deployment, supporting NIR imports and fixed-point quantization.
4. Scalable SoC Designs: Implementation of single and dual-core FeNN-DMA systems on the Kria KV260, capable of simulating up to 16,000 neurons per core.

Results

• Performance and Resources: The single-core FeNN-DMA achieves an operating frequency of 175 MHz. In terms of resource usage (LUTs, FFs, BRAMs), it is comparable to state-of-the-art fixed-function accelerators but supports significantly more neurons and synapses. While its raw throughput (0.92 GSOP/s for dense connectivity) is lower than some fixed-function systems (which can exceed 47 GSOP/s), it offers superior flexibility and scalability.
• Classification Accuracy: The system demonstrated state-of-the-art accuracy on challenging benchmarks:
  • Spiking Heidelberg Digits (SHD): Achieved 90.32% accuracy on a recurrent network with trainable synaptic delays, outperforming other FPGA-based approaches evaluated on this dataset.
  • Neuromorphic MNIST (N-MNIST): Achieved 98.46% accuracy using an extremely sparse (99% sparsity) recurrent model, surpassing dense models from previous FPGA works.
  • Braille Tactile Classification: Achieved 92.86% accuracy using a NIR-imported model, comparable to the best software backends.
• Scaling: The system can simulate balanced random networks of up to 16,000 neurons at faster-than-real-time speeds when using sparse connectivity. The measured throughput for sparse processing reached 3.72 GSOP/s (with 75% sparsity), effectively compensating for the lower per-operation efficiency compared to fixed-function hardware.

Significance and Claims
The paper claims that FeNN-DMA represents a significant step toward flexible, high-capacity SNN acceleration on FPGAs. By combining a RISC-V vector architecture with a custom DMA controller and a high-level software stack, the system bridges the gap between the programmability of general-purpose processors and the efficiency of dedicated accelerators.

The authors emphasize that their approach allows for the simulation of complex, biologically plausible models (featuring recurrent connectivity and delays) that are often unsupported by fixed-function accelerators. While acknowledging a throughput gap of 3–4× compared to fixed-function systems due to the software-driven spike propagation loop, they argue that the system's ability to stream from external memory and support large, complex models makes it a practical solution for state-of-the-art SNN inference. The work demonstrates that programmable vector architectures on FPGAs can achieve competitive energy efficiency and resource usage while offering the flexibility required for next-generation neuromorphic research.