💻 computer science

SiLIF: Structured State Space Model Dynamics and Parametrization for Spiking Neural Networks

This paper introduces SiLIF, a novel spiking neuron model inspired by structured state space models that leverages learnable discretization, logarithmic reparametrization, and complex-state initialization to achieve stable training, state-of-the-art performance on speech recognition tasks, and superior computational efficiency compared to existing SSMs.

Original authors: Maxime Fabre, Lyubov Dudchenko, Emre Neftci

Published 2026-02-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Maxime Fabre, Lyubov Dudchenko, Emre Neftci

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to recognize spoken words, like "yes," "no," or "stop."

Traditionally, we use Artificial Neural Networks (ANNs). Think of these like a massive, noisy factory floor where every single worker (neuron) is constantly shouting numbers to every other worker, all the time. It's powerful, but it's loud, energy-hungry, and slow.

Then, we have Spiking Neural Networks (SNNs). These are more like a biological brain. Instead of constant shouting, neurons only "fire" a tiny electrical spark (a spike) when they have something important to say. This makes them incredibly efficient and low-power, perfect for battery-operated devices. However, teaching these networks has been a nightmare. Because they are so sparse (quiet most of the time), the "teacher" (the training algorithm) often gets lost, leading to unstable learning where the network forgets everything or goes crazy.

On the other side of the room, we have a new, very smart tool called State Space Models (SSMs). Think of SSMs as a master librarian who can remember a whole book just by looking at a single page. They are amazing at understanding long sequences of information (like long sentences or music) and are very stable. But, they are usually "analog" (continuous numbers), not "digital" (spikes), so they don't get the energy efficiency of the biological brain.

The Big Idea: SiLIF

The authors of this paper asked: "What if we could build a Spiking Neuron that thinks like a Master Librarian?"

They created two new types of neurons called SiLIF and C-SiLIF. Here is how they did it, using some simple analogies:

1. The "Smart Clock" (Learnable Timestep)

In old spiking networks, the "clock" that ticks the neurons forward was fixed. It was like a metronome set to a speed that might be too fast for a slow song or too slow for a fast one.

The Fix: The SiLIF neuron has a smart, adjustable clock. It learns to speed up or slow down its own internal ticking depending on the task. This allows it to catch fast sounds and slow sounds equally well.

2. The "Logarithmic Safety Net" (Reparametrization)

Training these networks is like walking a tightrope. If you step too far left, the numbers get too small (vanish); too far right, they explode.

The Fix: The authors used a mathematical trick called logarithmic reparametrization. Imagine instead of walking on a flat tightrope, you are walking inside a deep, curved valley. No matter how much you stumble, the walls of the valley gently guide you back to the center. This keeps the training stable and prevents the network from crashing.

3. The "Oscillating Pendulum" (Complex States)

Some sounds, like a musical note or a vowel, vibrate. Old spiking neurons were mostly good at simple "add and subtract" math. They struggled with these vibrations.

The Fix: The C-SiLIF neuron uses "complex numbers" (a bit like having a 3D view instead of a 2D one). This allows the neuron to act like a swinging pendulum. It can naturally resonate with the rhythm of the sound, making it much better at recognizing speech patterns.

The Results: The Best of Both Worlds

The authors tested these new neurons on speech recognition tasks (like Google's Speech Commands).

Accuracy: They smashed the previous records. They recognized words better than any other spiking network before.
Efficiency: Here is the magic trick. They compared their new spiking neurons to the "Master Librarian" (the SSMs).
- The SSMs were accurate but required a lot of heavy lifting (computational power).
- The SiLIF neurons were just as accurate (or even better) but used half the energy and computing power.
- It's like the SiLIF neuron is a sprinter who runs as fast as a marathon runner (the SSM) but uses a fraction of the calories.

Why Does This Matter?

Currently, our phones and smartwatches rely on big, energy-hungry servers to understand our voice commands. This new technology means we could put a "Master Librarian" brain directly onto a tiny, battery-powered chip.

No more waiting: Faster recognition.
No more draining batteries: Devices could listen to you all day without dying.
Privacy: Since the processing happens on the device, your voice doesn't need to be sent to the cloud.

In short, the authors took the "brain-like" efficiency of spiking neurons and gave them the "super-memory" and "stability" of modern AI models. They built a neuron that is quiet, efficient, stable, and incredibly smart.

1. Problem Statement

Spiking Neural Networks (SNNs) offer a promising low-power alternative to traditional Artificial Neural Networks (ANNs), particularly for event-based and neuromorphic hardware. However, training SNNs using Backpropagation Through Time (BPTT) faces significant challenges:

Instability: Gradient propagation through binary spiking dynamics often leads to vanishing or exploding gradients.
Scalability Issues: Standard spiking neuron models (like Leaky Integrate-and-Fire) struggle to capture complex temporal dependencies required for long sequences or high-performance tasks.
Performance Gap: While State Space Models (SSMs) like S4 have achieved state-of-the-art (SOTA) results in long-sequence modeling due to their stable linear recurrent dynamics, they lack the sparsity and energy efficiency of SNNs. Conversely, existing SNN models often lag behind SSMs in accuracy on audio tasks.

The core problem is bridging the gap between the training stability and performance of SSMs and the computational efficiency and sparsity of SNNs.

2. Methodology

The authors propose two novel neuron models, SiLIF and C-SiLIF, which integrate the structural and parametric insights of SSMs (specifically the S4 and S4D models) into reset-based spiking neurons.

A. Theoretical Bridge

The authors establish a mathematical link between the Adaptive Leaky Integrate-and-Fire (AdLIF) neuron and linear SSMs. They demonstrate that the subthreshold dynamics of AdLIF can be mapped to a linear SSM block, with the spike-triggered reset mechanism providing the necessary nonlinearity.

B. SiLIF Model (SSM-Inspired LIF)

The SiLIF model enhances the AdLIF neuron by adopting S4-inspired training strategies:

Learnable Discretization Timestep ( $\Delta t$ ): Unlike previous works where $\Delta t$ is fixed, SiLIF treats $\Delta t$ as a learnable parameter for each neuron. This allows the model to adapt to diverse transition dynamics regimes.
Logarithmic Reparametrization: Instead of training decay constants ( $\alpha, \beta$ ) directly, the model trains their logarithmic counterparts ( $\log \lambda$ ) and $\log \Delta t$ . This ensures parameters remain strictly positive and prevents vanishing/exploding gradients, improving optimization smoothness.
Continuous-to-Discrete Mapping: The model trains continuous-domain state-space variables ( $\lambda = \tau^{-1}$ ) and converts them to discrete decay factors via exponential functions ( $\alpha = e^{-\lambda \Delta t}$ ), mirroring the S4 discretization process.

C. C-SiLIF Model (Complex-Valued SSM-Inspired LIF)

The C-SiLIF model extends SiLIF by incorporating complex-valued dynamics, drawing parallels to the Resonate-and-Fire (RF) neuron and the S4D (Diagonal SSM) model:

Complex State Representation: It utilizes a complex state $x = x_{real} + i x_{imag}$ , which corresponds to a structured $2 \times 2$ real-valued system with antisymmetric properties. This enables oscillatory regimes (resonance) that real-valued models often struggle to learn.
S4D Initialization: It employs the S4D-Lin initialization scheme (derived from orthogonal polynomial projections) to ensure robust state dynamics from the start of training.
Reset Mechanism: A specific "half-reset" mechanism is introduced where the spike feedback is scaled (fed back as $0.5 \cdot s$ ) to maintain consistency with the complex state projection.

D. Synaptic Delays

The authors also implement a variant with learnable synaptic delays ("SiLIF + Delays") using a DCLS (Dilated Convolution with Learnable Spacings) framework. This allows the network to model temporal shifts in spike propagation, further enhancing temporal processing capabilities.

3. Key Contributions

Theoretical Unification: Established a rigorous theoretical bridge between SSMs and SNNs, revealing shared training dynamics and specific differences (e.g., the role of the reset mechanism).
Novel Neuron Models: Proposed SiLIF and C-SiLIF, the first reset-based spiking neurons to successfully integrate S4/S4D parametrization, logarithmic reparametrization, and learnable timesteps.
SOTA Performance: Achieved new state-of-the-art accuracy on multiple audio keyword classification benchmarks (SHD, SSC, GSC), outperforming previous reset-based SNNs and even standard SSMs in specific efficiency contexts.
Efficiency Breakthrough: Demonstrated that SiLIF models with synaptic delays can surpass standard SSMs (like S4D and Mamba) in accuracy while requiring half the computational cost (Synaptic Operations).

4. Experimental Results

The models were evaluated on three standard datasets:

SHD (Spiking Heidelberg Digits): Event-based German/English digits.
SSC (Spiking Speech Commands): Event-based version of Google Speech Commands.
GSC (Google Speech Commands): Raw audio commands.

Key Findings:

Accuracy:
- SiLIF achieved 82.03% on SSC (4ms bins) and 95.49% on GSC, setting new SOTA records for non-delay SNNs.
- C-SiLIF reached 81.59% on SSC.
- SiLIF + Delays achieved 95.88% on GSC, outperforming all other SNNs and matching/exceeding SSM baselines.
Efficiency (Accuracy vs. SOPs):
- On the GSC task, SiLIF + Delays outperformed S4D and SpikingSSM while using 50% fewer Synaptic Operations (SOPs).
- The standard SiLIF model was Pareto-optimal, achieving top-tier accuracy at roughly 1/3 the computational budget of S4D.
Ablation Studies:
- Removing logarithmic reparametrization caused a >3% accuracy drop.
- Using a fixed, homogeneous timestep caused a >6% drop, proving the necessity of learnable, heterogeneous timesteps.
- The complex initialization (S4D-Lin) significantly boosted performance over standard random initialization.

5. Significance

This work represents a major step forward in neuromorphic computing and edge AI:

Bridging Paradigms: It successfully transfers the stability and performance benefits of modern deep learning (SSMs) to the sparse, event-driven domain of SNNs without sacrificing the biological plausibility of reset mechanisms.
Hardware Efficiency: By achieving SOTA accuracy with significantly fewer operations than dense SSMs, SiLIF models are highly suitable for deployment on low-power, event-based neuromorphic hardware (e.g., Loihi, TrueNorth).
Dynamical Richness: The introduction of complex-valued dynamics and learnable timesteps allows SNNs to explore a wider range of neuronal regimes (including oscillations), solving tasks that were previously difficult for standard spiking neurons.

In conclusion, SiLIF and C-SiLIF demonstrate that by rethinking neuron parametrization through the lens of Structured State Space Models, SNNs can overcome their historical training instabilities and compete with, or even surpass, the most efficient deep learning architectures in terms of the accuracy-efficiency trade-off.