STAER: Temporal Aligned Rehearsal for Continual Spiking Neural Network

Technical Summary of STAER: Temporal Aligned Rehearsal for Continual Spiking Neural Network

Problem Statement
Spiking Neural Networks (SNNs) possess inherent temporal dynamics and event-driven processing capabilities that theoretically make them well-suited for Continual Learning (CL), particularly in Class-Incremental Learning (CIL) scenarios where new classes are introduced sequentially without task identifiers. However, the application of SNNs to CIL has been hindered by two primary issues: catastrophic forgetting and temporal misalignment. While standard Artificial Neural Networks (ANNs) have made progress using techniques like Experience Replay (ER) and knowledge distillation (e.g., DER, DER++), these methods often fail to address the unique nature of spike-based information. In SNNs, knowledge is encoded not just in firing rates but in the precise timing of spikes. As new tasks are learned, the temporal structure of spike patterns encoding past classes drifts, leading to interference. Standard loss functions (e.g., based on spike counts) and naive replay strategies fail to penalize these temporal shifts or recover the correct temporal dynamics, resulting in poor performance compared to ANN baselines.

Methodology
The authors propose STAER (Spiking Temporal Alignment with Experience Replay), a framework designed to explicitly preserve the temporal structure of learned representations in SNNs. The methodology integrates three core components:

1. Deep Spiking Backbone: The approach utilizes a deep ResNet19 architecture adapted for SNNs, employing Leaky Integrate-and-Fire (LIF) neurons trained via surrogate gradient descent. This ensures scalability to complex image benchmarks like Sequential-MNIST and Sequential-CIFAR10.
2. Temporal Expansion and Contraction Mechanism: Inspired by biological memory processes where recall traces can undergo compression or dilation, the method stores replay samples in the buffer with three distinct temporal resolutions: $T$ (original), $T/2$ (contraction), and $2T$ (expansion). This is achieved by replicating static input images over different time windows during the encoding phase.
3. Soft-DTW Temporal Alignment Loss: To address temporal drift, STAER introduces a differentiable alignment loss based on Soft Dynamic Time Warping (Soft-DTW). During training on a new task, the network computes a Cross-Entropy loss on current data and a Temporal Alignment ($L_{TA}$) loss on replayed data. The $L_{TA}$ loss compares the current network's output logits (at resolution $T$) against the stored logits of past models at all three resolutions ($T, T/2, 2T$). This forces the network to maintain the fidelity of its temporal responses across tasks, regardless of sampling rates.

The total objective function is defined as $L = L_{CE} + \beta \cdot L_{TA}$, where $\beta$ balances task learning and temporal consistency.

Key Contributions

• Novel Temporal Alignment Framework: The paper presents the first SNN-CL method that utilizes differentiable Soft-DTW to explicitly align spike timings between tasks. This directly targets the temporal drift that causes catastrophic forgetting in spiking domains.
• Hybrid Coding Strategy: The approach adopts a dual coding principle, jointly encoding information via firing rates (averaged logits) and spike timing (temporal alignment), aligning with biological principles of neural computation.
• Scalability to Deep Architectures: Unlike prior SNN-CL works that often rely on shallow networks, this method successfully implements continual learning on a deep ResNet19 backbone, demonstrating that deep SNNs can be effectively trained for CIL.

Experimental Results
The method was evaluated on Sequential-MNIST and Sequential-CIFAR10 benchmarks under Class-Incremental Learning protocols.

• Performance: STAER achieves state-of-the-art performance among SNN methods, reaching up to 98.48% accuracy on Sequential-MNIST and 83.53% on Sequential-CIFAR10 (with a buffer size of 5120 and $T=4$).
• Comparison to Baselines: The method consistently outperforms spiking variants of ER, DER, and DER++. Notably, STAER bridges the performance gap between SNNs and strong ANN baselines, achieving accuracy competitive with or surpassing ANN models in certain constrained settings.
• Forgetting Reduction: Empirical results show that STAER yields the lowest forgetting rates across most configurations, even outperforming ANN baselines in the most constrained settings (small buffer sizes).
• Ablation Studies: Experiments confirm that explicit temporal alignment is critical. Removing the contraction/expansion terms (setting $\alpha_1 = \alpha_2 = 0$) significantly degrades performance, validating the utility of multi-resolution temporal alignment. The method is most effective with a moderate alignment weight ($\beta = 10^{-4}$) and balanced compression/dilation weights.

Significance and Claims
The paper claims that STAER positions SNNs as viable, scalable solutions for spike-native lifelong learning. By explicitly preserving temporal structure through Soft-DTW alignment and biological-inspired temporal expansion/contraction, the framework mitigates the catastrophic forgetting and temporal interference that have previously limited SNNs in CIL. The authors assert that their work demonstrates that deep SNNs can match or outperform traditional ANN baselines while preserving biologically plausible dynamics, paving the way for future applications on real-world neuromorphic datasets. The work suggests that maintaining both rate-based and temporal information is crucial for robust adaptive behavior in spiking neural networks.