Federated Learning of Spiking Neural Networks under Heterogeneous Temporal Resolutions

Technical Summary: Federated Learning of Spiking Neural Networks under Heterogeneous Temporal Resolutions

Problem Statement
Federated Learning (FL) enables collaborative model training across decentralized edge devices while preserving data privacy. However, in time-series applications, edge devices often operate under varying hardware and energy constraints, leading to the collection of data at different temporal resolutions (sampling rates). This temporal heterogeneity poses a fundamental challenge: parameters learned from data at one temporal resolution do not necessarily transfer directly to another. Consequently, standard Federated Averaging (FedAvg), which naively aggregates model parameters, becomes ineffective because it conflates models operating at different temporal scales, potentially degrading the global model's performance. While existing literature addresses data distribution heterogeneity (e.g., class imbalance) and temporal resolution changes in non-federated Spiking Neural Networks (SNNs), there is a gap in addressing temporal resolution mismatch specifically within a federated setting for stateful deep networks and SNNs.

Methodology
The authors propose FedTA (Federated Learning with Temporal Resolution Adaptation), a framework designed to align models trained on heterogeneous temporal resolutions before aggregation.

• Framework Structure: FedTA integrates an adaptation mechanism into the standard FedAvg loop. Before local models are aggregated at the server, they are adapted from their client-specific temporal resolution ($T_k$) to a central server resolution ($T_c$). Conversely, the updated global model is adapted back from $T_c$ to each client's local resolution $T_k$ before distribution.
• Scope of Adaptation: The adaptation is applied exclusively to the dynamical parameters ($\phi$) of the neurons (e.g., membrane decay rates, state transition matrices). Synaptic weights ($W$) and normalization statistics ($\psi$) remain unaltered during this process.
• Neuron Models: The study evaluates two families of neurons:
  1. Leaky Integrate-and-Fire (LIF): A standard SNN model.
  2. Linear-Dynamics Diagonal State Space Models (LD-SSM): A non-spiking baseline with linear state transitions.
     For each family, the authors compare a Standard variant (directly training dynamical parameters) against a $\Delta$-variant, which employs an exp-log reparameterization to explicitly map the local temporal resolution to a trainable parameter ($\Delta_{log}$).
• Adaptation Algorithms: Three specific adaptation methods are integrated:
  1. FedTA-Int (Integral): Derived from linear SSM theory, this method adapts parameters using an integral transformation ($A_{r2} = A_{r1}^\rho$).
  2. FedTA-Eul (Euler): A first-order approximation derived from linear SSMs ($A_{r2} = I + \rho(A_{r1} - I)$).
  3. FedTA-$\Delta$: Specifically for the $\Delta$-variants, this method applies an additive shift to the logarithmic temporal resolution parameter ($\Delta_{log, r2} = \Delta_{log, r1} + \ln(\rho)$), where $\rho$ is the ratio of temporal resolutions.

Key Contributions

1. Novel Framework: The paper introduces the first federated learning framework explicitly addressing temporal resolution heterogeneity for SNNs and stateful deep networks.
2. Adaptation Strategies: It proposes and integrates three distinct dynamics adaptation methods (Integral, Euler, and $\Delta$-shift) to bridge the gap between client-specific and global temporal resolutions.
3. Comprehensive Evaluation: The framework is evaluated on two neuromorphic benchmark datasets (SHD and DVS-Gesture) across two client scenarios (varying numbers of clients per resolution) and multiple neuron model configurations.

Results

• Accuracy Recovery: The proposed adaptation methods substantially recover accuracy lost due to temporal mismatch. On the SHD dataset, the combination of Standard-SSM and FedTA-Int achieved the highest accuracy across all scenarios and resolutions. On the DVS-Gesture dataset, performance varied, with $\Delta$-LIF combined with FedAvg often yielding the best results, though FedTA variants generally outperformed naive FedAvg in heterogeneous settings.
• Method Comparison:
  • For linear dynamics (SSM), the Integral method (FedTA-Int) significantly outperformed the Euler method, consistent with the theoretical expectation that Euler is a less accurate first-order approximation.
  • For non-linear dynamics (LIF), both Integral and Euler methods served as approximations. The $\Delta$-variants showed mixed results, sometimes performing comparably to or slightly worse than FedAvg depending on the target resolution ($T_c$).
• Central Resolution Impact: Performance was generally highest when the central server resolution ($T_c$) matched the finest client resolution ($T_c=1$) or the intermediate resolution ($T_c=2$), minimizing average mismatch.
• Computational Overhead: The computational cost of the adaptation steps (FedTA-Int, FedTA-Eul, FedTA-$\Delta$) was negligible compared to the baseline FedAvg. Training times remained within one standard deviation of the baseline.
• Energy Efficiency: The $\Delta$-LIF with FedAvg offered a compelling trade-off, achieving high accuracy with significantly lower energy consumption and faster training times compared to SSM-based models, leveraging the sparse, binary nature of SNNs.

Significance and Claims
The paper claims that FedTA enables each client to train at its local temporal resolution while remaining compatible with a global model, effectively solving the temporal misalignment problem in federated settings. The authors highlight that:

• FedTA-Int is particularly effective for models with linear state dynamics, as the adaptation method's assumptions align with the model structure.
• LIF-based models (SNNs) demonstrate robustness across settings and offer a superior energy-efficiency-to-accuracy trade-off, making them attractive for resource-constrained edge devices.
• The framework is a promising step toward robust federated learning under temporal heterogeneity, though the authors modestly note that the adaptation methods for non-linear dynamics (like LIF) are approximations and that future work should explore tailored adaptation for non-linear neuron dynamics and interactions with other forms of data heterogeneity (e.g., class imbalance).