🤖 machine learning

Benchmarking Fairness in Spiking Neural Networks: Data Bias, Spurious Features, and Hardware Effects

This paper introduces the first systematic fairness benchmark for Spiking Neural Networks that evaluates the impact of data bias, spurious features, and hardware constraints, revealing that existing models suffer significant performance disparities for underrepresented groups under resource limitations and highlighting the need for co-design principles to ensure trustworthy deployment.

Original authors: Hudi He, Fukun Wang, Zhe Wang, Xinyi Wang, Shuhan Ye, Jiarui Liu, Qing Qing, Ziqi Xu, Xikun Zhang, Renqiang Luo

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hudi He, Fukun Wang, Zhe Wang, Xinyi Wang, Shuhan Ye, Jiarui Liu, Qing Qing, Ziqi Xu, Xikun Zhang, Renqiang Luo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a team of tiny, ultra-fast robots to help sort people into groups (like identifying who is who in a crowd). These robots are special: instead of thinking in continuous thoughts like humans or standard computers, they communicate using tiny, rapid "sparks" or electrical pulses, much like a biological brain. These are called Spiking Neural Networks (SNNs).

The paper you provided is like a report card for these robots, but with a twist. Instead of just asking, "How smart are they?" the authors asked, "Are they fair?"

Here is the breakdown of their findings using simple analogies:

1. The Problem: The "Shortcut" Trap

Imagine you are teaching a robot to recognize different types of fruit. If you show it mostly red apples and green pears, it might learn a "shortcut": "If it's red, it's an apple." It doesn't actually learn what an apple looks like; it just learns the color.

The paper found that these SNN robots are obsessed with shortcuts.

The Bias: When looking at faces, the robots often ignore the actual shape of the face (the geometry) and instead focus entirely on skin tone or color brightness.
The Result: If the lighting changes, or if the skin tone is slightly different from what the robot "expects," it gets confused. It's like a robot that can only recognize people if they are wearing a specific color shirt. If you take away the color (turn the image black and white), the robot often fails completely, misidentifying people based on how bright or dark their skin looks rather than who they actually are.

2. The New Benchmark: A "Fairness Gym"

Before this paper, there was no standard way to test if these robots were fair. The authors built the first "Fairness Gym" for SNNs.

The Equipment: They used four different "training grounds" (datasets) featuring people of various races and genders.
The Stress Test: They didn't just test the robots in a perfect lab. They tested them under "real-world" conditions, like:
- Data Bias: What happens if the training data is unbalanced?
- Hardware Limits: What happens if the robot is running on a tiny, low-power chip (like in a smartwatch) instead of a giant supercomputer?
- Spurious Features: What happens if the robot relies on fake clues (like background color) instead of real clues?

3. The Shocking Results

When they ran 12 different types of these robots through the gym, they found some scary disparities:

The "Underrepresented" Penalty: The robots were much worse at recognizing people from underrepresented groups. For example, on some tests, the robots made 23% more mistakes (false alarms) on certain groups compared to others.
The Hardware Hit: This is the most unique finding. When they simulated running these robots on tiny, low-power edge devices (like a camera on a drone), the unfairness got much worse. The accuracy gap between groups grew by up to 41%.
- Analogy: Imagine a runner who is already tired. If you put a heavy backpack on them (the hardware limitation), they don't just run slower; they stumble over their own feet much more often than the other runners. The "backpack" of low-power hardware makes the bias explode.

4. The "Explosive" Learning Curve

The authors watched the robots learn second-by-second and found something strange called "Asymmetric Convergence."

The Scenario: Imagine a race where one runner (representing a specific group, like African faces in their study) sprints to the finish line in the first second. Another runner (representing a different group, like Indian faces) is still tying their shoes.
The Reality: The robots learned to recognize the "dominant" group almost instantly because those faces had high-contrast features (like dark skin) that triggered the robot's "sparks" immediately. The "minority" groups had to wait for the robot to slowly figure them out, and often, the robot never fully caught up.
The Takeaway: The bias isn't something that happens at the end of training; it happens in the very first second of learning. The robot gets "stuck" on the easy shortcuts immediately.

5. The Conclusion: We Need a New Rulebook

The paper concludes that we cannot just take the fairness rules we made for standard computers and apply them to these "spiking" robots.

The Old Way: "Train the robot to be accurate, then check if it's fair later."
The New Reality: Because these robots rely on sparks and timing, being "accurate" often means being "unfair." If you try to fix the fairness later, the robot might break.
The Solution: We need to design the robots and the fairness rules together from the start. We need to build "fairness" into the hardware and the code simultaneously, ensuring that the robot doesn't just rely on the easiest, most biased shortcuts.

In short: These super-efficient, brain-like robots are currently very good at being fast, but they are terrible at being fair. They rely too much on color and brightness, and when you put them on small, real-world devices, they become even more biased against certain groups of people. We need to redesign them to look at the whole picture, not just the easy clues.

Technical Summary: Benchmarking Fairness in Spiking Neural Networks

Problem Statement

While Spiking Neural Networks (SNNs) offer significant advantages in energy efficiency and low-power edge computing through their event-driven, binary spatiotemporal dynamics, their deployment in human-centric applications (e.g., facial analysis, autonomous perception) raises critical concerns regarding algorithmic fairness. Existing fairness research and toolkits (e.g., OxonFair) are designed for continuous Artificial Neural Networks (ANNs) and fail to account for the unique mechanisms of SNNs. Specifically, current assessments are limited by:

Superficial Dataset Diversity: Reliance on static datasets that do not reflect real-world demographic complexities.
Idealized Hardware Assumptions: Neglecting the impact of neuromorphic hardware constraints (e.g., reduced spike precision, edge device limitations) on algorithmic equity.
Mechanism Blindness: A lack of understanding of how SNN-specific phenomena, such as temporal leakage and binary spike generation, intrinsically alter the fairness landscape, potentially exacerbating biases through "shortcut learning" on spurious features like skin tone.

There is an urgent need for a specialized benchmark that bridges algorithmic fairness research with neuromorphic engineering to evaluate fairness under realistic resource constraints.

Methodology

The authors propose the first systematic fairness benchmark for SNNs, structured around three dimensions of realism:

Demographic Coverage Gaps: Evaluating fundamental data-level bias.
Spurious Feature Leakage: Assessing how asymmetric information filtering in SNNs triggers texture-based shortcut learning.
Deployment-Environment Mismatches: Quantifying the impact of hardware constraints on algorithmic equity.

Framework and Datasets

The benchmark integrates four cross-demographic datasets:

RFW (Racial Faces in-the-Wild): For identity verification across four racial groups.
UTKFace, FairFace, and DemogPairs: For attribute classification (race, gender) with varying degrees of balance and scale.

The study evaluates 12 state-of-the-art SNN architectures, categorized into:

Residual/Direct-Training Baselines: SEW-ResNet, SNN-ResNet18/19, VGGSNN.
Spiking Transformers: Spikformer, Spike-driven Transformer (SdT) variants, QKFormer.
Attention Enhancement: STAtten.

Evaluation Metrics and Protocols

Fairness Metrics: The study utilizes Statistical Parity (SP) and Equal Opportunity (EO). For single-label datasets like RFW, where the sensitive attribute and label are isomorphic, the authors employ a performance-based strategy measuring the maximum accuracy gap ( $\Delta Acc$ ) across demographic subgroups.
Hardware Simulation: The framework incorporates three neuromorphic hardware simulators (including Loihi 2 and SpiNNaker) to test models under constrained spike encoding.
Training Dynamics: The authors conduct epoch-by-epoch analysis (up to 150 epochs) to track convergence trajectories and identify "asymmetric learning" where specific groups converge prematurely while others lag.
Controlled Experiments: Experiments include data cleaning (removing black-and-white/abnormal-color images) and grayscale conversion to isolate the impact of color cues on SNN decision boundaries.

Key Contributions

Standardized Benchmark: The introduction of a unified framework evaluating 12 SNN models across four diverse datasets using SP, EO, and accuracy difference metrics.
Bias Source Identification: Through controlled data cleaning, the authors demonstrate that SNNs rely heavily on color and texture cues (e.g., skin tone) rather than robust geometric features. This reliance leads to systematic misclassification when these cues are ambiguous or degraded (e.g., in grayscale).
Asymmetric Convergence Discovery: The study reveals "Asymmetric Learning Trajectories," where models exhibit "explosive saturation" for dominant groups (e.g., African subsets reaching high accuracy in the first epoch) while disadvantaged groups (e.g., Indian subsets) suffer from "gradient starvation" and slow convergence. This suggests bias is solidified during initial parameter updates, not just late-stage overfitting.
Hardware-Fairness Trade-off: The benchmark demonstrates that hardware limitations, such as reduced spike precision, do not merely reduce overall accuracy but significantly amplify performance gaps across demographic groups. Strategies effective in cloud-based environments often degrade under edge constraints.

Empirical Results

Accuracy-Fairness Trade-off: A pervasive conflict exists between maximizing accuracy and satisfying fairness constraints. For instance, on UTKFace, the highest-accuracy model (SdT, 93.24%) exhibited a significant $\Delta SP$ (14.27%), while models with better parity suffered accuracy collapses.
Architectural Variance: While some models (e.g., STAtten) achieved Pareto-optimal results on specific datasets (DemogPairs), no single model was universally robust. SEW-ResNet, for example, performed well on UTKFace but degraded sharply on FairFace and DemogPairs.
Microscopic Failure Modes: Qualitative analysis of QKFormer on RFW revealed severe racial bias, with African accuracy at 86.38% versus 32.46% for Caucasians. Misclassifications were strongly correlated with grayscale conversion and color distortion, confirming that SNNs erroneously associate low color saturation with specific racial features.
Training Dynamics: In the first epoch (E0), African subsets in SRN19 (T) reached 92.89% accuracy, while Indian subsets lagged at 24.34%. This "zero-sum destruction" indicates that overfitting to dominant shortcuts actively overwrites fragile representations of minority groups.

Significance and Claims

The paper claims to provide the first systematic foundation for evaluating fairness in neuromorphic systems. By formalizing the interplay between algorithmic bias and hardware constraints, the work challenges the adequacy of existing SNN evaluations that often overlook these factors.

The authors argue that bias mitigation strategies developed for idealized, unconstrained cloud environments are insufficient for edge deployments. Consequently, they advocate for co-design principles that jointly optimize fairness and hardware efficiency. The benchmark enables the transition of fairness metrics from post-hoc evaluations to actionable optimization targets, suggesting that future neuromorphic models must incorporate constrained optimization paradigms (e.g., integrating SP/EO directly into loss functions) to prevent the premature maturation of spurious features.

The work positions itself as a critical step toward trustworthy SNNs in socially critical domains such as healthcare and autonomous systems, highlighting that fairness is not merely a data distribution issue but a fundamental property of spike-based processing dynamics.