💻 computer science

EventGait: Towards Robust Gait Recognition with Event Streams

This paper introduces EventGait, a robust end-to-end dual-stream framework that leverages event cameras and a novel Mixture of Spiking Experts architecture to achieve state-of-the-art gait recognition performance, particularly in challenging low-light conditions where traditional camera-based methods fail.

Original authors: Senyan Xu, Shuai Chen, Chuanfu Shen, Kean Liu, Zhijing Sun, Chengzhi Cao, Xueyang Fu

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Senyan Xu, Shuai Chen, Chuanfu Shen, Kean Liu, Zhijing Sun, Chengzhi Cao, Xueyang Fu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to identify someone walking down a street. Usually, security cameras (RGB cameras) act like standard human eyes: they take a picture every fraction of a second. If it's bright and sunny, this works great. But if it's pitch black, foggy, or the person is moving too fast, the camera gets confused, the image blurs, or it just goes dark. It's like trying to read a book in the dark; you can't see the words.

EventGait is a new system that uses a different kind of "eye" called an Event Camera to solve this problem. Here is how it works, broken down into simple concepts:

1. The "Fire Alarm" vs. The "Video Camera"

Think of a standard camera like a video recorder that takes a full photo 30 times a second, regardless of whether anything is happening. It captures the whole scene, including the background, the clothes, and the lighting.

An Event Camera is more like a room full of tiny, hyper-sensitive fire alarms. It doesn't take pictures. Instead, it only "screams" (sends a signal) when it sees a change in brightness at a specific pixel.

If a person walks by in the dark, the standard camera sees nothing. The Event Camera sees the person's movement as a stream of "changes" (events).
It ignores static things like a wall or a tree. It only cares about the motion.
It works perfectly in total darkness or blinding sunlight because it doesn't rely on "seeing" light; it relies on changes in light.

2. The Problem with Previous Attempts

Before this paper, researchers tried to use Event Cameras for walking recognition, but they made a mistake. They took all those tiny "change" signals and mashed them together into a single, blurry picture over a long time.

The Analogy: Imagine trying to understand a fast dance routine by taking a photo of the dancer every 10 seconds and blurring them together. You lose the specific moves (the high-speed details) and just see a ghostly blob.
This approach threw away the super-fast, precise timing that makes Event Cameras special.

3. The Solution: EventGait (The Two-Stream Team)

The authors built EventGait, which is like a two-person detective team. Instead of mashing everything together, they split the job into two specialized streams that work together:

Stream A: The "Motion Detective" (Dynamic Stream)

What it does: This team focuses purely on the movement. It looks at the fast, tiny changes in the event stream.
The Secret Weapon (MoSE): To handle different conditions (like a fast walk in bright sun vs. a slow shuffle in the dark), they use a Mixture of Spiking Experts (MoSE).
- Analogy: Imagine a team of specialists. Some are "fast runners" who are great at spotting quick movements in bright light. Others are "slow, steady observers" who are great at spotting faint movements in the dark.
- The system automatically decides which specialist to listen to based on the situation. This ensures the system never gets confused by bad lighting or weird speeds.

Stream B: The "Shape Architect" (Static Stream)

What it does: Event cameras are bad at showing what a person looks like (their body shape) because they only see movement, not the full body. This team tries to figure out the person's shape.
The Secret Weapon (CroSA): Since the Event Camera data is sparse (like a sketch with missing lines), the system uses a "Teacher" to help.
- Analogy: Imagine a student trying to draw a person based on a few scattered dots. The "Teacher" (a massive AI trained on millions of normal photos) looks at the dots and says, "Hey, those dots usually form a human leg here." The system learns to fill in the missing shape details using this guidance.

4. The New Training Grounds (Datasets)

One big problem was that there weren't enough real-world examples of people walking in front of Event Cameras to train the AI.

The Fix: The authors built a "simulator." They took existing videos of people walking and mathematically converted them into Event Camera data.
They created two huge new libraries of data (SUSTech1K-E and CCGR-Mini-E) so the AI could learn to recognize thousands of different people under many different conditions.

5. The Results: Why It Matters

The paper tested EventGait against standard cameras and expensive 3D laser scanners (LiDAR).

In Normal Light: EventGait performed just as well as the best standard cameras.
In the Dark: This is where it shines. While standard cameras failed completely in low light, EventGait kept working almost as well as it did in the day.
Cost: It achieves results similar to expensive 3D laser scanners but uses a much cheaper camera.

In Summary:
EventGait is a smart system that uses a special camera which only sees changes in light. It splits the job into two parts: one part that is an expert at tracking fast movements (even in the dark) and another part that learns to guess the person's body shape. By doing this, it can identify people walking reliably in conditions where normal cameras go blind.

Technical Summary: EventGait: Towards Robust Gait Recognition with Event Streams

Problem Statement
Gait recognition offers a non-intrusive, privacy-preserving method for human identification but faces significant challenges in uncontrolled environments. Conventional RGB-based approaches suffer from performance degradation due to illumination changes, motion blur, and occlusions. While LiDAR-based methods offer greater robustness, their high cost and energy consumption hinder large-scale deployment. Existing event-based gait recognition methods attempt to leverage the high temporal resolution and dynamic range of event cameras but typically aggregate event streams into grid-based event images over long time windows. This approach discards fine-grained temporal dynamics critical for gait analysis and produces sparse spatial representations that are difficult for standard deep networks to interpret effectively. Furthermore, the field lacks large-scale event-based gait datasets for rigorous evaluation.

Methodology
The authors propose EventGait, an end-to-end dual-stream framework designed to separately model motion and shape while preserving the inherent advantages of event data.

Event Representation: The framework utilizes a two-scale temporal design. Event streams are decomposed into:
- Short-term slices ( $E_d$ ): High temporal resolution slices capturing fine-grained motion dynamics.
- Long-term slices ( $E_s$ ): Aggregated events over a longer window to capture stable spatial shape information.
Dynamic Motion Stream (MoSE): This stream processes short-term slices to model high-frequency motion. It employs a Mixture of Spiking Experts (MoSE). Recognizing that a single spiking neuron configuration cannot adapt to diverse motion and illumination conditions (e.g., fast motion in bright light vs. slow motion in low light), MoSE integrates multiple spiking experts with distinct membrane time constants ( $\tau$ ). A lightweight gating network dynamically selects and weights these experts, enabling robust perception across complex scenarios.
Static Shape Stream (CroSA): This stream processes long-term slices to learn dense shape representations. Since event data is spatially sparse, the authors introduce Cross-modal Structure Alignment (CroSA). A pre-trained Vision Foundation Model (VFM), specifically DINOv2, acts as a teacher to provide dense structural priors. The event-based static encoder (student) is trained to align its feature representations with the VFM's output via an alignment loss, effectively transferring structural knowledge to the sparse event modality.
Fusion and Training: The dynamic ( $F_d$ ) and static ( $F_s$ ) features are fused to create a unified gait descriptor. The model is trained using a combination of cross-entropy loss, triplet loss, and the cross-modal alignment loss.

Key Contributions

EventGait Framework: A novel dual-stream architecture that decouples dynamic motion and static shape modeling, addressing the limitations of previous event-based methods that rely on temporal aggregation.
Mixture of Spiking Experts (MoSE): A mechanism that combines spiking neurons with diverse time constants to adaptively capture motion patterns under varying illumination and speed conditions.
Cross-modal Structure Alignment (CroSA): A strategy to distill dense structural priors from a Vision Foundation Model into the event-based static stream, overcoming the sparsity of event data.
New Benchmarks: To address the lack of large-scale data, the authors established a synthesis pipeline to convert RGB gait videos into event streams. They released two new datasets: SUSTech1K-E (derived from SUSTech1K) and CCGR-Mini-E (derived from CCGR-Mini), providing diverse covariates and views for evaluation.

Experimental Results
Extensive experiments were conducted on both synthesized (SUSTech1K-E, CCGR-Mini-E) and real-world (DVS128-Gait, EV-CASIA-B) benchmarks.

Performance: EventGait achieves state-of-the-art performance, outperforming existing camera-based and event-based methods. On the SUSTech1K-E dataset, it achieves a Rank-1 accuracy of 92.8% overall, surpassing the previous best event-based method by a significant margin.
Low-Light Robustness: The method demonstrates superior robustness in low-light conditions. In night scenarios (NT), EventGait outperforms camera-based methods by a large margin (e.g., +37.3% improvement over baselines on SUSTech1K-E), validating the advantage of event cameras in challenging illumination.
Comparison with LiDAR: EventGait achieves results comparable to, and in some views surpassing, LiDAR-based methods (e.g., LidarGait++), despite using a significantly more affordable sensor modality.
Ablation Studies: Experiments confirm the necessity of both the static and dynamic streams; removing either leads to performance drops. The MoSE module with 3 experts was found to offer the best trade-off between accuracy and efficiency.

Significance and Claims
The paper claims that EventGait sets a new state of the art for event-based gait recognition. It highlights that event-driven analysis is not only feasible but highly robust, particularly in scenarios where conventional cameras fail (low light) and where LiDAR is too expensive for mass deployment. The work emphasizes that preserving fine-grained temporal dynamics through spiking networks, rather than aggregating them into static images, is crucial for unlocking the potential of event cameras in biometric recognition. The release of large-scale synthetic benchmarks aims to facilitate further research in this domain.