💻 computer science

SpikingMOT: A Spike-Driven Multi-Object Tracker

The paper introduces SpikingMOT, a spike-driven multi-object tracker that leverages activation sparsity preference and spiking neural networks to achieve state-of-the-art performance on SportsMOT and DanceTrack benchmarks while significantly reducing parameters and energy consumption.

Original authors: Yiding Sun, Xiangyang Yang, Dongxu Zhang, Qirui Wang, Zijie Xu, Wenxuan Liu, Shuiwang Li, Jihua Zhu, Zhaofei Yu, Tiejun Huang

Published 2026-07-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yiding Sun, Xiangyang Yang, Dongxu Zhang, Qirui Wang, Zijie Xu, Wenxuan Liu, Shuiwang Li, Jihua Zhu, Zhaofei Yu, Tiejun Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a busy street corner from a high window. Hundreds of people are walking, running, and weaving through the crowd. Your brain doesn't try to track every single person by calculating their exact speed and direction every millisecond; instead, it focuses on the most important movements and ignores the rest. This is the magic of Multi-Object Tracking (MOT), a superpower for computers that lets them follow moving things like cars, players, or drones in videos. For a long time, computers tried to do this by using massive, "dense" brains (called Artificial Neural Networks) that fire off signals for everything at once, like a stadium full of people shouting at the same time. But this uses a lot of energy and can get confused by the noise. The big question scientists have been asking is: Do we really need all that shouting to keep track of who is who, or could a quieter, more selective approach work just as well?

This is where a new paper called SpikingMOT comes in. The researchers, led by Yiding Sun and his team, decided to test a wild idea: what if we made the computer's tracking brain act more like a human brain? Human brains use "spikes"—tiny, electric bursts that only happen when something important needs to be said. The team discovered that for tracking moving objects, being "sparse" (only firing when necessary) isn't just a backup plan; it's actually a superpower. They found that by using a special type of brain-inspired network called a Spiking Neural Network (SNN), they could track objects with incredible accuracy while using way less energy and computer memory.

The Problem: The Shouting Stadium

For years, the best trackers worked like a chaotic stadium. Every time a video frame changed, the computer's "dense" neural network would activate thousands of neurons to predict where objects would go next. The researchers noticed something strange: even when they forced the computer to be quiet and only let a few neurons speak (a process called "dropout"), the tracking actually got better in many cases. It was as if the stadium crowd was shouting so loud that the important messages were getting lost in the noise. They called this observation the Activation Sparsity Preference (ASP). Essentially, the data suggested that for predicting where a ball or a person will move next, you don't need a full choir; you just need the right soloists.

The Solution: The Brain's Secret Code

To solve this, the team built SpikingMOT, the first tracker that runs entirely on these "spikes." Instead of a constant hum of activity, SpikingMOT waits for specific moments to fire, much like a neuron in your own brain. But simply making the network quiet wasn't enough. The team realized they needed a smart way to decide which neurons should fire.

They designed a two-step "brain-inspired" loop:

The Decomposition (The Menu): Imagine the computer has a menu of possible moves (like "turn left," "speed up," or "stop"). Instead of trying to guess the exact move, SpikingMOT breaks the movement down into a set of "pseudo-trajectories" (fake paths). It's like having a deck of cards representing different ways a person might move.
The Calibration (The Correction): After the computer guesses where an object will be, it checks the real video to see if it was right. If the object was actually somewhere else, the computer uses that "mistake" (the error) to adjust its menu for the next second. It's like a chef tasting a soup, realizing it needs more salt, and then adjusting the recipe for the next batch. This creates a closed loop where the tracker learns from its own mistakes in real-time.

The Results: Faster, Smarter, and Greener

The results were impressive. When they tested SpikingMOT on tricky datasets like SportsMOT (where players run fast and change direction suddenly) and DanceTrack (where dancers look very similar and cross paths often), it didn't just keep up; it took the lead.

On SportsMOT, it achieved a top score of 74.9 (called HOTA), beating the previous best.
On DanceTrack, it hit 56.5, also a new record.
But the real magic was in the efficiency. By using this spike-driven approach, they reduced the number of computer parameters (the "brain size") by 72% and slashed the energy usage by 86.7%.

Why It Matters

The paper suggests that for tracking moving objects, we don't need to build bigger, louder, and more energy-hungry computers. Instead, we can build smarter, quieter ones that mimic how nature solves problems. The researchers showed that by letting the computer "spike" only when it has something important to say, and by constantly correcting its own predictions based on what it sees, it can track complex movements better than ever before. While the team notes that their energy savings are currently calculated based on math rather than tested on special "neuromorphic" hardware chips, the results strongly suggest that the future of tracking might be less about brute force and more about biological efficiency. They have opened a door to a new way of thinking: sometimes, to see the whole picture, you just need to listen to the right whispers.

Technical Summary: SpikingMOT

Problem Statement

Multi-object tracking (MOT) is a fundamental task in visual perception, typically following the Tracking-by-Detection (TBD) paradigm. While recent learning-based methods have improved motion modeling using densely activated artificial neural networks (ANNs), they often overlook whether such dense responses are necessary for accurate trajectory prediction. Classical filter-based methods struggle with complex motion patterns, and existing learning-based approaches (e.g., MambaTrack) rely on dense computations that may introduce redundant motion cues and unnecessary energy consumption.

The paper identifies a phenomenon termed Activation Sparsity Preference (ASP): empirical evidence suggests that moderate suppression of activation rates in trajectory predictors not only preserves but can improve tracking performance. Specifically, dense responses can amplify redundant motion cues, leading to inaccurate predictions or identity switches, whereas sparse activations can preserve informative motion responses. The core challenge is to formulate a model architecture that formally explains ASP and translates this sparsity into competitive tracking performance without sacrificing accuracy.

Methodology

The authors propose SpikingMOT, the first spike-driven tracker for MOT, which leverages Spiking Neural Networks (SNNs) to adaptively model sparse trajectory dynamics. The framework is built on the insight that state-conditioned sparse gating is theoretically no worse than state-independent dropout under the same activation rate.

SpikingMOT decomposes the tracking process into two alternating, brain-inspired optimization steps:

1. Motion-Basis Decomposition

Instead of a single dense prediction, SpikingMOT decomposes each trajectory state into pseudo-trajectory bases.

Population Coding: Inspired by biological population coding, the encoded motion state is decomposed into $K$ basis-conditioned motion responses.
Spike-Gated Reconstruction: A dense basis posterior is encoded and passed through Leaky Integrate-and-Fire (LIF) neurons. The membrane dynamics determine whether a spike is emitted.
Sparse Selection: The emitted spikes act as a gate, selecting a subset of informative motion bases while suppressing others. This creates a "pseudo-trajectory" where only relevant motion hypotheses are activated for the current frame.

2. Prediction-Error Calibration

To adapt the selected responses to new observations, the system employs a predictive coding loop.

Residual Calculation: After associating the predicted box with a detection (or ground truth during training), the system computes a scale-normalized box residual (prediction error).
Calibration Head: This residual is fed into a calibration head along with the current state and spike gates. The head predicts a logit correction for all motion bases.
Posterior Update: The corrections update the posterior probabilities of the motion bases, which are then carried forward to calibrate the prediction for the next frame ( $f+1$ ). This forms a closed loop where prediction errors drive the selection of future motion dynamics.

The overall loss function combines box prediction loss with regularizers for spike sparsity and basis diversity, ensuring the model learns an efficient, sparse motion distribution.

Key Contributions

ASP Phenomenon: The paper explores and formalizes the Activation Sparsity Preference in MOT, providing biological plausibility through population coding and predictive coding theories.
SpikingMOT Architecture: It introduces the first spike-driven tracker for MOT, utilizing SNNs to provide adaptive sparse activations for motion prediction.
Brain-Inspired Strategy: The authors propose a novel strategy comprising pseudo-trajectory decomposition (for structured motion representation) and prediction-error calibration (for online adaptation), enabling effective sparse selection.
Efficiency and Performance: Extensive experiments demonstrate that SpikingMOT achieves state-of-the-art (SOTA) performance while significantly reducing computational costs.

Experimental Results

The authors evaluated SpikingMOT on four benchmarks: DanceTrack, SportsMOT, SeaDroneSee, and MOT17.

Performance:
- On SportsMOT, SpikingMOT achieved a HOTA of 74.9, outperforming dense ANN baselines like MambaTrack (72.6).
- On DanceTrack, it achieved a HOTA of 56.5, surpassing previous methods.
- The model showed particular strength in scenarios with complex, non-linear motion (SportsMOT) and crowded interactions (DanceTrack), maintaining consistent identities where dense models suffered from identity switches.
Efficiency:
- Compared to the baseline MambaTrack, SpikingMOT reduced parameters by 72% and energy consumption by 86.7%.
- It achieved these gains while using significantly fewer active units (e.g., 15.4% spike firing rate with 8 bases).
Ablation Studies:
- Experiments confirmed that simple sparsity mechanisms (e.g., random dropout, $\ell_1$ pruning) do not yield the same benefits as the proposed spike-gated decomposition and calibration.
- The LIF neuron type was found to offer the best balance between temporal memory and optimization complexity.
- The method demonstrated generalization across different detectors (SpikeYOLO and YOLOX variants).

Significance and Claims

The paper claims that SpikingMOT brings SNNs into the domain of multi-object tracking, opening a promising direction for efficient tracking. The primary significance lies in demonstrating that sparsity is a learnable principle for motion reasoning.

The authors argue that:

Dense activation is not strictly necessary for trajectory prediction; in fact, sparse, adaptive gating can improve robustness against redundant motion cues.
The proposed brain-inspired loop (decomposition + calibration) effectively mimics biological mechanisms to handle complex motion dynamics.
While current energy estimates are based on operation counts, the results suggest that deploying such models on neuromorphic hardware could yield even greater efficiency gains.

The paper concludes that future work should focus on deploying these models on neuromorphic chips and evaluating them with spike-camera data in real-world scenarios.