🤖 AI

Detecting AI-Generated Videos with Spiking Neural Networks

This paper introduces MAST, a Spiking Neural Network-based detector that leverages the unique event-driven processing of temporal residuals and semantic trajectories to achieve superior cross-generator generalization in detecting AI-generated videos, outperforming existing methods on the GenVidBench benchmark.

Original authors: Minsuk Jang, Yujin Yang, Hee-Seon Kim, Minseok Son, Younghun Kim, Changick Kim

Published 2026-08-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minsuk Jang, Yujin Yang, Hee-Seon Kim, Minseok Son, Younghun Kim, Changick Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a movie where the actors are so perfect, the lighting is flawless, and every single frame looks like a high-resolution photograph. For a long time, if you wanted to tell if a video was real or made by a computer, you just looked at one still picture. If the ears were slightly weird or the hands had too many fingers, you knew it was fake. But computers have gotten so good that they can now draw a perfect face in a single frame. So, the game has changed. Now, the "fake" isn't in the picture itself; it's in the movement between the pictures.

This is the world of AI-generated video detection. The core idea is simple: real life is messy. When you move your head, the wind blows your hair, and the light shifts, the changes happen in a specific, chaotic, natural rhythm. AI, however, often tries to be too perfect. It smooths things out too much, making the movement between frames feel a bit like a slow-motion dream rather than real life. Scientists have been trying to build detectors that spot these "smoothness" glitches. But here's the tricky part: different AI programs make different kinds of glitches. One might make the hair move too smoothly, while another might make the background flicker. A detector built to catch one type of glitch often fails when it sees a new type. This is why researchers are looking for a smarter way to watch the whole movie, not just the stills.

Enter a team from KAIST (Korea Advanced Institute of Science and Technology) who decided to try a very different approach. Instead of using the standard, heavy-duty computer brains (called Artificial Neural Networks) that try to memorize every single pixel, they turned to something inspired by the human brain: Spiking Neural Networks, or SNNs. Think of an SNN like a nervous system that only fires a signal when something changes. It ignores the boring, static parts of a video and only pays attention to the moments of action.

The researchers, led by Minsuk Jang and colleagues, discovered something fascinating about AI videos when they fed them into this "change-sensitive" brain. They found that while real videos create a scattered, natural pattern of "spikes" (signals) across the whole screen, AI-generated videos tend to concentrate their "spikes" only at the edges of objects, like the outline of a person or a moving car. It's as if the AI is so busy making the inside of the object look perfect that it forgets to make the edges move naturally.

To catch this, they built a new detector called MAST. Imagine MAST as a two-part detective. One part is a "frozen" expert that looks at the general story of the video (the semantic meaning) to understand what is happening. The other part is the SNN, which acts like a high-speed motion sensor. It breaks the video down into tiny "pseudo-events"—basically, it asks, "Did this pixel change?" If the answer is yes, it sends a tiny electrical spike. Because AI videos have those weird, smooth edges, the SNN sees a very specific pattern: a lot of spikes clumped right at the boundaries, but very few in the middle.

The team tested MAST on a tough challenge called GenVideo, where they trained it on videos made by one AI and then asked it to spot videos made by ten different AI programs it had never seen before. This is like teaching a security guard to spot a specific type of fake ID, and then seeing if they can spot any fake ID from a different country. The result? MAST got it right 93.14% of the time. That's better than most of the other top detectors that use the standard, heavy computer brains.

What makes this even cooler is that MAST is also much faster and uses less energy. Because it only fires when something changes (like a nervous system reacting to a sudden noise), it doesn't waste energy processing the boring, still parts of the video. The paper suggests that this "event-driven" style is a perfect match for catching AI fakes, because the glitches in AI videos are often hidden in those tiny, rapid moments of change.

The authors are careful to note that this isn't a magic bullet that solves everything forever. If a video is very short, or if the AI gets even better at faking the movement, the detector might struggle. But for now, they have shown that using a brain that thinks in "spikes" and "events" is a powerful new way to spot the smooth, unnatural motion of AI-generated videos. It's a reminder that sometimes, to find the truth in a moving picture, you don't need to look at everything at once; you just need to know exactly when to look for a change.

Technical Summary: Detecting AI-Generated Videos with Spiking Neural Networks (MAST)

Problem Statement
Modern AI-generated videos have achieved photorealism at the single-frame level, rendering static artifact detection ineffective. The primary distinction between real and synthetic video now lies in inter-frame temporal dynamics. Existing detectors typically address this by feeding full sequences to temporal backbones, reducing cues to fixed video-level descriptors, or comparing temporal statistics against real-video baselines. However, these strategies suffer significant performance degradation under cross-generator evaluation, where the type and timescale of temporal artifacts vary widely across different generative models. The core challenge is to detect temporal inconsistencies that are sparse, asynchronous, and localized, without collapsing time-varying evidence into static summaries that lose critical dynamic information.

Methodology: MAST
The authors propose MAST (Multi-channel pseudo-event SNN with Adaptive Spiking Temporal integrators), a detector that leverages Spiking Neural Networks (SNNs) to process temporal residuals as event-like signals. The architecture consists of two complementary branches:

Spike-Driven Temporal Branch (SDTB):
- Pseudo-Event Generation: Instead of processing raw RGB frames, the model converts inter-frame differences into "pseudo-events." It computes four pixel-level residual channels: high-frequency Laplacian, Sobel edge magnitude difference, absolute frame difference, and second-order frame difference. These are transformed into soft binary events using a differentiable threshold function.
- Adaptive Temporal Integration: The pseudo-events are fed into a Per-Channel Leaky Integrate-and-Fire (LIF) module. Unlike standard SNNs with fixed time constants, MAST employs learnable time constants ( $\tau_c$ ) and firing thresholds ( $V_{th,c}$ ) for each channel. This allows the network to adaptively integrate signals at different temporal scales specific to each residual type.
- Spike-Driven Attention: The integrated spikes are processed by a Spike-Driven Transformer (based on SDT-V3) using spike-based self-attention to capture long-range temporal dependencies and generate spatial gate maps.
Semantic Trajectory Branch:
- This branch utilizes a frozen X-CLIP encoder to extract semantic features. It provides two key inputs:
  - A global video-level embedding ( $Z_{video}$ ) for semantic priors.
  - A sequence of patch tokens used to compute trajectory curvature and displacement in the semantic feature space.
- The authors observe that AI-generated videos exhibit more compact and less curved trajectories in semantic space compared to real videos.
Fusion and Training:
- The final representation concatenates the video-level semantic feature, the temporal anomaly vector (derived from the SDTB's gate maps), and the spatial gate vector.
- Training employs a multi-objective loss: binary cross-entropy on the fused prediction, an auxiliary cross-entropy on the SDTB-only prediction (to ensure the temporal branch is self-sufficient), and supervised contrastive learning to separate real and fake clusters in the feature space.

Key Findings and Observations
Prior to proposing MAST, the authors conducted a paired analysis on the GenVidBench dataset, revealing two distinct signatures of AI-generated videos:

Pixel Level: Generated videos exhibit smoother, late-accumulating temporal residuals. When fed into an SNN, fake clips elicit firing predominantly at object and motion boundaries, whereas real clips produce more diffuse activations.
Semantic Level: The trajectories of generated videos in the feature space occupy smaller, less curved regions compared to natural videos.
SNN Suitability: The authors argue that SNNs are a natural fit for this task because their event-driven, sparse activation dynamics align with the structure of inter-frame residuals, which are sparse and concentrated at moments of change.

Experimental Results

Cross-Generator Generalization: On the GenVideo benchmark, MAST was trained on Kinetics-400 and Pika clips and tested on 10 unseen generators (including ModelScope, MorphStudio, MoonValley, HotShot, Show1, Gen2, Crafter, Lavie, Sora, and WildScrape). MAST achieved a 93.14% mean accuracy, outperforming the second-best baseline (NSG-VD at 91.46%) and ranking first on 5 of the 10 unseen generators.
Backbone-Matched Comparison: On the IvyFake benchmark, when compared against detectors using the same X-CLIP semantic backbone but different temporal modules (Mamba-based DeMamba, RAFT-based D3), MAST achieved 85.91% mACC, surpassing the ANN-based alternatives.
Efficiency: In a parameter-matched comparison (9.3M parameters for SNN vs. 10.7M for ANN), MAST achieved higher accuracy (93.14% vs. 92.78%) with 1.6x lower latency (33.88ms vs. 54.19ms).
Ablation: Removing the pseudo-event channels or the learnable time constants resulted in significant performance drops, validating the necessity of both the event-driven formulation and adaptive integration.

Significance and Claims
The paper claims to present the first SNN-based detector specifically designed for AI-generated video detection. Its significance lies in demonstrating that:

SNNs are practical for AIGV detection: By treating inter-frame residuals as pseudo-events, SNNs can match or surpass the performance of dense ANN backbones while offering superior energy efficiency and lower latency.
Temporal Structure is Key: Effective detection requires preserving the time-varying structure of residuals rather than collapsing them into fixed descriptors.
Adaptive Integration: Different temporal cues operate at different timescales, necessitating learnable, channel-specific integration constants rather than a single global time constant.

The authors maintain a modest tone regarding limitations, noting that MAST's performance degrades on very short clips where temporal signals are weak, and that applying SNNs directly to dense RGB domains remains less effective than ANNs, a gap they leave for future work.

Technical Summary: Detecting AI-Generated Videos with Spiking Neural Networks (MAST)

More like this