💻 computer science

Do vision models perceive illusory motion in static images like humans?

This paper demonstrates that most current optical flow models fail to perceive the illusory motion in static images like the Rotating Snakes illusion, whereas a human-inspired Dual-Channel model with recurrent attention mechanisms successfully replicates human perception under simulated saccadic conditions, highlighting a significant gap between machine and human motion processing.

Original authors: Isabella Elaine Rosario (Columbia University), Fan L. Cheng (Columbia University), Zitang Sun (Kyoto University), Nikolaus Kriegeskorte (Columbia University)

Published 2026-04-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Isabella Elaine Rosario (Columbia University), Fan L. Cheng (Columbia University), Zitang Sun (Kyoto University), Nikolaus Kriegeskorte (Columbia University)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at a picture of a snake that seems to be slithering in a circle, even though it's just a flat, static drawing on a piece of paper. This is the "Rotating Snakes" illusion. If you stare at it, or if your eyes make tiny, involuntary jumps (called microsaccades), your brain tricks you into seeing motion where there is none.

For a long time, scientists have wondered: Do computers see this trick too? Or do they just see a still picture, like a camera that doesn't have a brain?

This paper is essentially a "magic trick test" for Artificial Intelligence. The researchers set up a series of optical illusions and asked various computer vision models (the "brains" behind self-driving cars and video analysis) to tell them what they see.

Here is the breakdown of what they found, using some simple analogies:

1. The Test: The "Still Snake" Challenge

The researchers took the famous "Rotating Snakes" image and showed it to 10 different computer models. These models are usually experts at tracking real movement, like a car driving down a road or a ball being thrown.

The Human Experience: When a human looks at the snake, their brain says, "Hey, that's spinning counter-clockwise!"
The Computer Experience: Most of the computer models said, "I see a picture. Nothing is moving." They were completely fooled by the fact that the image was static. They failed to see the "ghost" motion that humans perceive.

2. The "Eye Movement" Twist

The researchers realized something important: Humans don't just stare perfectly still. Our eyes are constantly making tiny, jittery movements (microsaccades) and bigger jumps (saccades). These tiny movements are actually required for us to see the snake spin.

So, the researchers simulated these eye movements for the computers. They took the static image and "jittered" it slightly, just like our eyes do.

The Result: Even with the jitter, most computers still saw nothing but noise or just the direction of the jitter. They couldn't separate the "fake" spin from the "real" shake.
The Exception: One specific model, called "Dual," started to see something. When the image jittered, the Dual model began to generate a flow of motion that looked a bit like the snake spinning. It wasn't perfect, but it was the only one that got close to the human experience.

3. Why Did the "Dual" Model Win? (The Secret Sauce)

The researchers wanted to know why the Dual model was special. They took it apart like a mechanic taking apart a car engine to see which part was making it work.

They found that the Dual model had a two-lane highway system for processing motion, whereas the others had only one lane.

Lane 1 (First-Order): This lane looks at simple changes in brightness (like a shadow moving).
Lane 2 (Higher-Order): This lane looks at more complex patterns, like textures and shapes moving.

The Analogy: Imagine trying to hear a whisper in a noisy room.

Most computers are like a microphone that only listens for loud, clear voices (simple brightness changes). They miss the whisper.
The Dual model is like a microphone with two settings: one for loud voices and one for subtle whispers. It combines both to understand the full picture.

The study showed that to see the illusion, a computer needs both lanes working together, plus a "memory" system (recurrent integration) that remembers what it saw a split second ago to smooth out the motion.

4. The Big Takeaway: Humans vs. Machines

The main conclusion is that current AI is still very different from human vision.

Current AI: It's like a super-fast, high-definition camera. It is amazing at measuring how far a car moved from point A to point B. But it lacks the "soul" of human vision. It doesn't have the biological quirks that make us see ghosts in static images.
Human Vision: Our brains are messy, biological machines that use tiny eye jitters and complex processing to create a stable, rich world. Sometimes, this creates cool illusions (like the spinning snake), but it also makes us robust and adaptable.

Why Does This Matter?

You might ask, "Who cares if a computer sees a spinning snake?"

The answer is: Safety and Reliability.
If we want to build self-driving cars or robots that can navigate the real world, they need to understand motion the way humans do. If a robot only sees "pixel displacement," it might fail in tricky situations (like fog, reflections, or weird lighting) that a human driver handles easily.

By studying these illusions, scientists are learning how to build "biologically inspired" AI—machines that don't just calculate math, but actually perceive the world more like we do.

In short: The paper proves that while computers are getting better at seeing motion, they still miss the "magic" of human perception. To fix this, we need to build computers that mimic the messy, jittery, two-lane processing of the human brain.

1. Problem Statement

While deep neural networks (DNNs) have achieved state-of-the-art performance in optical flow estimation benchmarks, they often fail to replicate the robustness and specific computational strategies of the human visual system. A critical gap exists in understanding whether these models can perceive illusory motion in static images—a phenomenon where humans perceive motion in a stationary image due to specific luminance asymmetries and eye movements.

The paper focuses on the Rotating Snakes illusion, a classic example of "peripheral drift" where static concentric rings appear to rotate. The authors investigate whether current optical flow models can reproduce this illusion under simulated viewing conditions (including eye movements) and identify the computational mechanisms required to do so.

2. Methodology

Experimental Design

Stimuli: The study utilized the Rotating Snakes illusion (in grayscale, blue-yellow, and red-green variants) and control images where luminance/color sequences were permuted to eliminate the illusion while preserving global layout. Additional illusions (Peripheral Drift, Central Drift, Ouchi) were also tested.
Viewing Conditions: To mimic human perception, the authors simulated four viewing conditions:
1. Static: Identical frames presented over time.
2. Stimulus Onset: A blank frame followed by the static image (simulating stimulus appearance).
3. Microsaccade: Small image shifts (15–120 pixels) simulating involuntary eye movements.
4. Saccade: Larger image shifts simulating voluntary eye movements.
Models Evaluated: Ten representative motion-estimation models were tested, categorized into:
- Multi-scale architectures: PWC-Net, LiteFlowNet2 (LFN2).
- Recurrent-decoding architectures: RAFT, CCMR, FlowDiffuser.
- Bio-inspired architectures: FFV1MT, DorsalNet, ME-Attention, and Dual (a model incorporating first-order and higher-order motion pathways with recurrent integration).

Evaluation Metrics

Primary Metric: A normalized correlation ( $\rho$ ) between the model's predicted optical flow field and a "veridical" human perceptual flow field (a smooth, counter-clockwise rotational field). This metric prioritizes directional alignment over exact magnitude.
Secondary Metrics: Average Endpoint Error (EPE) and Average Angular Error (AE) were also computed for completeness.

3. Key Contributions

Systematic Evaluation: The first comprehensive benchmark of diverse optical flow models against human illusory motion perception, specifically the Rotating Snakes illusion.
Unified Pipeline: Development of a standardized experimental pipeline that integrates controlled viewing simulations (static, onset, eye movements) with quantitative alignment metrics.
Mechanistic Insights: Through ablation and probing analyses, the authors identified that dual-channel processing (first-order + higher-order motion) combined with recurrent integration and oculomotor transients (simulated eye movements) are critical for reproducing human-like illusory motion.
Identification of the "Dual" Model: Demonstrated that the neuroscience-inspired Dual model is the only architecture capable of generating flow fields consistent with human perception under specific conditions, whereas standard DNNs fail.

4. Key Results

Performance of Standard DNNs

Failure to Perceive Illusion: Most conventional models (PWC-Net, RAFT, FlowDiffuser, etc.) failed to generate rotational flow for the Rotating Snakes illusion.
Static/Onset Conditions: Under static or stimulus-onset conditions, these models produced little to no meaningful flow or generated flow dominated by the direction of image shifts rather than the illusion.
Control Stimuli: These models produced similar (null) results for both illusion and control images, indicating they rely on global texture matching rather than local luminance asymmetries.

Performance of the Dual Model

Human-Like Alignment: The Dual model was the only model to exhibit significant positive correlation with human perceptual flow, particularly under microsaccade and saccade simulations.
Color Dependence: It showed strong alignment for grayscale and blue-yellow variants but negative correlation (clockwise rotation) for red-green variants, mirroring human psychophysical data where color interactions can reverse the illusion.
Mechanism: The illusion signal emerged only when:
1. Eye movements (image shifts) were simulated.
2. Both first-order (luminance-based) and higher-order (feature-based) motion pathways were active.
3. Recurrent integration (iterative refinement) was applied to fuse these signals.

Ablation and Probing Analysis

Pathway Necessity: Removing either the first-order or higher-order pathway in the Dual model drastically reduced alignment with human perception, proving both are necessary.
Recurrent Integration: The illusion signal strengthened with recurrent iterations (peaking at Stage II-6), suggesting that integrating local cues over time is essential for generating the global rotational percept.
Training Data: Training on specific data subsets (non-textured and non-diffuse object motion) was found critical for developing higher-order motion perception capabilities.

Other Illusions

Ouchi Illusion: Both Dual and ME-Attention successfully segmented the center from the surround, mimicking human perception of relative sliding motion, suggesting this illusion may rely more on first-order mechanisms.
Peripheral/Central Drift: Only the Dual model showed partial success in generating coherent rotation for these, and only under microsaccade conditions.

5. Significance and Implications

Gap in Current AI: The study highlights a fundamental disconnect between current optical flow models (which track pixel displacements) and human vision (which relies on complex, multi-stage integration of motion energy and eye movements).
Bio-Inspired Design: The success of the Dual model suggests that future robust vision systems must incorporate biologically plausible components, such as:
- Dual-pathway processing (first and second-order motion).
- Recurrent attention mechanisms for temporal integration.
- Simulation of fixational eye movements (microsaccades) during inference or training.
Robustness: Incorporating these mechanisms could lead to more robust computer vision systems capable of handling occlusions, non-rigid motion, and complex real-world environments where standard DNNs currently struggle.
Scientific Insight: The findings support the hypothesis that anomalous motion illusions are not merely artifacts but reveal specific computational principles (latency differences, higher-order integration) used by the primate visual system.

In conclusion, the paper demonstrates that while current DNNs excel at standard motion estimation, they lack the specific architectural and dynamic properties required to perceive static illusions. Bridging this gap requires moving beyond pure pixel-matching toward models that integrate biological motion processing principles and realistic eye movement dynamics.