💻 computer science

GeoMag: Geometric-Aware Video Motion Magnification via State Space Model

The paper introduces GeoMag, a geometric-aware video motion magnification framework leveraging State Space Models for globally consistent, linear-complexity amplification, supported by the new Geo-200K dataset to address limitations in existing methods regarding structural consistency and training diversity.

Original authors: Kecheng Han, Yuchen Zhang, Bingqing Liu, Boqiang Guo, Wenbin Zheng, Shiyuan Pei

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kecheng Han, Yuchen Zhang, Bingqing Liu, Boqiang Guo, Wenbin Zheng, Shiyuan Pei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a video of a hummingbird's wings or a vibrating machine part. To the naked eye, these movements are so tiny they look like a blur or just static noise. Video Motion Magnification (VMM) is like a "motion microscope" that tries to make these invisible wiggles huge so we can see them.

However, making these tiny movements bigger is tricky. If you just turn up the volume on the motion, you also turn up the "static" (noise) and often break the picture, making things look wobbly or distorted.

The paper introduces a new tool called GeoMag that solves these problems. Here is how it works, using simple analogies:

1. The Problem: The "Blurry vs. Broken" Dilemma

Previous methods tried to magnify motion using two main approaches, both of which had flaws:

The Local Detective (CNNs): These are like a detective looking at one small room at a time. They are fast, but they don't know what's happening in the rest of the house. This leads to local distortions where parts of the image look warped.
The Global Watcher (Transformers): These are like a detective with a drone view of the whole city. They see the big picture perfectly, but they are so slow and expensive to run that they can't handle high-definition videos in real-time.

GeoMag finds the "Goldilocks" zone: it sees the whole picture (global consistency) but runs as fast as the local detective.

2. The Secret Sauce: The "Mamba" Engine

GeoMag uses a new type of AI architecture called a State Space Model (specifically, a variant called Mamba).

The Analogy: Imagine reading a book. Old methods (Transformers) try to read every single word and compare it to every other word in the book at once to understand the story. This is slow.
GeoMag's Approach: It reads the book linearly, word by word, but it has a "selective memory." It remembers the important plot points (the actual motion) and instantly forgets the random scribbles on the page (the camera noise). This allows it to understand the whole story (the whole video frame) without getting bogged down, making it incredibly fast and efficient.

3. The Training Ground: "Geo-200K"

AI models need to be trained on examples to learn how to magnify motion without breaking things.

The Old Way: Most previous models were trained on videos where objects just moved in straight lines (like a car driving forward). It was like teaching a pilot to fly only on a straight, empty runway.
The New Way (Geo-200K): The authors built a massive new training dataset called Geo-200K. They created a "virtual playground" where objects don't just move straight; they spin, twist, and rotate. They also added realistic "camera glitches" like grain and blur.
The Result: It's like taking that pilot and training them in a storm with wind gusts and sharp turns. Now, when GeoMag faces a real-world video, it's not surprised by complex movements or camera noise. It knows exactly how to handle them.

4. How GeoMag Works (The Two-Stream Kitchen)

The system has two "chefs" working together to cook the final video:

Chef 1 (The Motion Specialist): This chef uses the Mamba engine to find the moving parts. They take the tiny movement, amplify it (make it bigger), and then use their "selective memory" to smooth out any jagged edges or noise. They ensure the moving object stays rigid and doesn't turn into a blob.
Chef 2 (The Detail Specialist): This chef focuses on the background and static details (like the texture of a wall or a shirt). Their job is to make sure that while the motion is getting bigger, the rest of the picture doesn't get blurry or lose its sharpness.
The Final Dish: They combine their work. You get a video where the motion is huge and clear, but the background remains sharp and the object doesn't look like it's melting.

5. The Results

The paper tested GeoMag against the best existing methods:

Better Quality: It produces clearer videos with fewer "artifacts" (weird visual glitches like ripples or broken shapes).
Faster: It is about 5 times faster than the most advanced previous methods that used the "Global Watcher" (Transformer) approach.
More Robust: Because it was trained on the complex Geo-200K dataset, it handles spinning objects and noisy cameras much better than models trained only on simple straight-line movements.

In summary: GeoMag is a new, fast, and smart way to zoom in on invisible movements in videos. It uses a clever "selective memory" system to keep the picture stable and a super-charged training dataset to ensure it works even when things are spinning or the camera is shaky.

Technical Summary: GeoMag

Problem Statement
Video Motion Magnification (VMM) aims to reveal imperceptible dynamics in video sequences, acting as a "motion microscope" for applications in structural analysis, biomedicine, and micro-expression detection. However, a core challenge remains: distinguishing minute, meaningful motion from imaging noise without compromising structural integrity. Existing learning-based methods face a structural dilemma. Convolutional Neural Networks (CNNs) are computationally efficient but lack the long-range dependencies required for global geometric consistency, often leading to local distortions. Conversely, Transformer-based approaches offer global context but suffer from quadratic computational complexity, making high-resolution processing prohibitive. Furthermore, current training protocols (e.g., LBVMM) rely heavily on simple linear translations, failing to capture the complex geometric transformations (rotation, compound motion) and sensor degradations found in real-world scenarios, leading to models that overfit to simplified motion patterns.

Methodology
The authors propose GeoMag, a geometric-aware VMM framework built upon State Space Models (SSMs), specifically leveraging the Mamba architecture, to achieve globally consistent motion amplification with linear computational complexity.

Architecture: GeoMag employs a dual-stream design:
1. Primary Motion Stream: Utilizes a Deep Feature Extractor (DFE) and a Mamba-based Sequence Awareness (MMSA) encoder. The DFE, composed of stacked Vision Mamba blocks, extracts deep semantic representations. The MMSA encoder processes manipulated features using a 2D Selective Scan mechanism. This mechanism acts as a spatially-variant filter that leverages input-dependent selectivity to retain coherent motion patterns while suppressing stochastic noise, ensuring global consistency with linear complexity.
2. Static Feature Refiner (SFR): A parallel branch operating on shallow, high-resolution features to preserve high-frequency texture details and stabilize the static background, preventing the loss of fine details common in deep encoders.
3. Feature Manipulation: Unlike traditional phase-based methods, GeoMag computes motion differentials directly in the latent feature space. These differentials are scaled by a magnification factor $\alpha$ and processed by a non-linear manipulation function before being refined by the MMSA and fused with static details.
Data Synthesis (Geo-200K): To address the limitations of existing training data, the authors constructed Geo-200K, a large-scale synthetic dataset. This pipeline generates training pairs by compositing foreground objects from CocoNut and backgrounds from OpenImages. Crucially, it introduces:
- Complex Geometric Transformations: A mix of pure translation, pure rotation, and combined transformations (30%/30%/40% ratio).
- Sensor-Realistic Degradations: A physics-based simulation pipeline that includes photon shot noise, readout artifacts, tone mapping, and analog-to-digital conversion quantization.
- Kinematic Constraints: Strict bounds on displacement and rotation to ensure physical plausibility and numerical stability.

Key Contributions

GeoMag Framework: A novel VMM architecture based on State Space Models that effectively handles complex geometric motions while reducing visual artifacts and blurring compared to existing CNN and Transformer methods.
Geo-200K Dataset: A large-scale synthetic dataset incorporating complex geometric transformations and realistic sensor degradations, providing diverse and realistic supervision for VMM training.
Comprehensive Validation: Extensive experiments demonstrating that the proposed framework and dataset outperform state-of-the-art methods in visual fidelity and computational efficiency.

Experimental Results

Quantitative Performance: On four synthetic benchmarks (covering translation, rotation, noise, and blur), GeoMag trained on Geo-200K consistently outperformed existing baselines. Notably, on the rotation-heavy Synthetic-II dataset, GeoMag achieved the lowest LPIPS (0.16) and RMSE, significantly outperforming Transformer-based baselines like STB-VMM and EulerMormer, which struggled with structural rigidity under non-linear rotation.
Efficiency: GeoMag achieves a favorable trade-off between performance and cost. It requires approximately 5 $\times$ fewer GFLOPs (296.1G) than STB-VMM (1584G) while processing a 300-frame 720p video in 13.9 seconds, compared to 31.6 seconds for STB-VMM and 50.0 seconds for EulerMormer.
Qualitative Analysis: Visual comparisons on real-world videos (e.g., baby breathing, engine vibration, rotor rotation) show that GeoMag preserves structural continuity better than prior methods. It avoids the ringing artifacts of hand-crafted filters, the texture blurring of CNNs, and the geometric inconsistencies (spatial discontinuities) observed in attention-based models.
Ablation Studies: Removing the SFR module caused a marked degradation in RMSE (from 24.22 to 29.42), confirming its necessity for stabilizing static regions. Replacing the Mamba backbone with CNN or Transformer variants resulted in significantly lower performance, validating the efficacy of the SSM-based design.

Significance
The paper claims that GeoMag provides a unified solution to the trade-off between long-range dependency modeling and computational costs in video motion magnification. By integrating State Space Models with selective scanning, the framework captures global geometric consistency without the quadratic complexity of Transformers. Furthermore, the introduction of Geo-200K addresses the critical bottleneck of data quality, offering a high-quality data foundation that simulates complex real-world kinematics and imaging physics. The authors position this work as a strong benchmark for efficient motion magnification research, demonstrating that superior visual fidelity can be achieved with significantly reduced computational resources.