💻 computer science

Gyroscope-Guided Optimal Transport for SuperGlue: A Training-Free Epipolar-Arc Prior

This paper proposes a training-free epipolar-arc prior that leverages gyroscope data to guide SuperGlue's matching process, significantly improving pose estimation accuracy—particularly under motion blur and for pipelines that ignore inertial data—by rewarding candidates based on their angular distance to the geometric arc defined by the rotation and estimated translation direction.

Original author: Ufuk Asil

Published 2026-11-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original author: Ufuk Asil

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of robotics and autonomous navigation, machines often rely on cameras to understand where they are and how they are moving. This process, known as visual odometry, works by finding matching points between consecutive images, like connecting the dots on a map to trace a path. However, when a camera moves quickly or the image becomes blurry, these digital "dots" can become hard to find, leading the machine to lose its way. To solve this, engineers have long turned to a second source of information: the gyroscope. Just as a human inner ear helps us sense rotation even when our eyes are closed, a gyroscope in a robot measures how the camera spins, independent of the image quality. The challenge has been figuring out how to combine this spinning data with modern, AI-powered matching systems that were originally designed to work using only visual information.

A new study by Ufuk Asil at Ostim Technical University explores how to bridge this gap without having to retrain the complex AI models from scratch. The researcher focused on a popular AI system called SuperGlue, which is excellent at matching features between images but ignores the gyroscope data entirely. The goal was to feed the gyroscope's rotation measurements directly into the AI's decision-making process to help it find the correct matches, even when the camera is spinning fast or the image is blurred. The study tested this idea on real-world flight and walking data, comparing the AI's performance with and without the help of the gyroscope.

The initial attempt to simply tell the AI to look for matches in the direction the gyroscope predicted the camera was spinning failed. The researchers found that this simple approach actually made the system slightly worse. The reason was twofold: first, when a camera moves forward, objects in the scene shift in a way that a simple rotation prediction cannot account for; second, the AI has a special "trash bin" for points it cannot match, and the simple rotation rule accidentally pushed too many good points into this bin. To fix this, the team developed a new method called the Epipolar-Arc Prior. Instead of just guessing a single point where a match should be, this method draws a curved path, or an arc, on the image. This arc represents the only possible places a correct match could exist, based on the known rotation and an estimated direction of movement. The AI is then rewarded for finding matches along this specific curve and left alone if the points fall elsewhere.

The results of this new method were striking. When tested on fifteen different video sequences from two major datasets, the system using the new arc-based guidance consistently outperformed the standard version. In scenarios with normal images, the improvement in tracking accuracy was noticeable, but the gains became dramatic when the images were blurred or the camera spun rapidly. Under conditions of synthetic motion blur, the accuracy improved by nearly ten percentage points, a massive leap for such a system. The method proved so effective that it worked even when the researchers applied it to a different, more recent AI matcher called LightGlue without any retraining, though the benefits were most visible when the image quality was poor or the number of detectable points was low.

Crucially, the study also clarified where this improvement comes from. The researchers discovered that the biggest boost to a robot's ability to know its position comes not from the matching system itself, but from using the gyroscope data later in the process, during the final calculation of the robot's location. However, for systems that do not use the gyroscope in that final step, this new matching method provides a vital safety net. It helps the robot find more correct connections between images, effectively closing the gap between a standard camera system and one that uses advanced inertial sensors. The study concludes that while the method is not a magic fix for every situation, it is a powerful, training-free tool that makes visual navigation significantly more robust when the world gets blurry or the camera spins fast.

Technical Summary: Gyroscope-Guided Optimal Transport for SuperGlue

Problem Statement
Learned feature matchers like SuperGlue and LightGlue rely exclusively on image evidence to establish correspondences between frames. However, visual-inertial platforms (VIO) inherently possess gyroscope data that measures inter-frame rotation independently of image quality. While classical trackers utilize this inertial data to initialize search windows or pre-warp images, modern learned matchers typically operate as "visual-only" black boxes, incorporating inertial data only after the matching stage (e.g., in the pose solver). The author investigates how to integrate gyroscope data directly into a pretrained learned matcher without retraining, addressing the challenge that simple integration of rotation priors often fails to improve pose accuracy due to translation-induced parallax and the specific mechanics of the matcher's "dustbin" (unmatched class).

Methodology
The proposed method introduces a Training-Free Epipolar-Arc Prior (EAP) designed for the Optimal Transport (OT) layer of SuperGlue. The approach operates as follows:

Single GNN Pass: The pretrained SuperGlue Graph Neural Network (GNN) is executed once to generate the initial score matrix $S$ .
Translation Estimation: Using the known gyroscope rotation ( $R$ ) and the provisional matches from the first OT solve, a robust two-point solver (RANSAC) estimates the translation direction ( $\hat{t}$ ). This leverages the fact that with known rotation, the epipolar constraint becomes linear in translation.
Epipolar-Arc Construction: Instead of penalizing all candidates far from a pure-rotation prediction (which fails due to parallax), the method defines a "great-circle arc" on the unit sphere. This arc connects the rotated bearing vector (infinite depth) to the estimated translation direction (zero depth).
Log-Prior Injection: A geometric log-potential is added to the score matrix. Crucially, this is a reward-only prior: candidates lying close to the arc receive a score boost, while inconsistent candidates remain unchanged. This avoids the issue where penalty-only priors shift probability mass toward the "dustbin" (unmatched class) because the dustbin does not receive the prior.
Second OT Solve: The assignment layer (Sinkhorn algorithm) is run a second time on the modified score matrix ( $S + P$ ) to produce the final matches.

The method also explores optional Positional Encoding (PE) compensation, where keypoint positions are rotated by the gyroscope data before entering the GNN, though this requires a second GNN pass.

Key Contributions

Epipolar-Arc Prior (EAP): A novel, training-free mechanism that combines gyroscope rotation with a data-driven translation estimate to constrain matches along an epipolar arc rather than a single point.
Analysis of Prior Injection: The paper identifies two reasons why simple rotation priors fail in OT-based matchers: (1) translation-induced parallax requires a loose tolerance that negates the prior's benefit, and (2) penalty-only priors disproportionately penalize the "dustbin," shifting mass to "unmatched." The EAP addresses both by using a tight arc constraint and a reward-only offset.
Rigorous Evaluation Protocol: The author establishes a held-out evaluation using 15 sequences from TUM-VI and EuRoC datasets, with hyperparameters frozen on a single development sequence. They control for RANSAC variability via point-order permutations and use Holm-corrected permutation tests for statistical significance.

Results

Pose Accuracy: On held-out TUM-VI and EuRoC sequences, the EAP improves the Area Under the Curve (AUC) of pose error at $10^\circ$ by +1.9 points on TUM-VI and +2.2 points on EuRoC for unmodified images. The pooled gain across all regimes is +4.4 points on TUM-VI and +2.7 points on EuRoC, with gains reaching up to +9.9 points under synthetic motion blur.
Consistency: The method increases the number of epipolar-consistent matches on all 15 held-out sequences. The performance gain correlates positively with the magnitude of rotation; gains are larger for larger inter-frame rotations.
Comparison to Baselines:
- Simple rotation-only priors (penalty-based) generally degraded or provided negligible performance.
- Post-hoc filtering (discarding matches outside the arc after matching) yielded significantly smaller gains (+1.0 to +0.1 points) compared to integrating the prior into the assignment.
- The EAP outperformed unmodified LightGlue on both datasets, though the gains were smaller and less consistent than with SuperGlue, likely due to LightGlue's lack of a dustbin and different assignment mechanism.
Runtime: The method adds approximately 23 ms per pair (dominated by the second Sinkhorn solve and 2-point RANSAC) on an NVIDIA RTX 3090, without requiring a second GNN pass.

Significance and Claims
The paper claims that the EAP effectively bridges the gap between classical inertial-aided tracking and modern learned matchers without retraining. Its primary significance lies in demonstrating that inertial data can improve the matching stage itself, not just the downstream pose estimation.

However, the author is modest about the scope of these gains:

The improvement is most significant for pipelines where the pose solver does not use inertial data. When a known-rotation 2-point solver is used downstream, the EAP adds only marginal value (+0.5 points), as the pose solver already exploits the gyroscope.
The method assumes a static scene; dynamic objects violate the epipolar geometry assumptions, though the truncated nature of the prior limits negative impact.
The evaluation is limited to synthetic motion blur and does not yet cover real-world motion blur or full VIO back-end integration with online bias estimation.

In conclusion, the work provides a practical, training-free augmentation for SuperGlue that leverages gyroscope data to handle large rotations and motion blur more robustly, particularly in scenarios where the pose estimation pipeline does not natively fuse inertial measurements.

Technical Summary: Gyroscope-Guided Optimal Transport for SuperGlue

More like this