💻 computer science

Robust Fundamental Matrix Estimation from Single Image Motion Blur

This paper proposes a robust method for estimating the fundamental matrix from a single motion-blurred image by establishing correspondences between time instances within the exposure window, addressing the unique time-direction ambiguity and incorporating uncertainty measurements to enable 3D camera motion inference and downstream motion segmentation.

Original authors: Bao-Long Tran, Per-Erik Forssén, Fredrik Viksten

Published 2026-05-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Bao-Long Tran, Per-Erik Forssén, Fredrik Viksten

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are taking a photo of a busy street, but your camera shakes while the shutter is open. Instead of a sharp picture, you get a blurry mess where cars and people look like smeared streaks of paint. Usually, photographers try to fix this by "de-blurring" the image to make it sharp again.

This paper proposes a different idea: What if we use the blur itself as a clue?

The authors suggest that those messy streaks (called "smears") actually contain a hidden map of how the camera moved through space. Their goal is to extract a mathematical object called a Fundamental Matrix from just that single blurry photo. Think of this matrix as a "motion fingerprint" that tells us the 3D direction the camera was traveling during the split second the photo was taken.

Here is how they solved the puzzle, broken down into simple steps:

1. The "Time Travel" Problem

In normal photography, if you take two photos a second apart, you can see exactly how things moved from Photo A to Photo B. But in a motion-blurred photo, everything is squashed into one image.

The Analogy: Imagine watching a movie played in reverse. If you see a ball rolling, you can't tell from a single blurry frame if it rolled left-to-right or right-to-left. The blur looks the same either way.
The Challenge: Because we don't know which way time is flowing (forward or backward) for each streak, standard math tools used in computer vision break down. They get confused and can't figure out the camera's path.

2. The "Double-Angle" Trick

To fix the confusion about direction, the researchers invented a special way to describe the streaks.

The Analogy: Imagine you have a compass needle. If you spin it 180 degrees, it points the opposite way, but it's still the same needle. Standard math treats "North" and "South" as totally different. The researchers used a "double-angle" trick: they spun the needle twice as fast. Now, whether the original motion was North or South, the "double-angle" version points in the exact same direction. This removes the confusion about which way time is flowing.

3. The "Trust Me" Filter (Uncertainty)

Not all blurry streaks are helpful. Some are caused by objects moving on their own (like a car driving past a stationary building), and some are caused by parts of the image being hidden (occlusion).

The Analogy: Imagine asking a group of people for directions. Some are sure, some are guessing, and some are looking at the wrong street. If you listen to everyone equally, you'll get lost.
The Solution: The computer learns to assign a "confidence score" to every streak. It learns to say, "I'm 90% sure this streak shows the camera moving," and "I'm only 10% sure about this one because it looks weird." When calculating the final motion path, the computer ignores the low-confidence guesses and only listens to the "experts."

4. The Result: A Motion Map

By combining these tricks, the system can look at a single blurry photo and draw invisible lines (called epipolar lines) that show where the camera was pointing at the start and end of the exposure.

The Proof: They tested this on both computer-generated images and real-world photos. The system successfully drew lines that aligned with the background scenery, proving it understood the camera's motion.
Real-World Use: They showed this works for a specific task called motion segmentation. This means the computer can look at a blurry photo and say, "The background is moving because the camera is shaking, but that car is moving differently, so let's highlight it."

What It Can't Do

The paper is honest about its limits. If a scene is chaotic with many people or cars moving in completely different directions at once, the system gets confused and fails to find the camera's path. It also struggles if the camera barely moves at all or if the scene is just a flat wall.

In summary: The paper teaches computers to stop fighting motion blur and start reading it. By treating the blur as a coded message rather than a mistake, they can figure out how a camera moved, even without a second photo to compare it to.

Technical Summary: Robust Fundamental Matrix Estimation from Single Image Motion Blur

Problem Statement

The paper addresses a novel and challenging computer vision task: estimating the fundamental matrix (which encodes relative 3D camera motion) from a single motion-blurred image.

In standard computer vision, the fundamental matrix is derived from correspondences between two distinct views. However, in a single motion-blurred frame, the "two views" are effectively the start and end of the camera's exposure window. The primary challenge is temporal direction ambiguity: a motion blur smear represents a path between two points, but the image alone cannot distinguish which point is the start and which is the end. Consequently, a correspondence $(p, p')$ is ambiguous with $(p', p)$ . This ambiguity renders classic estimation methods, such as the 8-point algorithm or the Gold Standard method, inapplicable, as they rely on known temporal ordering. Furthermore, existing work on motion blur primarily focuses on image de-blurring or optical flow estimation, often treating blur as a nuisance rather than a source of geometric constraints.

Methodology

The proposed framework operates in two stages: Smear Path Estimation and Fundamental Matrix Estimation.

1. Smear Path Estimation

The first stage involves a neural network (based on a U-Net architecture) that predicts per-pixel smear vectors and their associated uncertainty.

Data Generation: Synthetic motion-blurred images are generated by averaging intermediate frames interpolated between two sharp frames using ground-truth optical flow (from the Monkaa and Driving datasets). A semi-real dataset is created from the GoPro dataset, and a real-world dataset is derived from RBI with pseudo-ground truth.
Smear Representation: To resolve the directional ambiguity where a vector $(u, v)$ is indistinguishable from $(-u, -v)$ , the authors employ a double-angle representation. The vector angle $\phi$ is mapped to $2\phi$ , converting the two equivalent cases into a unique vector $(u', v')$ .
Uncertainty and Cross-Checking: The network outputs a variance map $\sigma$ alongside the smear vector. To distinguish reliable smear patterns from unreliable ones (caused by occlusions or objects moving out of frame), a cross-check map ( $m_{cr}$ ) is generated using forward-backward flow consistency between the start and end frames.
Loss Function: A custom loss function is designed to supervise the double-angle vectors on reliable pixels (using Gaussian negative log-likelihood) while penalizing the variance (uncertainty) on unreliable pixels identified by the cross-check map. This encourages the network to assign high uncertainty to ambiguous or occluded regions.

2. Robust Fundamental Matrix Estimation

The second stage estimates the fundamental matrix $F$ from the predicted smear endpoints.

Ambiguity Handling: Since the temporal direction is unknown, a smear endpoint pair $(\bar{p} - \bar{s}, \bar{p} + \bar{s})$ must satisfy the epipolar constraint $x^T F y = 0$ for either the original order or the transposed order. The authors formulate this as a constrained optimization problem where the objective is to minimize the error for either $F$ or $F^T$ :
$F^* = \arg \min_F \| \min(|M_{vec}(F)|, |M_{vec}(F^T)|) \|^2$
subject to $\det(F)=0$ and $\|F\|=1$ .
Robust Sampling: To handle outliers and local motions, the method uses Preemptive RANSAC. Crucially, the sampling process is guided by the uncertainty estimates from the first stage; only smear vectors with low uncertainty (high reliability) are considered.
Symmetric Error Metric: Standard Sampson error cannot be used directly due to the direction ambiguity. The authors introduce a symmetrized Sampson error ($SErrMin$), which takes the minimum error between the hypothesis $F$ and its transpose $F^T$ . This metric allows for robust consensus set definition without knowing the true temporal direction.

Key Contributions

Novel Task Formulation: The paper introduces the task of estimating the fundamental matrix from a single motion-blurred image, addressing the unique challenge of per-smear temporal direction ambiguity.
Robust Estimation Framework: A two-stage pipeline is proposed that combines deep learning for smear extraction with a geometric solver that explicitly accounts for the ambiguity of time direction (allowing for $F$ or $F^T$ ).
Uncertainty-Aware Learning: The authors propose a loss function that learns to predict uncertainty, enabling the system to down-weight unreliable smear predictions (e.g., from occlusions) during the robust estimation phase.
Representation and Annotation: The use of the double-angle representation is shown to be advantageous for learning smear paths, and a procedure using cross-checking maps is presented to generate precise annotations for training.

Experimental Results

The method was evaluated on synthetic (Monkaa, Driving), semi-real (GoPro), and real-world (RBI) datasets.

Smear Estimation: The proposed double-angle representation combined with the cross-check map yielded the lowest End-Point Error (EPE-S) across all datasets. The uncertainty estimates were shown to correlate well with true errors, effectively filtering out unreliable predictions.
Fundamental Matrix Estimation: Using the symmetrized Sampson error and uncertainty-guided RANSAC, the method successfully estimated fundamental matrices that aligned with ground-truth correspondences. The best performance was observed when selecting the top 35% to 45% of smear vectors based on uncertainty.
Downstream Application: The estimated fundamental matrix and smear paths were successfully applied to local motion segmentation, effectively separating objects moving independently from the global scene motion (e.g., a car moving differently from the background) in blurred images.
Limitations: The method struggles in scenes with multiple independent local motions (dynamic scenes) or near-zero camera translation, where the global motion signal is weak or degenerate.

Significance and Claims

The authors claim that their work demonstrates the feasibility of extracting 3D camera motion constraints from a single frame, a task previously considered ill-posed due to temporal ambiguity. By leveraging the geometric constraints inherent in motion blur, the method provides a way to perform visual odometry and structure-from-motion tasks with reduced latency (single-frame input), which is critical for applications like autonomous driving and drone flight. The paper emphasizes that while the result is ambiguous up to a transpose (reflecting the unknown direction of time), this is sufficient to define the epipolar geometry and enable downstream tasks like motion segmentation. The work bridges the gap between treating motion blur as a nuisance to be removed and utilizing it as a rich source of geometric information.