💻 computer science

Spherical-to-ERP Epipolar Rectification for Single-Axis Disparity in 360 Stereo

This paper demonstrates that projecting spherical stereo images to equirectangular format effectively restores single-axis epipolar geometry, enabling the adaptation of the existing RAFT+EACS framework to achieve accurate, real-time disparity estimation for 360-degree imaging.

Original authors: Sahereh Obeidavi, Dieter Landes

Published 2026-06-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sahereh Obeidavi, Dieter Landes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to measure the distance to objects using two eyes (a stereo camera). In the normal, flat world we live in, this is like looking at a picture on a wall: if you shift your eyes left or right, objects move horizontally. This makes it easy for computers to calculate distance because the "shift" only happens in one direction.

But now, imagine those two eyes are actually fisheye lenses that can see a full 360-degree circle around you, like a security camera on a dome. This is great for seeing everything, but it creates a geometric nightmare for computers.

The Problem: The Curved World

In this 360-degree world, the "lines of sight" between the two eyes don't move in straight lines. Instead, they curve along the surface of a sphere (like lines of longitude on a globe).

The Analogy: Imagine trying to draw a straight line between two points on a basketball. If you stretch a string tight, it curves. If you try to measure the distance between two dots on that ball using a flat ruler, your measurement will be wrong because the surface is curved.
The Result: When a computer tries to find matching points between the two fisheye images, the "shift" isn't just left-to-right; it's also up-and-down. It's a messy, two-dimensional jumble. Standard computer vision tools get confused by this and make mistakes.

The Solution: Unfolding the Ball

The authors of this paper propose a clever trick: Unfold the ball.

They take the curved, 360-degree fisheye images and mathematically "flatten" them out onto a rectangular map, similar to how a cartographer turns a globe into a flat map of the world. This specific type of map is called an Equirectangular Projection (ERP).

The Analogy: Think of peeling an orange. The peel is curved and hard to work with. But if you carefully peel it and lay the pieces flat on a table, you can easily measure things with a ruler again.
What Happens: Once the images are "unfolded" into this flat ERP format, the curved lines of sight magically straighten out. The "shift" between the two eyes becomes a simple, straight horizontal (or vertical) line again.

The Engine: Using a Standard Tool

Once the images are flattened, the authors don't need to build a brand-new, complicated computer brain to solve the problem. They use a standard, high-speed tool called RAFT (which is excellent at tracking how pixels move) and a simple filter they previously invented called EACS.

The Analogy: Imagine you have a very fast, high-tech car (RAFT) designed to drive on a straight highway. You can't drive it on a muddy, winding mountain trail (the raw fisheye images). But, if you first build a bridge to turn that trail into a straight highway (the ERP projection), you can drive your fast car right across it without changing a single part of the engine.
The Process:
1. Flatten: Turn the fisheye images into flat ERP maps.
2. Track: Use the standard RAFT tool to see how pixels move between the two flat maps.
3. Filter: Use the EACS filter to ignore any "up-and-down" movement and only keep the "left-and-right" movement, which tells us the distance.

The Results

The paper tested this method on computer-generated 360-degree scenes. Here is what they found:

Accuracy: The method produced very clean and accurate distance maps, almost perfectly matching the ground truth.
Speed: It runs in real-time (about 10 frames per second), which is fast enough for things like robots or autonomous vehicles to use immediately.
Simplicity: They didn't have to train a new, complex AI from scratch. They just added a "flattening" step before using existing, proven tools.

The Bottom Line

This paper proves that you don't need to reinvent the wheel to handle 360-degree cameras. By simply "unfolding" the curved images into a flat format first, you can use standard, fast, and reliable tools to figure out how far away things are, even in a full-circle view. It turns a confusing, curved puzzle into a simple, straight-line problem.

Technical Summary: Spherical-to-ERP Epipolar Rectification for Single-Axis Disparity in 360° Stereo

Problem Statement
Omnidirectional (spherical/fisheye) cameras offer full-surround perception essential for robotics and autonomous navigation, yet their geometric properties violate the fundamental assumptions of classical stereo disparity estimation. In rectilinear stereo, disparity is a one-dimensional quantity aligned with a single epipolar axis. In contrast, raw spherical stereo pairs exhibit curved epipolar trajectories (great-circle arcs) on the viewing sphere. Consequently, correspondences in the image plane manifest as two-dimensional displacements (both horizontal and vertical components), even under ideal calibration. Naïvely applying planar disparity heuristics or standard optical flow magnitudes to these inputs leads to systematic errors and brittle performance, particularly in low-texture regions. While recent deep learning approaches for omnidirectional depth often utilize Equirectangular Projection (ERP) as a stable geometric domain, there remains a gap in understanding whether standard, unmodified optical flow-based pipelines can effectively extract single-axis disparity from spherical inputs after such rectification.

Methodology
The proposed approach adopts a modular pipeline that externalizes geometric complexity through preprocessing rather than learning it within a monolithic network. The methodology consists of three primary stages:

Spherical-to-ERP Rectification: The raw fisheye stereo images are projected into the Equirectangular Projection (ERP) domain. This mapping transforms spherical coordinates (longitude and latitude) into a 2D grid. Crucially, this projection straightens the curved great-circle epipolar curves into straight scanlines aligned with the image axes. Depending on the stereo rig configuration:
- Left–Right rigs: Epipolar lines become horizontal, yielding purely horizontal disparities ( $\Delta u$ ).
- Top–Bottom rigs: Epipolar lines become vertical, yielding purely vertical disparities ( $\Delta v$ ).
  This step restores the one-dimensional disparity structure required by correspondence-based algorithms.
Dense Optical Flow Estimation: Once the images are in the ERP domain, a standard dense optical flow network, specifically RAFT [13], is applied to the rectified stereo pair. RAFT estimates a two-channel flow field representing the 2D displacement between the left and right images.
Epipolar-Aligned Channel Selection (EACS): Building on the authors' prior work [16], the pipeline applies EACS to the flow field. Since ERP rectification ensures that true disparity exists only along the baseline-aligned axis, the method discards the orthogonal flow component.
- For left–right rigs, only the horizontal flow channel is retained.
- For top–bottom rigs, only the vertical flow channel is retained.
  This projection converts the optical flow directly into a single-axis disparity map without retraining the network or designing a specialized spherical architecture.

Key Contributions
The paper makes four specific contributions:

Geometry-Aware Rectification: It demonstrates that projecting fisheye pairs to ERP yields epipolar-aligned scanlines, effectively converting inherently 2D spherical displacements into single-axis disparities.
Direct Disparity Extraction: It establishes that in the ERP domain, disparity can be obtained by simply selecting the baseline-aligned flow component and discarding the orthogonal one, enabling lightweight post-processing without network retraining.
Theory-to-Practice Pipeline: The work formalizes the fisheye-to-ERP mapping and orientation constraints, providing a practical recipe for single-axis disparity extraction from spherical inputs, validated on synthetic spherical data.
Modularity and Deployability: The approach functions as a drop-in preprocessing stage for existing correspondence estimators (including RAFT), facilitating integration into robotics stacks where latency and interpretability are critical.

Experimental Results
The pipeline was evaluated on a synthetic omnidirectional stereo dataset generated at TU Chemnitz, consisting of fisheye RGB views with dense ground-truth disparity. All methods were tested on the same ERP-rectified inputs at 1600×1600 resolution without fine-tuning.

Accuracy: The proposed ERP + RAFT + EACS pipeline achieved a Mean Absolute Error (MAE) of 0.3007 and a Root Mean Squared Error (RMSE) of 0.9470. The D1-all outlier rate was 0.40%, with negligible rates for errors greater than 3 pixels (0.30%) and 5 pixels (0.05%).
Comparison: The method outperformed established RAFT-derived stereo networks (CREStereo, RAFT-Stereo, LEAStereo) in terms of accuracy on this specific ERP-rectified spherical data. For instance, CREStereo showed a higher MAE (0.3581) and a higher D1-all outlier rate (0.98%).
Efficiency: The pipeline achieved a throughput of 10 frames per second (FPS) on an NVIDIA TU102 GPU, significantly outperforming the compared methods (which ranged from 2.0 to 5.0 FPS).
Qualitative Analysis: Visual results indicated that the method produces smooth, structurally consistent disparity maps that preserve thin structures. Errors were primarily concentrated at depth discontinuities and undefined regions (e.g., the sky), consistent with the low outlier rates.

Significance and Claims
The paper claims that explicit geometric rectification via ERP enables standard, mature correspondence algorithms to operate correctly on spherical stereo inputs without the need for bespoke spherical networks. The significance is twofold:

Practical Bridge: It provides a principled pathway for practitioners to reuse existing, high-performance optical flow estimators for 360° imaging, avoiding the complexity of training specialized spherical stereo networks.
Efficiency and Stability: By restoring the one-dimensional nature of disparity, the approach improves numerical stability and computational efficiency. It collapses the correspondence search to a single axis, eliminates the need for memory-intensive 4-D cost volumes, and allows for real-time deployment on commodity hardware.

The authors conclude that while the geometric role of ERP is well-documented, this work validates that the rectified representation is sufficient for single-axis disparity extraction using unmodified flow-based pipelines, offering a straightforward and efficient solution for omnidirectional depth estimation.