💻 computer science

Underwater360: Reconstructing Underwater Scenes from Panoramic Images with Omnidirectional Gaussian Splatting

The paper proposes Underwater360, a physics-informed omnidirectional 3D Gaussian Splatting framework that reconstructs underwater panoramic scenes by performing ray casting in spherical space and decoupling intrinsic radiance from medium effects to achieve superior rendering quality and appearance restoration.

Original authors: Jiangbei Hu, Weichao Song, Shibo Yu, Mohan Wang, Zihan Yi, Rui Wu, Mingkang Xiang, Na Lei, Shengfa Wang, Zhongxuan Luo, Ying He

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiangbei Hu, Weichao Song, Shibo Yu, Mohan Wang, Zihan Yi, Rui Wu, Mingkang Xiang, Na Lei, Shengfa Wang, Zhongxuan Luo, Ying He

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to take a perfect, 360-degree photo of a shipwreck deep underwater. It's a nightmare for two main reasons:

The "Fishbowl" Distortion: If you use a standard camera, you have to take hundreds of overlapping photos to cover the whole scene, and stitching them together underwater is like trying to assemble a puzzle where the pieces keep changing shape.
The "Murky Soup" Effect: Water isn't clear air. It's like looking through a thick, green fog. The water absorbs light (making things look blue or green) and scatters light (creating a hazy glow), which hides the true colors and shapes of the objects.

The paper "Underwater360" proposes a new way to solve this. Think of it as building a virtual, crystal-clear 3D model of an underwater world, even though the photos you fed it were blurry and distorted.

Here is how they did it, using some simple analogies:

1. The "Spherical Balloon" Instead of a Flat Map

Standard 3D computer graphics usually try to flatten a 3D scene onto a 2D screen, like projecting a globe onto a flat map. This causes the edges to stretch and warp (like how Greenland looks huge on a flat map).

The Paper's Fix: Instead of flattening the world, the authors built their system to work inside a giant, transparent balloon. They cast "rays" (like laser beams) directly from the center of the balloon out to the surface. This avoids the stretching and warping that happens when you try to force a 360-degree view into a flat picture. It's the difference between trying to wrap a gift with a flat sheet of paper (wrinkles everywhere) versus using a pre-molded balloon that fits the shape perfectly.

2. The "Magic Filter" (Separating the Soup from the Object)

When you look at a red fish underwater, it doesn't look red; it looks muddy brown because the water has eaten the red light and added a gray haze.

The Paper's Fix: They created a smart "digital filter" that acts like a chef separating ingredients.
- The Chef's Goal: Figure out what the fish actually looks like (the intrinsic color) versus how the water changed it (the attenuation and backscatter).
- How it works: The system learns to guess two things simultaneously:
  1. The "Clean" Image: What the scene would look like if the water were clear.
  2. The "Water" Image: How much the water blocked the light and how much "fog" (backscatter) was added.
- By separating these two, the computer can mathematically "undo" the water's effect, revealing the true colors and sharp details of the underwater scene.

3. The "Smart Paint Sprayers" (Gaussian Splatting)

To build the 3D model, the system uses thousands of tiny, fuzzy 3D "paint splatters" (called Gaussians) instead of rigid 3D blocks.

The Analogy: Imagine trying to recreate a cloud. You could use hard plastic cubes, but it would look blocky. Instead, you use thousands of soft, glowing mist puffs that overlap. The computer figures out exactly where to place these puffs and how transparent they should be to match the photos.
The Innovation: Usually, these "puffs" are designed for clear air. Underwater, the author's system teaches these puffs to also carry a "water map" attached to them. This allows the system to know, "This puff is 5 meters away, so the water has made it look 20% darker and 10% hazier."

The Result: A New "Underwater Map"

The authors didn't just build the tool; they also built the training ground.

They created a new dataset called OmniUW (synthetic scenes made in a video game engine) and Insta360 (real photos taken in a swimming pool).
They tested their method against other top AI models. The results showed that their "Balloon + Magic Filter" approach created much sharper, more accurate 3D models of underwater scenes than previous methods, which often left things blurry or the wrong color.

In short: Underwater360 is a new recipe for 3D reconstruction that uses a spherical camera view to avoid distortion and a smart mathematical filter to "peel away" the murky water, revealing the clear, colorful world hidden underneath.

Technical Summary: Underwater360

Problem Statement
High-fidelity reconstruction of underwater 3D scenes is critical for immersive exploration but faces significant hurdles. Traditional methods relying on standard perspective cameras struggle underwater due to the unstable aquatic environment (currents, suspended particles) and severe optical degradation caused by light absorption and scattering. These factors weaken cross-view correspondences, leading to unreliable camera pose estimation and reconstruction artifacts. Furthermore, conventional narrow-field cameras require dense, overlapping views for coverage, which is difficult to achieve reliably underwater. While 3D Gaussian Splatting (3DGS) has revolutionized terrestrial scene reconstruction, existing underwater adaptations are primarily designed for perspective imagery and fail to address the coupled challenges of spherical projection distortion (inherent to 360° panoramas) and participating-media degradation. Conversely, existing omnidirectional rendering methods typically assume clear-air conditions and do not model underwater light transport.

Methodology
The authors propose Underwater360, a physics-informed omnidirectional 3DGS framework designed to reconstruct underwater panoramic scenes. The framework integrates three core components:

Omnidirectional Gaussian Splatting:
Instead of projecting 3D Gaussians onto a 2D Equirectangular Projection (ERP) image (which introduces geometric distortion), the method performs ray casting directly in spherical camera space. For each pixel in the ERP image, a viewing ray is defined in spherical coordinates. The contribution of each Gaussian is evaluated along these rays within a local tangent plane. This approach, built upon the SPaGS framework, mitigates spherical projection distortions and provides accurate ray-splat intersection depths necessary for subsequent physical modeling.
Physics-Based Appearance-Medium Modeling:
To handle underwater degradation, the authors decouple intrinsic scene radiance from depth-dependent backscatter and attenuation using the Underwater Image Formation Model (UIFM).
- Intrinsic Appearance: A pose-conditioned appearance embedding module adjusts the base color of each Gaussian using a lightweight MLP. This disentangles the intrinsic scene radiance from view-dependent effects and medium-induced shifts, ensuring the base Gaussian representation remains stable.
- Medium Estimation: Two lightweight neural networks estimate the physical parameters of the water.
  - Backscatter-Net: Takes the depth map as input to predict attenuation coefficients for backscatter, modeling the asymptotic saturation of waterlight color at infinite distance and near-field deviations.
  - DeattenuateNet: Takes the depth map and view embedding as input to predict spatially varying attenuation coefficients, accounting for non-homogeneous water conditions.
    The final degraded image is synthesized by combining the clean radiance, estimated attenuation, and backscatter according to the physical model: $I(p) = J(p) \odot A(p) + B(p)$ .
Joint Optimization:
The system optimizes the Gaussian attributes (position, covariance, opacity, color) and the parameters of the appearance and medium networks jointly. The loss function combines pixel-wise L1 loss and structural similarity (L-SSIM) to supervise the reconstruction against the raw underwater panoramic observations.

Key Contributions

Unified Framework: The paper presents the first unified formulation that couples omnidirectional Gaussian rendering with explicit underwater image formation, enabling joint reconstruction of geometry, view-dependent appearance, and medium degradation from panoramic inputs.
Omnidirectional Gaussian Splatting: Introduction of a module that performs spherical ray casting directly in camera space, effectively reducing geometric distortions associated with 2D projection approximations in 360° fields of view.
Physics-Aware Modeling: A novel architecture that explicitly disentangles intrinsic scene radiance from underwater attenuation and backscatter using depth priors and pose-conditioned embeddings, allowing for physically grounded scene restoration.
New Benchmarks: The establishment of a new panoramic underwater benchmark dataset, OmniUW (synthetic, 5 scenes in Unreal Engine 5) and Insta360 (real-world, 5 scenes captured with an Insta360 X4 camera), comprising both synthetic and real-world underwater 360° scenes.

Results
Extensive experiments were conducted on both the synthetic OmniUW and real-world Insta360 datasets, comparing Underwater360 against state-of-the-art NeRF-based and 3DGS-based methods (including underwater-specific and omnidirectional variants).

Quantitative Performance: Underwater360 achieved superior or near-superior performance across most scenes and metrics (PSNR, SSIM, LPIPS) on both datasets. Notably, it demonstrated significant robustness in scenes with severe degradation where other methods struggled or produced unstable results.
Qualitative Improvements: Visual comparisons showed that the method recovers clearer local details and more faithful scene appearances. The decomposition of the scene into clean radiance, attenuation, and backscatter was found to be more spatially coherent and physically plausible compared to prior works.
Ablation Insights: The results indicated that while omnidirectional Gaussian splatting alone is effective for clear scenes, explicit underwater image formation modeling is crucial for handling severe attenuation and backscatter.

Significance and Claims
The authors claim that Underwater360 offers a new perspective on underwater scene reconstruction by unifying panoramic scene representation with explicit physical modeling. By leveraging the rich coverage of omnidirectional imaging and the efficiency of 3DGS, the method provides stronger global constraints for reconstruction in challenging underwater environments. The work is positioned as a foundational step toward robust underwater 3D vision, providing the community with the first benchmark dataset and a framework capable of handling the coupled challenges of spherical distortion and medium degradation. The authors acknowledge limitations regarding static scene assumptions, lack of refraction modeling for camera housings, and instability in extremely turbid or texture-sparse environments, suggesting these as directions for future work.

1. The "Spherical Balloon" Instead of a Flat Map

2. The "Magic Filter" (Separating the Soup from the Object)

3. The "Smart Paint Sprayers" (Gaussian Splatting)

The Result: A New "Underwater Map"

Technical Summary: Underwater360

More like this