💻 computer science

Opti-Acoustic Scene Reconstruction in Highly Turbid Underwater Environments

This paper presents a real-time, open-source opti-acoustic scene reconstruction method that fuses sonar range data with camera elevation information to enable robust underwater navigation in highly turbid environments where traditional vision-based approaches fail.

Original authors: Ivana Collado-Gonzalez, John McConnell, Paul Szenher, Brendan Englot

Published 2026-03-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ivana Collado-Gonzalez, John McConnell, Paul Szenher, Brendan Englot

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a 3D model of a shipwreck or a pier, but you are doing it while wearing thick, foggy goggles in a muddy swimming pool. This is the daily reality for underwater robots (AUVs) trying to navigate and work in "turbid" (murky) water.

This paper presents a clever new way for these robots to "see" and map their surroundings, even when the water is so dirty that cameras are useless and sonar is too blurry.

Here is the breakdown of the problem and their solution, using some everyday analogies.

The Problem: The "Blind" and the "Deaf"

Underwater robots usually have two main senses, but both have a major flaw in murky water:

The Camera (The Blind Artist):
- How it works: It takes pictures to see shapes and colors.
- The Flaw: In muddy water, light scatters like a flashlight in a foggy room. The image becomes a blurry, greenish soup. The robot can't find "features" (like corners or edges) to build a map. It's like trying to draw a portrait of a person through a thick, dirty window.
- The Missing Piece: Even if the camera could see, a single camera doesn't know how far away things are (depth). It's like looking at a flat painting; you can't tell if a car is 10 feet away or 100 feet away.
The Sonar (The Deaf Architect):
- How it works: It sends out sound waves (pings) and listens for echoes to measure distance. It works perfectly in mud because sound doesn't get blocked by dirt like light does.
- The Flaw: Sonar images are very low-resolution and "flat." It's like looking at a shadow on a wall. You know something is there, but you don't know if it's a tall, thin pole or a short, wide box. The sonar knows the distance but loses the height (elevation) information.

The Solution: The "Handshake"

The authors (Ivana, John, Paul, and Brendan) created a system that forces the Camera and the Sonar to hold hands and help each other. They call this "Opti-Acoustic Scene Reconstruction."

Instead of trying to find tiny, sharp points (like a corner of a brick) in the blurry camera image—which is impossible in mud—they changed the strategy entirely.

The Analogy: The Silhouette and the Ruler
Imagine you are in a dark room with a foggy window (the camera) and a laser pointer (the sonar).

The Sonar shoots a laser beam and says, "I hit something 5 meters away." But it doesn't know if that object is on the floor or the ceiling.
The Camera looks at the foggy window and says, "I can't see the details, but I can see a big, dark blob in the middle of the view."

The Magic Trick:

Ignore the Details: The robot stops trying to find sharp corners in the blurry photo. Instead, it looks for big blobs (regions of interest). It asks, "Where is the big dark shape?"
The Shadow Match: The robot takes the "5 meters away" data from the sonar and projects it onto the camera's view. It asks, "Does the sonar's 5-meter hit line up with the big dark blob the camera sees?"
The Fusion:
- If they overlap, the robot says, "Aha! The sonar told me the distance, and the camera told me the height (because the blob is high up in the image)."
- By combining the Sonar's "Ruler" (distance) with the Camera's "Silhouette" (height), the robot can build a full 3D shape.

Why This is a Big Deal

No Training Needed: Many modern AI solutions require "teaching" the computer with thousands of examples. This method is a simple, logical algorithm. It doesn't need to be "trained"; it just follows the rules of geometry.
Real-Time: It happens fast enough for a robot to use while moving.
Single Shot: You don't need to drive around an object 10 times to build a map. The robot can do it from just one spot.

The Results: From the Lab to the Marina

The team tested this in two ways:

The "Muddy Pool" (Lab): They took clear photos of a model pier and digitally added "mud" to them to simulate different levels of pollution.
- Result: Traditional camera methods failed completely in the "mud." The sonar-only method worked but looked like a flat, blurry shadow. Their new method built a clear, 3D model that looked just like the real object, even in the worst "mud."
The Real Marina (Field Test): They took a robot to a real, dirty marina in New York.
- Result: The camera couldn't see anything. The sonar saw the general shape but missed small details (like small pipes). Their new method successfully mapped the wooden pilings and the corrugated sea walls, capturing details the sonar alone missed.

The Bottom Line

This paper is about teaching robots to be smart detectives rather than just high-tech cameras. Instead of demanding a perfect, clear picture, the robot uses the "rough sketch" from the camera and the "distance measurement" from the sonar to solve the puzzle of what the underwater world looks like.

It's a robust, open-source solution that could help robots clean up oil spills, inspect underwater bridges, or explore shipwrecks in waters that are currently too dirty for them to navigate safely.

1. Problem Statement

Autonomous Underwater Vehicles (AUVs) require precise scene geometry reconstruction for close-range manipulation and navigation, particularly in highly turbid environments (e.g., sediment-laden or polluted waters). Existing reconstruction methods face significant limitations in these conditions:

Vision-based methods (Monocular/Stereo): Rely on salient point features and consistent lighting. In turbid water, light scattering causes blurring, halo effects, and color distortion, making feature detection unreliable. Furthermore, monocular systems suffer from scale ambiguity.
Sonar-based methods: While robust to turbidity and lighting, imaging sonars provide 2D projections of 3D spaces with elevation ambiguity (they measure range and bearing but lack precise vertical elevation data). They also suffer from low resolution, making them unsuitable for detecting fine features or close-range objects critical for manipulation.
Existing Fusion Methods: Most current opti-acoustic approaches rely on visual features as the primary data source, using sonar only for refinement. These fail in turbid waters where visual features are absent. Additionally, many fusion methods require complex training (Machine Learning) or multiple viewpoints/movement, which is not always feasible in space-constrained or close-range scenarios.

The Core Challenge: How to fuse sonar and monocular camera data to reconstruct 3D scenes in real-time within highly turbid water, without relying on unreliable visual point features or requiring extensive training data.

2. Methodology

The authors propose a real-time opti-acoustic reconstruction framework that treats sonar data as the foundation and uses visual data to resolve elevation ambiguity. The pipeline consists of four main stages:

A. Optical Image Segmentation (Region-Based)

Instead of detecting point features (which fail in turbid water), the algorithm identifies Regions of Interest (ROIs) corresponding to objects:

Preprocessing: Apply Gaussian filtering to reduce noise and adaptive thresholding to handle non-uniform lighting.
Segmentation: Use the Watershed algorithm to segment the image into independent regions ( $R$ ).
Filtering: Discard small regions (noise/particles) below a pixel threshold ( $T$ ), retaining only significant object regions.

B. Sonar Processing

Feature Detection: Apply the SOCA-CFAR (Constant False Alarm Rate) filter to the sonar intensity array to remove noise and "second returns" (false echoes).
Clustering: Retain only the closest return for each bearing angle (assuming foreground objects are closest). Cluster these points using DBSCAN (Density-Based Spatial Clustering) to form distinct sonar clusters ( $C$ ).

C. Sonar-to-Image Projection

To resolve the elevation ambiguity of the sonar:

Vertical Expansion: For each sonar return (defined by range $r$ and bearing $\theta$ ), the algorithm projects the point to all possible elevation angles ( $\phi$ ) within the sonar's vertical aperture.
Mapping: These potential 3D points are transformed into the camera frame and projected onto the 2D image plane. Due to the fixed calibration, a single sonar beam appears as a vertical line in the camera image.

D. Matching and 3D Reconstruction

Overlap Matching: The algorithm matches segmented image regions ( $R$ ) with sonar clusters ( $C$ ) if they spatially overlap ( $C \cap R \neq \emptyset$ ). If multiple clusters overlap, the closest one is selected.
Elevation Resolution: For a matched region, the algorithm expands the sonar field of view to match the shape of the visual region.
Depth Calculation: To determine the precise 3D coordinates:
- Range ( $r$ ): Derived from the original sonar return.
- Elevation ( $\phi$ ): Determined by the vertical position of the pixel in the camera image.
- Distance ( $z_c$ ): The mean distance of the sonar beam points within the overlap area is computed and applied to all pixels in that column of the image region.
Output: A dense set of 3D points in Euclidean space, combining sonar range with visual elevation.

3. Key Contributions

Turbidity-Optimized Fusion: The first method to specifically merge sonar and monocular camera data by using sonar as the primary geometric source and visual data solely for elevation, making it robust in conditions where visual features are non-existent.
Region-Based Matching: Avoids the failure-prone step of point-feature detection in turbid water by matching image regions to sonar clusters.
Real-Time, Single-Pose Operation: The algorithm operates in real-time and can reconstruct scenes from a single robot pose, eliminating the need for multi-view movement or complex SLAM trajectories required by other methods.
No Training Required: Unlike deep learning approaches (e.g., AONeuS, Gaussian Splatting), this method is geometric and does not require training on limited or specific underwater datasets.
Open Source: The code is released to facilitate reproducibility.

4. Experimental Results

The authors validated the approach using a BlueROV2 equipped with an Oculus M750d sonar and a low-light camera.

A. Indoor Tank Experiments (Simulated Turbidity)

Setup: Tested against four baselines (Stereo Sonar, ORBSLAM3, LSD SLAM, COLMAP) across four water types (Type I to Type 9C, representing clear to highly turbid).
Findings:
- Vision-only methods (ORBSLAM3, LSD SLAM, COLMAP): Performance degraded significantly as turbidity increased. In highly turbid water (Type 7C/9C), they failed completely (0 voxel coverage) or produced sparse, inaccurate clouds.
- Stereo Sonar: Maintained stability but suffered from low coverage and inability to capture fine details (e.g., small pipes, diagonal beams).
- Proposed Method: Demonstrated stable performance across all turbidity levels. As turbidity increased, its coverage advantage over vision-based methods grew significantly. It achieved lower absolute distance errors and captured fine geometric details (like corrugated walls) that sonar alone missed.

B. Outdoor Field Experiments (Marina Environment)

Setup: Deployed at the US Merchant Marine Academy marina (naturally turbid). Compared only against Stereo Sonar, as vision-based methods failed to find features.
Structures: A pier with wooden pilings and a corrugated seawall.
Findings:
- Coverage: The proposed method achieved ~6x to 8x higher voxel coverage than Stereo Sonar (e.g., 24,956 vs. 3,867 voxels for the pier).
- Detail: Successfully reconstructed small pipe structures and diagonal beams that were completely missed by the sonar-only approach.
- Geometry: Accurately recreated the corrugated geometry of the seawall, expanding the vertical field of view beyond the sonar's native limits.

5. Significance and Limitations

Significance:
This work addresses a critical gap in underwater robotics: enabling close-range manipulation and navigation in environments where neither pure vision nor pure sonar is sufficient. By leveraging the complementary strengths of both sensors (Sonar for range, Camera for elevation) without relying on fragile visual features, it offers a robust solution for real-world operations in polluted or sediment-heavy waters.

Limitations:

Assumption of Constant Distance: The method assumes that for a given sonar bearing, the distance is constant across the vertical span of the matched visual region. This limits effectiveness in scenes with highly complex, non-uniform geometries where a single sonar beam might hit multiple objects at different depths within the same visual column.
Close-Range Focus: The approach is optimized for close-range interactions; performance at very long ranges may be constrained by sonar resolution and beam divergence.

In conclusion, this paper presents a practical, real-time solution for underwater scene reconstruction that significantly outperforms state-of-the-art vision and sonar-only methods in turbid conditions, paving the way for more reliable AUV autonomy in challenging environments.