💻 computer science

Wat3R: Underwater 3D Geometry Learning without Annotations

This paper introduces Wat3R, a cross-domain semi-supervised learning framework that enables 3D geometry reconstruction in underwater environments without any annotated data by adapting air-trained models to unlabeled video through a teacher-student architecture and cross-view consistency loss, alongside the release of a new benchmark dataset called Water3D.

Original authors: Jiangwei Ren, Xingyu Jiang, Zijie Song, Wei Xu, Hongkai Lin, Dingkang Liang, Xiang Bai

Published 2026-07-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiangwei Ren, Xingyu Jiang, Zijie Song, Wei Xu, Hongkai Lin, Dingkang Liang, Xiang Bai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to build a 3D map of a sunken ship, but you are blindfolded, and the water around you is thick with fog, swirling sand, and weird colors that make everything look blurry. That is the daily struggle for computers trying to understand underwater scenes. For years, the best way to teach a computer to see underwater was to show it millions of photos with perfect, hand-drawn 3D labels. But here is the catch: nobody has those labels for the ocean. Making them is impossible because the water is too murky and the light behaves strangely.

Enter Wat3R, a new method that acts like a clever student who learns to dive without ever needing a teacher to hand them a map.

The Problem: The "Land-Only" Brain

Most modern 3D vision models are like students who only studied in a sunny, clear classroom (on land). They are amazing at seeing dry things, but when you drop them into the ocean, they get confused. The water absorbs light, scatters it, and turns everything blue or green. If you try to use a "land-trained" model underwater, it stumbles. And since we can't just hire a team of divers to draw 3D maps of the ocean floor for every video, we can't just retrain these models the old way.

The Solution: A Teacher and a Student in the Deep

The authors of this paper created a system called Wat3R that uses a "teacher-student" trick to solve this without any underwater labels.

Think of it like this:

The Teacher: Imagine a super-smart robot that knows 3D geometry perfectly because it studied millions of clear, dry photos. But it doesn't know how water works.
The Simulation: The researchers take the Teacher's clear photos and digitally "drown" them. They use a physics formula to add fake fog, color shifts, and light scattering, turning the clear land photos into fake underwater ones. Now, the Teacher has a "labeled" dataset of fake underwater scenes.
The Student: This is the model we want to train. It starts by learning from the Teacher's fake underwater photos.
The Real Dive: Then, the Student goes out and watches 5,504 real underwater video clips (about 359,000 images) that have no labels at all. It just watches the fish, the rocks, and the bubbles.

Here is the magic: The Teacher (which is a slightly older, more stable version of the Student) watches the same real videos and guesses what the 3D shapes look like. The Student tries to match those guesses. If the Student gets it right, it learns. If it gets it wrong, it adjusts. This happens over and over, letting the Student learn the "rules" of underwater geometry just by watching real videos, without anyone telling it what the answer is.

The Secret Weapon: "Cross-View Consistency"

Underwater, one single photo might be so blurry you can't see anything. But if you have a video, you have many angles. The paper introduces a special rule called Cross-View Consistency.

Imagine you are looking at a rock through a dirty window. You can't see it well. But if you look at the same rock from a slightly different angle through a cleaner part of the window, you can see it better. Wat3R does this mathematically. If the model is confused by the fog in one view, it looks at the other views in the video to figure out what the rock should look like. It uses the clear parts of the video to fix the blurry parts. This helps the model ignore the "noise" of the water and focus on the actual shapes.

What They Proved (and What They Didn't)

The authors didn't just guess; they tested this rigorously.

The Dataset: They built a new dataset called Water3D, containing 42 real underwater scenes with accurate 3D maps (poses and depths) to test their work. This is a big deal because, as they note, existing underwater datasets are often too small or lack proper 3D labels.
The Results: When they tested Wat3R on standard underwater datasets like Sea-thru and FLSea Stereo, it beat the current best models (like VGGT, DA3, and MapAnything).
- On the Sea-thru dataset, Wat3R reduced the error rate by about 12.1% compared to the previous best model (VGGT) in a 10-view test.
- In terms of depth accuracy, it improved by 23.7% in some metrics.
- Even when looking at just one single photo (monocular depth), Wat3R was better than models specifically trained on single images, proving that the "cross-view" learning helped it understand the water better.

What They Explicitly Ruled Out

The paper is very clear about what doesn't work:

Just enhancing the image first: They tried taking underwater photos, cleaning them up with existing tools (like "Sea-thru" or "PSPL"), and then feeding them to a 3D model. This failed to improve results much. The authors argue that cleaning the image often introduces new errors or inconsistencies that confuse the 3D model. It's better to teach the model to see through the mess, not to try to fix the mess first.
Synthetic data alone: While they used fake underwater data to start, they showed that using only fake data isn't enough. You need the real, unlabeled videos to make the model truly robust.

How Sure Are They?

The authors are confident in their numbers because they tested on four different public datasets plus their own Water3D dataset. They didn't just simulate the results; they measured them against real ground truth where available.

They showed that Wat3R works better in "mild" to "severe" degradation, though they admit that in extremely turbid (muddy) water or with fast-moving objects, the model still struggles because there simply isn't enough visual information to go on.
They explicitly state that their method is the first semi-supervised framework to generalize to underwater scenes without needing underwater 3D annotations.

The Bottom Line

Wat3R is like a diver who learns to navigate the deep ocean not by reading a map, but by watching how the light bends and how objects look from different angles, all while ignoring the fog. It proves that you don't need perfect labels to teach a computer to see underwater; you just need a smart way to let it learn from the chaos of real video. The code and their new Water3D dataset are available for anyone to try, opening the door for better underwater robots, archaeology, and mapping without the impossible task of labeling every drop of water.

Technical Summary: Wat3R – Underwater 3D Geometry Learning without Annotations

Problem Statement
Estimating 3D geometry (camera poses, depths, and point clouds) in underwater environments is hindered by unique physical challenges, including light attenuation, scattering, and view-dependent degradation. These factors cause significant domain shifts compared to terrestrial scenes. Furthermore, the scarcity of large-scale, high-quality 3D annotations for real underwater data prevents the direct training of state-of-the-art (SOTA) feed-forward reconstruction models, which typically rely on massive dense ground truths. Existing methods either fail to generalize due to domain shifts or require impractical amounts of annotated underwater data.

Methodology: Wat3R Framework
The authors propose Wat3R, a cross-domain semi-supervised learning framework designed to adapt the feed-forward 3D reconstruction model VGGT to underwater scenes without requiring any annotated underwater data. The framework operates on a teacher-student architecture (Mean Teacher paradigm) and utilizes two primary data sources:

Synthetic Labeled Data: Existing annotated terrestrial datasets (VGGT's original training set) are augmented with physics-based underwater rendering to simulate various underwater degradations. This initializes strong geometric priors for the underwater domain.
Unlabeled Real Data: A large collection of real-world underwater video footage is used as an unlabeled training set to refine and generalize the model.

Key Technical Components:

Physics-Based Rendering: To generate labeled synthetic data, the authors employ a simplified underwater imaging model ( $I = J e^{-\beta^D z} + B^\infty(1-e^{-\beta^B z})$ ) that simulates direct transmission attenuation and backscatter. Dense depth maps from a monocular model (DA3) are used to render underwater appearances while preserving original geometric annotations for supervision.
Sequence-Level Augmentation: To prevent model collapse and increase geometric diversity on unlabeled video data, the framework employs sequence-level augmentation. This includes shuffling frame orders, random subsampling, rotation, and color jittering, ensuring the model learns robust geometric cues rather than temporal continuity.
Consistency-Aware Training Objective: The total loss function combines three components:
- Supervised Loss: Applied on synthetic data using ground-truth annotations.
- Per-View Consistency Loss: Applied on unlabeled data, treating the teacher's predictions as pseudo-labels.
- Cross-View Consistency Loss: A novel loss designed to address information degradation. It leverages geometric cues from other views to compensate for visibility loss in the current view. It utilizes a static mask (derived from K-means on depth maps and multi-view consistency checks) to filter out dynamic objects and unstable regions (e.g., open water or severe turbidity), ensuring that cross-view supervision is only applied to reliable, static foreground pixels.

Key Contributions

Wat3R Framework: The first semi-supervised framework based on VGGT that generalizes to diverse underwater scenes without requiring underwater 3D annotations. It demonstrates that robust geometry representations can be learned solely from unlabeled real underwater videos.
Cross-View Consistent Loss: A mechanism that aggregates geometric information from multiple views to compensate for the visual degradation caused by water attenuation and scattering, enabling learning from heavily degraded regions.
Water3D Dataset: The construction of a new, diverse underwater multi-view dataset containing 42 scenes with comprehensive 3D annotations (camera poses and depths). This dataset addresses the lack of comprehensive evaluation benchmarks in the field.

Experimental Results
Extensive experiments were conducted on public datasets (Sea-thru, FLSea, SQUID) and the new Water3D dataset across four tasks:

Multi-view Depth Estimation: Wat3R outperforms SOTA methods (including VGGT, DA3, $\pi^3$ , and MapAnything) and two-stage pipelines that combine image enhancement with 3D models. On the Sea-thru dataset, Wat3R achieved a 23.7% improvement in RMSE over VGGT.
Point Cloud Reconstruction: On Water3D, Wat3R demonstrated superior accuracy and completeness compared to baselines, with significant improvements in overall metrics (e.g., +31.6% improvement in overall accuracy over VGGT).
Camera Pose Estimation: Evaluated on SeaThru-NeRF, Wat3R showed improved robustness in pose estimation, achieving a 37.8% improvement in AUC@5° over VGGT.
Monocular Depth Estimation: Despite not being trained with single-image supervision, Wat3R achieved the best performance across four datasets, indicating that the learned geometric priors successfully transfer to single-view inputs.

Significance and Claims
The paper claims that Wat3R provides a practical paradigm for geometric learning in complex environments where annotations are scarce. By eliminating the need for annotated underwater data and leveraging unlabeled real videos, the method achieves robust performance even in poor visibility conditions. The authors emphasize that their approach successfully bridges the domain gap between terrestrial and underwater vision, offering a solution that is both effective and scalable for underwater robotics, mapping, and archaeology. The work highlights that direct adaptation of powerful feed-forward models, guided by consistency constraints and synthetic-to-real transfer, is a viable path forward for underwater 3D perception.