💻 computer science

ISOPoT: Imaging Sonar Odometry by Point Tracking

This paper introduces ISOPoT, a robust imaging sonar odometry method that leverages modern multi-frame point tracking techniques to overcome the noise and lack of semantic structure in sonar images, demonstrating superior performance over state-of-the-art methods on both real-world and benchmark datasets.

Original authors: Jaša Samec, Vid Rijavec, Marko Peljhan, Aleksander Grm, Andrej Androjna, Danijel Skočaj, Matej Dobrevski

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jaša Samec, Vid Rijavec, Marko Peljhan, Aleksander Grm, Andrej Androjna, Danijel Skočaj, Matej Dobrevski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to navigate a submarine through a murky, foggy ocean where you can't see anything with your eyes. You have a special "acoustic camera" (a sonar) that sends out sound waves and listens for echoes to create a picture of the world. But here's the problem: these sonar pictures are incredibly noisy, full of static, and look nothing like the clear photos we take with regular cameras. They are like trying to read a map drawn in sand while a storm is blowing.

For a long time, robots tried to navigate these waters by looking for specific "landmarks" (like a rock or a pole) in one picture, finding the same rock in the next picture, and guessing how far they moved. The authors of this paper call this "pairwise matching." They argue that this method is like trying to navigate a foggy city by only looking at one street sign at a time; if the sign is blurry or looks like another sign, you get lost.

The New Idea: ISOPoT

The team introduces a new system called ISOPoT (Imaging Sonar Odometry by Point Tracking). Instead of looking at two pictures and trying to match single dots, ISOPoT watches a whole movie clip of the sonar feed and tracks many dots simultaneously over time.

Here is how it works, using simple analogies:

1. The "Crowd Watcher" (Point Tracking)

Imagine you are at a crowded party in a dark room. If you try to follow just one person from one side of the room to the other, you might lose them in the crowd. But if you watch the movement of everyone in the room at once, you can see the general flow of the crowd. Even if one person disappears, the movement of the others tells you where the crowd is going.

ISOPoT does this with sound. It picks hundreds of tiny points across the sonar image and tracks their paths across several frames of video at once. It uses a smart AI model (called TAPNext) that is really good at guessing where a dot will be next, even if the picture is fuzzy.

2. The "Grid Manager" (Keeping Order)

Sometimes, the AI might pick too many dots in one corner of the image and none in the other. This is like having a crowd of people all standing in one corner of a room while the rest of the room is empty; you can't tell how the whole room is moving.

To fix this, ISOPoT uses a Grid Point Manager. It divides the sonar image into a 16x16 checkerboard. It makes sure there are only a few dots in each square. If a square gets too crowded, it tosses out the extras. This ensures the robot gets a balanced view of the entire underwater scene, not just one noisy corner.

3. The "Fine-Tuner" (Refinement)

The AI is great at guessing, but it's not perfect. It might say, "That dot moved a little to the left," when it actually moved a little to the right.

ISOPoT has a Match Refinement stage. Think of this like a detective double-checking the clues. Once the AI makes a rough guess about how the robot moved, the system zooms in on the specific areas where the dots moved. It compares the texture of the sound waves in those spots to make a final, precise calculation. This turns a "good guess" into a "solid fact."

4. The "Safety Net" (Optional Helpers)

The system is designed to work using only the sonar (like navigating by sound alone). However, if the robot also has other sensors (like a compass or a speedometer), ISOPoT can use them as a safety net. If the sonar gets confused by a weird echo, the system can say, "Hey, the compass says we are still heading North, so let's trust that for a moment."

Why It Matters

The authors tested this on two very different underwater datasets:

Aracati 2017: A dataset full of confusing "ghost" images (reflections that look like real objects) and stripes of noise.
Portoroz 2025: A dataset where the bottom of the sea is just a flat, sandy texture with very few distinct landmarks.

In both cases, ISOPoT was much better than previous methods.

In the noisy, ghost-filled water, old methods got confused by the fake reflections. ISOPoT ignored the noise because it was watching the flow of many points, not just matching single dots.
In the flat, sandy water, old methods failed because they couldn't find any "landmarks" to grab onto. ISOPoT succeeded because it tracked the subtle, shifting patterns of the sand itself, like watching the ripples in a stream.

The Bottom Line
ISOPoT is like switching from trying to navigate by spotting individual street signs in a fog (which often look alike or are missing) to watching the movement of the entire crowd of people in the fog. By tracking the flow of many points over time, the robot can figure out exactly where it is going, even in the murkiest, most confusing underwater environments.

Technical Summary: ISOPoT – Imaging Sonar Odometry by Point Tracking

Problem Statement
Reliable navigation in underwater environments is hindered by the attenuation of electromagnetic signals, rendering GNSS and optical cameras ineffective in turbid or low-visibility conditions. Forward-looking imaging sonars are the primary alternative for long-range perception; however, their imagery presents unique challenges for odometry. Sonar frames suffer from speckle noise, sensor artifacts, weak local texture, and strong appearance variations due to viewpoint and insonification angles. Consequently, standard computer vision pipelines that rely on detecting, describing, and matching pairwise local keypoints often fail. These methods are brittle in low-structure regions and prone to globally inconsistent motion estimates. Furthermore, the scarcity of high-quality labeled underwater datasets with accurate ground truth exacerbates the difficulty of training and evaluating robust systems.

Methodology
The authors propose ISOPoT (Imaging Sonar Odometry by Point Tracking), a pipeline designed to estimate planar SE(2) motion from a sliding window of forward-looking sonar images. Instead of relying on pairwise keypoint matching, ISOPoT leverages modern "Tracking Any Point" (TAP) techniques to establish multi-frame correspondences.

The pipeline operates as follows:

Sliding Window Processing: The system processes a window of sonar images ( $W$ ), consisting of overlapping frames ( $B$ ) from the previous iteration and new frames ( $N$ ). This overlap acts as a warm-up period to suppress transient prediction errors common in sonar data.
Query Point Initialization: Query points are initialized using a Sobel-based detector on the first frame of the window, combined with reliable inliers carried over from the previous window.
Grid Point Manager (GPM): To ensure spatial coverage and prevent clustering, a GPM divides the image into a $16 \times 16$ grid, limiting the number of query points per cell. It filters out outliers, points predicted as invisible, and points outside the valid scan region.
Point Tracking (TAPNext): The selected query points are tracked across the entire window using TAPNext, a state-of-the-art tracker. TAPNext predicts trajectories and visibility flags for each point, exploiting temporal context and global image information to maintain correspondences despite local ambiguities.
Coarse Motion Estimation: For each target frame, a 2-D Euclidean transform is estimated from the tracked points in the reference frame ( $F_1$ ) to the target frame ( $F_t$ ) using RANSAC. This approach estimates motion relative to a common reference rather than consecutive frames, improving robustness against drift.
Match Refinement: A lightweight refinement stage addresses residual per-point errors. It propagates query points to target frames using the coarse transform, extracts dense features via a ResNet50 backbone, and correlates reference descriptors with search regions to generate heatmaps. A global optimizer then maximizes the summed correlation across all reliable points to refine the single global motion hypothesis.
Optional Auxiliary Correction: While the core method is sonar-only, an optional stage allows for the fusion of external odometry and magnetometer data. This is achieved by weighting the sonar estimate based on the inlier ratio of the RANSAC fit, replacing the sonar yaw with magnetometer heading when available to correct long-term drift.

Key Contributions

Paradigm Shift: The paper demonstrates that modern tracking-any-point models transfer effectively to forward-looking sonar imagery. It establishes that multi-frame point tracking is a more robust correspondence primitive for sonar odometry than traditional pairwise keypoint matching.
ISOPoT Pipeline: The authors introduce a complete pipeline combining TAPNext-based tracking, grid-based point management, and a correlation-based global refinement stage. This combination yields stable correspondences in cluttered, weakly textured, and artifact-heavy sequences.
Comprehensive Evaluation: The method is evaluated on two distinct datasets: the public Aracati 2017 dataset (characterized by persistent stripe artifacts and reflections) and a new Portoroz 2025 dataset (characterized by seabed-dominated scenes with weak local texture).

Results

Aracati 2017: In sonar-only scenarios, ISOPoT substantially outperforms the state-of-the-art SONIC method, achieving significantly lower Absolute Trajectory Error (ATE) and Relative Error (RE) across all test sections. Even when compared to DISO (which relies on auxiliary odometry and magnetometers), ISOPoT achieves competitive or superior results, particularly in relative translation error.
Portoroz 2025: In seabed-dominated environments where local keypoint detectors (like AKAZE) often fail to find sufficient features, ISOPoT remains stable. It achieves substantially lower ATE and RE than both SONIC and DISO, despite operating without auxiliary sensors while DISO utilizes them.
Ablation Studies: Removing the match refinement stage caused the largest degradation in performance, confirming the necessity of converting coarse tracks into a global motion estimate. The overlap buffer was found to be critical for long-term consistency, while the Grid Point Manager improved stability in point-dense regions.

Significance and Claims
The paper claims that ISOPoT addresses the fundamental mismatch between sonar imagery characteristics and the pairwise matching paradigm. By shifting to multi-frame point tracking, the method provides a more natural and robust way to handle the ambiguity and noise inherent in underwater acoustic data. The authors assert that ISOPoT consistently outperforms prior state-of-the-art methods in both sonar-only and multi-sensor settings, demonstrating that improving the front-end correspondence mechanism is beneficial even when auxiliary motion information is available. The work highlights the potential of leveraging temporal consistency and global information to solve the challenging problem of underwater navigation without relying on expensive or difficult-to-obtain ground truth for training.