💻 computer science

Angle-I2P: Angle-Consistent-Aware Hierarchical Attention for Cross-Modality Outlier Rejection

The paper proposes Angle-I2P, a novel outlier rejection network for image-to-point-cloud registration that combines scale-invariant angular consistency constraints with a global-to-local hierarchical attention mechanism to significantly improve inlier ratio and registration recall, particularly in scenarios with low initial matching quality.

Original authors: Muyao Peng, Shun Zou, Pei An, You Yang, Qiong Liu

Published 2026-05-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Muyao Peng, Shun Zou, Pei An, You Yang, Qiong Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant 3D puzzle. You have two pieces of information: a flat photo (like a picture of a room) and a cloud of 3D dots (a digital scan of that same room). Your goal is to figure out exactly how the photo and the 3D scan line up so a robot can understand where it is and what it's looking at.

The problem? When you first try to match the photo to the 3D dots, you get a huge mess. You might accidentally match a picture of a chair leg to a picture of a lamp, or a wall to a floor. These are called "outliers" (the wrong matches). If you try to solve the puzzle with these wrong pieces, the whole thing falls apart.

This paper introduces a new tool called Angle-I2P to clean up that mess. Here is how it works, using simple analogies:

1. The "Scale" Problem: The Shrink-Ray Issue

First, the authors have to turn the flat photo into a 3D cloud of dots so they can compare it to the real 3D scan. They use a special camera trick (monocular depth estimation) to guess how far away things are.

The Catch: This trick is like looking at a photo through a "shrink ray." It gets the shape right, but the size is wrong. A table might look like a giant in the photo but a tiny toy in the 3D scan.

Old methods tried to measure the distance between dots to see if they matched. But because the sizes were different, the distances were wrong, and the computer got confused.
Angle-I2P's Fix: Instead of measuring how far apart things are (which changes if you shrink or grow the object), they measure the angle between them.
- Analogy: Imagine two people holding hands. If you shrink the whole room, the distance between their hands gets smaller, but the angle their arms make with their bodies stays exactly the same. Angle-I2P ignores the size and only looks at the angles, making it immune to the "shrink ray" error.

2. The "Zoom-In and Zoom-Out" Strategy

Once they have the angles, they need to filter out the bad matches. They use a two-step attention system, like a detective looking at a crime scene.

The Global View (Zoom Out): First, the system looks at the whole room to understand the big picture. It picks out the most important landmarks (like the corners of the room) to get a sense of the overall shape.
The Local View (Zoom In): Then, it zooms in on small groups of dots to check the fine details.
The "Hierarchical Attention": The system constantly switches between these two views. It asks, "Does this small group of matches make sense with the big picture of the room?"
- Analogy: Imagine a teacher grading a test. First, they look at the whole page to see if the handwriting is messy (Global). Then, they zoom in to check if the math is right (Local). If a student writes a correct answer in the wrong place, the teacher catches it by comparing the local answer to the global layout. Angle-I2P does this to spot matches that look okay locally but are wrong globally.

3. The Result: A Cleaner Puzzle

By combining these "angle-only" measurements with the "zoom-in/zoom-out" checking, Angle-I2P acts like a super-efficient filter. It throws away the wrong matches (outliers) and keeps only the correct ones (inliers).

What the paper found:

They tested this on three different sets of data: standard indoor scenes (7Scenes), a larger set of rooms (RGBD Scenes V2), and a brand new set of rooms they filmed themselves in a lab.
In every test, Angle-I2P was better at cleaning up the matches than previous methods.
Because the matches were cleaner, the final calculation of where the robot is (the registration) became much more accurate.

Summary

Think of Angle-I2P as a smart bouncer at a club.

Old bouncers tried to measure how tall people were to let them in, but the lighting made everyone look different heights, so they let in the wrong people.
Angle-I2P ignores height (scale) and only checks the angle of their posture.
It also has a manager (the hierarchical attention) who checks both the whole crowd and individual groups to make sure everyone fits the vibe.

The result is a much cleaner, more accurate lineup, allowing robots to navigate and interact with the world more reliably.

Technical Summary: Angle-I2P

Problem Statement

Image-to-Point Cloud (I2P) registration is a fundamental task in robotics, enabling applications such as manipulation, grasping, and localization. The core objective is to compute an alignment transformation $T \in SE(3)$ between an image and a point cloud. While existing deep learning methods have achieved promising results by aligning features in a learned representation space, they face significant challenges when the initial set of correspondences contains a low ratio of inliers.

In scenarios with limited training data or significant point cloud noise, conventional Perspective-n-Points (PnP) methods often fail to produce accurate results due to the prevalence of incorrect pixel-to-point correspondences. Existing outlier rejection approaches, such as linear classifiers (e.g., GoMatch) or graph-based methods (e.g., GraphI2P), struggle to effectively filter these mismatches. Specifically, methods relying on monocular depth estimation to bridge the modality gap suffer from inherent scale ambiguity, which compromises the performance of traditional spatial consistency constraints used to distinguish inliers from outliers.

Methodology

The authors propose Angle-I2P, an outlier rejection network designed to leverage angle-consistent geometric constraints and a hierarchical attention mechanism. The pipeline consists of three primary stages:

1. Estimated Point Cloud Generation and Scale Alignment

To address the modality gap between images and point clouds, the method converts the input image into a point cloud using a pre-trained monocular depth estimator (Depth Anything V2).

Back-Projection: The image is back-projected into 3D space using camera intrinsics.
Scale Estimation: Recognizing that monocular depth estimation introduces scale ambiguity ( $s$ ) and bias ( $t$ ), the network estimates a global average scale factor ( $s_{est}$ ) by comparing the distances of points from their respective centroids in the estimated point cloud versus the target point cloud.
Rescaling: The estimated point cloud is rescaled to align globally with the target point cloud, mitigating the impact of scale errors on subsequent geometric computations.

2. Angle-Based Spatial Consistency

Traditional spatial consistency constraints based on Euclidean distances are sensitive to scale errors. Angle-I2P introduces a scale-invariant geometric constraint based on angular consistency:

Centroid Translation: Points are translated to a coordinate system centered at their respective centroids to eliminate translational degrees of freedom.
Angular Metric: For any pair of matching points, the method computes the cosine of the angle between the vectors originating from the centroid to the points.
Scale Invariance: The authors prove that the cosine of the angle between these vectors is independent of the scale factor $s$ . This allows the network to compute spatial consistency ( $\theta_{ij}$ ) using the estimated point cloud without being compromised by scale errors. If the angular difference between the estimated and target point clouds is small, the pair is likely an inlier; otherwise, it is an outlier.

3. Angle-Consistent-Aware Hierarchical Attention

To effectively filter outliers, the network employs a Global-to-Local Hierarchical Attention mechanism:

Graph Construction: Two graphs are constructed: a local graph ( $G_L$ ) based on $k$ -nearest neighbors and a global graph ( $G_G$ ) based on top-scoring keypoints representing global scene geometry.
Feature Integration: The network utilizes self-attention and cross-attention mechanisms to refine features. Global and local spatial consistency scores (derived from the angle-based metric) are used to reweight the attention scores.
Classification: The refined features, which integrate both global and local geometric context, are passed through a Multi-Layer Perceptron (MLP) to generate confidence scores for inlier/outlier classification.

Key Contributions

Scale-Invariant Geometric Constraint: The paper proposes a novel method to calculate spatial consistency using angular metrics rather than Euclidean distances. This explicitly addresses the scale ambiguity inherent in monocular depth estimation, providing a robust geometric constraint for rigid transformations.
Hierarchical Attention Mechanism: The authors design an Angle-Consistent-Aware Hierarchical Attention module that integrates global and local geometric information. This allows the model to filter geometrically inconsistent matches more effectively than methods relying solely on local context.
End-to-End Outlier Rejection Framework: Angle-I2P provides a complete pipeline for I2P outlier rejection that transforms the problem into 3D space, aligns scales, and applies geometric constraints to improve the Inlier Ratio (IR) and Registration Recall (RR).

Experimental Results

The method was evaluated on three benchmarks: 7Scenes, RGBD Scenes V2, and a self-collected dataset.

7Scenes: Angle-I2P achieved a state-of-the-art Inlier Ratio (IR) of 49.5%, outperforming the second-best method (GraphI2P) by 4.2%. It also achieved a Registration Recall (RR) of 78.5%, surpassing GoMatch by 2.3%.
RGBD Scenes V2: In cross-scene evaluation, the method improved IR by 5.9% and RR by 0.9% over the second-best baseline (GraphI2P).
Self-Collected Dataset: Demonstrating practical potential, the method achieved an IR of 16.8% and an RR of 47.8%, outperforming GraphI2P by 6.6% in RR.

Ablation Studies confirmed the necessity of each component:

Removing scale alignment led to a drop in IR and RR.
Replacing angle-based consistency with distance-based consistency degraded performance, highlighting the importance of scale invariance.
Removing the global-to-local cross-attention or the reweighting mechanism significantly reduced the model's ability to distinguish inliers, particularly in rigid scenarios where global context is crucial.

Significance and Claims

The paper claims that Angle-I2P offers enhanced potential for practical deployment in real-world scenarios where initial matches often contain significant noise and outliers. By transforming the matching problem into 3D space and utilizing angle-consistent constraints, the method effectively minimizes the modality gap and overcomes the limitations of scale ambiguity that plague existing depth-based approaches.

The authors position their work as a solution to the specific challenge of eliminating incorrect image-to-point cloud matches when inlier ratios are low. They assert that their approach achieves state-of-the-art performance across diverse benchmarks, demonstrating superior generalization in few-shot and cross-scenario settings. The paper concludes that this framework contributes to advancements in image-point cloud mismatch removal, thereby improving the robustness of downstream robotic tasks like localization and manipulation.

The authors modestly acknowledge limitations, noting that the analysis of scale errors remains relatively coarse and that the coarse-to-fine matching strategy of the underlying feature extractor (2D3D-MATR) can lead to clustered matches, which improves IR but may not always yield proportional gains in RR. They identify these areas as targets for future investigation.