💻 computer science

C3VDReg: A Benchmark for Local-to-Local Colonoscopic Registration toward Anatomical Localization

This paper introduces C3VDReg, a new benchmark and dataset derived from the Colonoscopy 3D Video Dataset that evaluates local-to-local point cloud registration for anatomical localization, revealing that high geometric overlap is insufficient for accurate pose recovery due to translation ambiguity in repetitive tubular structures and highlighting the need for stronger anatomical constraints.

Original authors: Linzhe Jiang, Jiayuan Huang, Sophia Bano, Matthew J. Clarkson, Zhehua Mao, Mobarak I. Hoque

Published 2026-08-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Linzhe Jiang, Jiayuan Huang, Sophia Bano, Matthew J. Clarkson, Zhehua Mao, Mobarak I. Hoque

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Colonoscopy is a vital medical procedure for detecting and monitoring colorectal cancer, yet it presents a unique navigational challenge. During the exam, a doctor guides a flexible camera through the colon, seeing only a small, shifting patch of the inner wall at any given moment. To make sense of this journey, doctors rely on preoperative scans, such as computed tomography (CT) images, which provide a complete three-dimensional map of the patient's anatomy. The goal of modern medical technology is to link the live video feed from the camera directly to this static 3D map in real time. This connection would allow the system to tell the doctor exactly where they are, how much of the colon has been examined, and whether they have returned to a previously visited area. Achieving this requires a process called registration, where a computer must align the partial, noisy view from the video with the corresponding section of the detailed 3D model.

However, the inside of the colon is a difficult environment for computers to navigate. Unlike a room with distinct corners or a landscape with unique trees, the colon is a long, smooth tube with repeating folds that look nearly identical over long stretches. When a computer tries to match a small video patch to the 3D map, it often struggles because the surface features are repetitive and the video data is incomplete. A new study introduces a specialized benchmark called C3VDReg to test how well current computer vision methods can solve this specific problem. The researchers created a dataset of over ten thousand pairs of point clouds, which are digital collections of points representing the colon's surface. For each pair, one set of points comes from a video frame's depth data, and the other comes from a 3D model of the colon derived from a CT scan, both captured from the exact same viewpoint. This setup isolates the core difficulty of the task: aligning two partial views of the same anatomy without the distraction of searching for the correct location on the entire organ.

The researchers tested several existing algorithms, ranging from classical geometric methods to modern learning-based systems, against this benchmark. They found that even the most advanced methods failed to reliably align the views. The best-performing system managed to correctly match the position and orientation in fewer than one out of every five test cases under strict criteria. This result was surprising because the researchers had ensured that the two views shared a large amount of visible surface area, with overlap ranging from roughly 74 percent to over 93 percent. In many other fields, such high overlap would guarantee a successful match. The study explicitly rules out the idea that the failure is caused by a lack of shared surface or missing data. Instead, the analysis reveals that the primary obstacle is a type of confusion caused by the colon's repetitive shape. Because the tube looks the same in many places, the computer can easily align the local features correctly but still place the patch in the wrong spot along the length of the colon, sliding it forward or backward by several centimeters.

Further investigation showed that this error is not just a matter of missing information but a fundamental ambiguity in the geometry itself. The researchers broke down the errors into two types: movement along the path of the colon and movement across the wall. They discovered that the algorithms frequently made large mistakes in both directions, with translation errors often exceeding 80 millimeters for the best methods. This means the system might think it is looking at a specific fold when it is actually looking at a similar fold several inches away. The study also highlighted that local visual similarity can be deceptive; a computer might find a match that looks perfect in terms of surface texture and shape but is completely wrong in terms of global location. This phenomenon explains why simply improving the quality of local matching is not enough to solve the problem. The findings suggest that future systems will need to incorporate stronger constraints, such as understanding the sequence of the journey or the overall topology of the organ, rather than relying solely on matching small patches of surface.

The benchmark also examined the trade-off between speed and accuracy. Some methods were extremely fast, processing data in under 70 milliseconds, but they failed to produce a single correct alignment. Others were more accurate but significantly slower, taking nearly 200 milliseconds per frame. The most robust method, which used a transformer-based architecture, achieved the highest success rate but still left the majority of test cases unsolved. The researchers emphasize that their work does not solve the full problem of colon navigation, which involves finding the correct region on the entire organ and then aligning it. Instead, C3VDReg provides a controlled environment to study the intermediate step of aligning two matched views. By isolating this specific challenge, the study demonstrates that current technology is not yet ready for reliable, anatomy-aware guidance in the colon. The results indicate that a breakthrough will require new approaches that can handle the inherent ambiguity of repetitive tubular structures, moving beyond simple surface matching to incorporate broader anatomical context.

Technical Summary: C3VDReg

Problem Statement

The paper addresses the challenge of local-to-local rigid point cloud registration in colonoscopy, a critical step toward anatomy-aware navigation and augmented guidance. While linking intraoperative endoscopic video to preoperative CT-derived anatomy is essential for coverage assessment and revisited-region awareness, standard point cloud registration benchmarks (typically based on CAD objects, indoor RGB-D, or outdoor LiDAR) fail to capture the specific difficulties of the colon.

The colon presents unique geometric and observational challenges:

Locally homogeneous surfaces: The mucosal surface lacks distinct features.
Repetitive structures: Haustral folds repeat over long extents, creating matching ambiguity.
Partial observations: Endoscopic frames capture only a small patch of the surface.
Noisy depth: Reconstructed depth from video is noisy and incomplete compared to CT meshes.

Existing datasets focus on depth estimation, reconstruction, or procedure-level analysis but lack a fixed benchmark for rigid registration between video-derived partial observations and CT-derived anatomy. Furthermore, evaluating "local-to-global" localization conflates two subproblems: retrieving the correct region on a long, repetitive surface and estimating the rigid transformation. This paper argues that to understand registration failures, one must first isolate the local registration component under a controlled protocol where the target region is known.

Methodology: The C3VDReg Benchmark

The authors introduce C3VDReg, a dataset and benchmark derived from the publicly available Colonoscopy 3D Video Dataset (C3VD). The core innovation is a viewpoint-matched partial-to-partial registration protocol.

Data Construction

For each frame in the C3VD dataset:

Source Point Cloud ( $P_S$ ): Generated by reprojecting the released depth map using camera intrinsics and poses.
Target Point Cloud ( $P_T$ ): Generated by raycasting the preoperative CT mesh from the same matched camera pose.
Nature of the Task: This creates pairs where the source and target share the same visible anatomy (removing the "retrieval" subproblem) but differ in sensing modality (video depth vs. CT mesh), density, and noise.

Evaluation Protocol

The benchmark defines a strict "contract" to ensure fair comparison:

Dataset Size: 10,015 pairs (5,952 train, 1,975 val, 2,088 test).
Test Set: Six held-out C3VD sequences (cecum, sigmoid, transverse) to assess anatomy generalization.
Input Budget: 8,192 points per cloud.
Perturbations: Rigid transformations ( $R \in [25^\circ, 90^\circ]$ , $T \in [100, 500]$ mm) applied only to the source cloud.
Metrics:
- Registration Recall (RR@5, RR@10): Fraction of pairs with Relative Rotation Error (RRE) $\le 5^\circ/10^\circ$ and Relative Translation Error (RTE) $\le 5$ mm/$10$mm.
- Marginal Hit Rates: Separate metrics for rotation ( $R_{\le 5}$ ) and translation ( $T_{\le 5}$ ).
- Geometry Diagnostics: Visible nearest-neighbor (NN) distance and 90% trimmed Chamfer distance.
- Overlap Analysis: Ground-truth overlap computed post-alignment to stratify results.

Key Contributions

C3VDReg Dataset: A frame-wise benchmark with 10,015 viewpoint-matched partial-to-partial point cloud pairs, generated via depth reprojection and CT raycasting.
Fixed Evaluation Contract: A reproducible protocol with held-out scenes, specific perturbation ranges, and explicit metrics, addressing the lack of standardized evaluation in medical rigid registration.
Systematic Failure Analysis: The paper provides a deep investigation into why current methods fail, moving beyond simple performance metrics to identify geometric bottlenecks.

Results and Analysis

The authors evaluated classical (ICP) and learning-based baselines (PointNetLK, DCP, RegTR, GeoTransformer) under the C3VDReg protocol.

Main Findings

Low Overall Performance: Even the strongest baseline, GeoTransformer, achieves only 17.43% RR@5 and 35.63% RR@10. Most test pairs remain unsolved.
Translation is the Bottleneck: There is a significant gap between rotation and translation recovery. GeoTransformer recovers rotation correctly in 56.18% of cases ( $R_{\le 5}$ ) but only achieves correct translation in 18.15% ( $T_{\le 5}$ ).
High Overlap Does Not Guarantee Success: Contrary to the assumption that low overlap causes failure, ground-truth overlap between source and target is high (74.3%–93.1%) after alignment. However, registration recall remains low and non-monotonic across overlap quantiles.
Tubular Ambiguity: The primary failure mode is translation ambiguity caused by the smooth, repetitive nature of the colon.
- Axial vs. Radial Errors: Decomposing translation errors reveals that models often slide along the colon trajectory (axial error) or mislocalize across the wall (radial error).
- Local Fit Illusion: Qualitative analysis shows that models can achieve low local residuals (good local fit) while being globally incorrect (large RTE). This is due to the repetitive tubular geometry making displaced patches look locally plausible.

Runtime vs. Robustness

A trade-off exists between speed and accuracy. DCP is the fastest (64.3ms) but has 0% strict recall. GeoTransformer is the most robust but slower (186.8ms). ICP is CPU-based and significantly slower.

Significance and Claims

The paper claims that C3VDReg exposes a fundamental gap between general-purpose registration benchmarks and the specific challenges of colonoscopic geometry.

Challenge to Assumptions: The results challenge the common assumption that increasing overlap or improving local correspondence quality is sufficient for accurate registration. The authors demonstrate that translation ambiguity in repetitive tubular anatomy is the dominant bottleneck.
Need for New Constraints: The findings suggest that future methods require stronger constraints beyond independent local patch matching, such as sequence context, lumen topology, anatomical priors, or retrieval-aware registration.
Role of the Benchmark: C3VDReg is positioned not as a clinical deployment validator, but as a technical intermediate benchmark. It isolates the rigid registration problem to enable systematic comparison of algorithms before tackling the harder problems of full local-to-global retrieval or non-rigid deformation.

The paper concludes that while current state-of-the-art methods show promise in rotation recovery, they fail to reliably recover global position in the colon, highlighting the need for methods that can resolve translation ambiguity in smooth, repetitive structures.