ColonSplat: Reconstruction of Peristaltic Motion in Colonoscopy with Dynamic Gaussian Splatting

This paper introduces ColonSplat, a dynamic Gaussian Splatting framework that achieves superior 3D reconstruction of peristaltic colon motion by preserving global geometric consistency, supported by a new synthetic benchmark dataset called DynamicColon and a critical analysis of existing methods' limitations.

The Gist

Imagine your colon is a long, flexible, rubbery tunnel. Now, imagine a tiny camera traveling through it, taking pictures to help doctors find problems like polyps.

The problem is that this tunnel isn't a static hallway; it's alive. It constantly squeezes, stretches, and ripples (a process called peristalsis) to move things along. This makes creating a 3D map of the inside very difficult.

Here is a simple breakdown of the paper "ColonSplat" and what the researchers achieved:

The Problem: The "Magic Trick" Gone Wrong

Imagine you are trying to build a 3D model of a wiggly worm using a series of photos.

• Old Methods: Previous AI tools tried to build this 3D model, but they got confused by the movement. Instead of realizing the tunnel was actually moving, they tried to trick the computer. They would take a stationary 3D model and just spin it, shrink it, or make parts of it transparent to make it look like it was moving.
• The Result: If you looked at the model from the camera's original path, it looked okay. But if you stepped back and looked at the whole 3D shape from the outside, it looked like a melted, glitchy mess. It didn't respect the laws of physics; it was just a "magic trick" that fooled the camera but failed to show the real shape.

The Solution: ColonSplat (The "Living Sculpture")

The researchers created a new system called ColonSplat. Think of it as a way to build a 3D model out of millions of tiny, glowing, fuzzy balls (called "Gaussians").

Instead of just spinning or shrinking these balls to fake movement, ColonSplat actually moves the balls.

• The Analogy: Imagine a sculpture made of thousands of tiny, floating balloons.
  • Old AI: To make the sculpture look like it's dancing, it would just rotate the whole thing or change the color of the balloons.
  • ColonSplat: It physically pushes the balloons forward, pulls them back, and stretches the whole shape to match the real squishing of the colon. It keeps the "skeleton" of the tunnel intact so that even if you view it from the outside, it still looks like a tube, not a blob.

The New Tool: A "Fake" Colon for Testing

One of the biggest hurdles in medical AI is that you can't easily get a perfect 3D map of a real human colon while it's moving inside a body. You can't see the "ground truth" (the perfect answer) to check if the AI is right.

To fix this, the team built DynamicColon.

• The Analogy: Since they couldn't get a perfect map of a real moving colon, they built a perfect video game version of one. They created a digital colon that moves exactly as a real one does, and they know the exact 3D coordinates of every single point at every single moment.
• Why it matters: This allowed them to test their new AI against a known perfect answer, proving that their method actually understands the geometry, not just the colors.

Why This Matters

1. Better Navigation: If a surgeon is using a robot or a camera to navigate the colon, they need a map that doesn't glitch when the colon moves. ColonSplat provides a stable, reliable map.
2. Real Physics: By forcing the AI to move the 3D points physically rather than just faking it with colors or rotations, the resulting model is much more useful for future medical simulations.
3. The Benchmark: They showed that almost all other current methods fail at this specific task because they treat the colon like a rigid pipe, not a living, breathing tube.

In a Nutshell

The researchers realized that previous AI tried to "fake" the movement of the colon, which caused the 3D map to fall apart. They built a new system (ColonSplat) that actually moves the 3D pieces to match the real squishing motion, and they created a perfect digital test environment (DynamicColon) to prove it works. This leads to clearer, safer, and more accurate 3D maps for doctors to use during colonoscopies.

Technical Summary

Here is a detailed technical summary of the paper "ColonSplat: Reconstruction of Peristaltic Motion in Colonoscopy with Dynamic Gaussian Splatting."

1. Problem Statement

The paper addresses a critical gap in medical image analysis: the accurate 3D reconstruction of colonoscopy data that accounts for peristaltic motion (the continuous, non-rigid deformation of the colon wall).

• Limitations of Existing Methods: Current state-of-the-art dynamic reconstruction methods (based on NeRF or 3D Gaussian Splatting) are designed for general endoscopy or laparoscopy. They struggle in the colon because:
  • Constrained Topology: The colon is a narrow, deforming tube where the camera constantly translates forward, causing previously seen areas to leave the field of view.
  • Lack of Landmarks: The absence of persistent global landmarks makes it difficult to maintain global geometric consistency.
  • Physical Implausibility: Existing methods often fail to update the spatial coordinates ($x, y, z$) of the 3D primitives (Gaussians) to reflect true anatomical motion. Instead, they "cheat" by encoding deformation through changes in rotation, scale, or opacity. This leads to physically implausible reconstructions that look correct from the initial camera trajectory but collapse or produce severe artifacts when viewed from new angles or globally.
• Data Scarcity: There is a lack of clinical datasets with reliable 3D ground truth (point clouds) for every timestep, making rigorous evaluation of global geometric fidelity impossible.

2. Methodology: ColonSplat

The authors propose ColonSplat, a dynamic 3D Gaussian Splatting framework specifically tailored to preserve global structural integrity while modeling complex peristaltic motion.

Core Components

1. Gaussian Representation:

   • The scene is represented by a collection of 3D anisotropic Gaussians.
   • Initialization is performed using a monocular depth estimator (ColonCrafter or AnyDepth) to generate a point cloud from RGB images.

2. Dynamic Deformation Modeling (Geometric):

   • A Multi-Layer Perceptron (MLP), denoted as $\theta$, predicts time-dependent geometric offsets based on the timestep $t$ and the canonical Gaussian parameters.
   • Key Innovation: The model explicitly updates the spatial coordinates ($x$) of the Gaussians to simulate true tissue movement.
   • Constraints: To prevent the model from reverting to "scale/rotation" tricks, the updates for scale ($s$) and rotation ($r$) are clipped to small thresholds. Opacity ($\alpha$) is fixed over time to prevent the model from hiding/revealing Gaussians to fake motion.

3. Color Modeling:

   • A dedicated network branch ($\theta_{color}$) handles appearance variations caused by lighting, fluids, and tissue properties.
   • Color updates are modeled multiplicatively ($c' = c \odot (1 + \Delta c)$) to capture light-surface interactions, utilizing learnable per-Gaussian embedding vectors.

4. Regularization and Loss Functions:

   • K-Nearest Neighbor (KNN) Consistency: To ensure locally coherent non-rigid motion, the model penalizes inconsistencies between a Gaussian and its $K$ nearest neighbors in canonical space. This encourages smooth, tissue-like deformation.
   • Depth Supervision: Uses an $L_1$ loss between rendered depth and monocular depth estimates.
   • Color Regularization: Penalizes spurious color offsets and excessive color variance to prevent motion from being explained by color changes.
   • Total Variation (TV) Loss: Applied to renders to promote spatial smoothness.

3. Key Contributions

1. Benchmark Analysis: The authors conducted a thorough evaluation of state-of-the-art dynamic endoscopic methods on colonoscopic scenes. They demonstrated that these methods fail to model true anatomical motion, relying instead on non-physical parameter adjustments (scale/rotation) that result in geometric inconsistencies.
2. DynamicColon Dataset: To enable rigorous evaluation of global reconstruction (which is impossible with standard clinical video alone), they introduced a synthetic dataset derived from C3VDv2 meshes.
   • It includes ground-truth point clouds and camera trajectories for every timestep.
   • It features views from cameras positioned outside the colon, allowing for the detection of global structural artifacts.
3. ColonSplat Framework: A novel method that successfully captures peristaltic-like motion while maintaining global geometric consistency, outperforming existing baselines in both image quality and geometric fidelity.

4. Results

The method was evaluated on the C3VDv2 (realistic synthetic with artifacts) and the new DynamicColon datasets.

• Quantitative Performance:

  • Image Quality: ColonSplat achieved superior PSNR, SSIM, and LPIPS scores compared to baselines (Endo4DGS, SurgicalGS, Deform3DGS, etc.) on both datasets.
  • Geometric Fidelity (DynamicColon): On the dataset with ground-truth geometry, ColonSplat significantly outperformed all baselines in Chamfer Distance (CD) and Hausdorff Distance (HD95).
    • Example: On DynamicColon, ColonSplat achieved a CD of 0.162 vs. 0.372–0.497 for baselines, and an HD95 of 0.730 vs. 1.017–1.143.
  • Ablation Study: Removing constraints (allowing free optimization of scale/rotation) or KNN regularization led to significant degradation in geometric metrics (e.g., HD95 jumped to 1.033 without KNN) and visual artifacts.

• Qualitative Performance:

  • Visual comparisons show that while baselines produce "mimicked" deformations that look correct only from the training view, they collapse or warp when viewed from outside the colon.
  • ColonSplat maintains a physically plausible, continuous tubular structure even when viewed from external camera angles, effectively capturing the global deformation of the organ.

5. Significance

• Clinical Relevance: Accurate 3D reconstruction of the colon is vital for surgical navigation, polyp localization, and retrospective diagnostics. By modeling true peristaltic motion rather than local overfitting, ColonSplat provides a more reliable foundation for these applications.
• Physics-Based Simulation: The method's ability to enforce physically plausible deformations (via spatial coordinate updates rather than opacity tricks) makes the output suitable for integration with downstream physics-based simulations.
• Evaluation Standard: The introduction of the DynamicColon dataset fills a critical void in the field, providing the first benchmark with ground-truth geometry for evaluating global reconstruction quality in dynamic endoscopy.

In summary, ColonSplat shifts the paradigm from "local appearance matching" to "global geometric consistency" in dynamic endoscopy, solving the specific challenges posed by the constrained, deforming environment of the colon.