Stop Holding Your Breath: CT-Informed Gaussian Splatting for Dynamic Bronchoscopy

1. Problem Statement

The Core Challenge: Bronchoscopic navigation relies on registering real-time endoscopic video to preoperative CT scans. However, respiratory motion causes significant airway deformation (5–20 mm), creating a "CT-to-body divergence" that severely limits localization accuracy.
Current Limitations:

• Breath-hold protocols: Clinicians currently ask patients to hold their breath to match the static CT. This is difficult to reproduce consistently, disrupts workflow, and increases patient discomfort.
• Static assumptions: Existing video-to-CT registration methods often assume a static airway or use rigid models, failing to account for the coherent global displacement of the airway caused by breathing.
• Ambiguity: In bronchoscopy, camera motion and airway deformation are intrinsically coupled, making it difficult to disentangle viewpoint changes from anatomical motion using unconstrained deformation fields.
• Lack of Evaluation: There are no existing datasets with dense ground truth for geometry, pose, and breathing phase to quantitatively evaluate deformable reconstruction methods in this domain.

2. Methodology

The authors propose a framework that eliminates the need for breath-holds by leveraging patient-specific respiratory modeling derived from paired inhale/exhale CT scans (standard for planning).

A. CT-Informed Deformation Space

Instead of learning unconstrained per-frame deformation fields, the method constrains the airway geometry to a 1D subspace defined by the patient's own imaging:

• Input: Paired inspiration ($V_{insp}$) and expiration ($V_{exp}$) CT scans.
• Deformation Field: A fixed displacement field $\Delta = V_{exp} - V_{insp}$ is precomputed via deformable CT-to-CT registration.
• Scalar Parameter: The state of the airway at any time $t$ is governed by a single scalar breathing phase $\hat{\alpha}_t \in [0, 1]$, where 0 is full inspiration and 1 is full expiration.
• Mesh Interpolation: The mesh vertices are interpolated linearly: $V(\hat{\alpha}_t) = V_{insp} + \hat{\alpha}_t \cdot \Delta$. This ensures all reconstructed configurations lie on an anatomically observed trajectory.

B. Mesh-Anchored Gaussian Splatting

The approach builds upon BridgeSplat but adapts it for the bronchoscopic regime:

• Representation: 3D Gaussians are anchored to a parent triangle of the airway mesh using learnable barycentric coordinates.
• Anchoring Benefits: As the mesh deforms according to $\hat{\alpha}_t$, the Gaussians automatically inherit the local stretch, rotation, and translation. This eliminates the need for per-Gaussian deformation parameters and prevents "off-surface drift."
• Optimization: A lightweight estimator infers the breathing phase $\hat{\alpha}_t$ directly from the endoscopic RGB video.
• Activation Function: A cosine-based parameterization with a linear gradient leak is used to estimate $\hat{\alpha}_t$. This avoids gradient vanishing at the extremes (full inhale/exhale), ensuring stable optimization throughout the respiratory cycle.

C. Training Strategy

The optimization follows a three-phase schedule to prevent appearance parameters from absorbing geometric errors:

1. Optimize breathing phase ($\theta_t$) with appearance frozen.
2. Optimize appearance with phase frozen.
3. Joint optimization.
   The loss function combines photometric fidelity (L1 + SSIM) and temporal regularization, removing the need for complex geometric regularization terms (like ARAP) required by unconstrained methods.

3. Key Contributions

A. The RESPIRE Simulation Framework

To enable quantitative evaluation, the authors introduced RESPIRE (Respiratory Endoscopic Scenes with Paired Inspiration-expiration CTs for REconstruction):

• Physically Grounded: Generates realistic synthetic bronchoscopy videos from any pair of inspiration/expiration CTs.
• Dense Ground Truth: Provides per-frame annotations for geometry, pose, breathing phase, and deformation.
• Realism: Uses Physically Based Rendering (PBR) in Blender with subsurface scattering, specular reflections, and variable lighting to mimic clinical tissue appearance and camera behavior.
• Significance: This is the first framework to allow rigorous benchmarking of deformation-aware bronchoscopic reconstruction.

B. CT-Informed Gaussian Splatting

• Novelty: First application of Gaussian Splatting to dynamic bronchoscopy that explicitly models respiratory motion using clinical CT data.
• Efficiency: Replaces complex, unconstrained deformation learning with a single scalar parameter, drastically reducing computational cost.
• Clinical Workflow: Removes the need for breath-hold protocols, allowing continuous navigation during normal breathing.

4. Experimental Results

Experiments were conducted on 9 cases from the COPDGene dataset using the RESPIRE pipeline.

Metric	Performance vs. Baselines (BridgeSplat)
Depth Accuracy	Reduced RMSE by 94% (from 53.8mm to 3.2mm).
Rendering Quality	Improved PSNR by 32% and SSIM by 20% over unconstrained baselines.
Breathing Phase	Achieved MAE of 0.158 and Pearson $r$ of 0.833 in estimating breathing phase from RGB alone.
Target Localization	Achieved 1.22 mm target localization error (contour RMSE: 1.24 mm). This is well within the ~3 mm clinical tolerance for biopsy targeting.
Training Speed	22× faster training time (17.4 min vs. 383.4 min) compared to unconstrained BridgeSplat.

Key Finding: The method successfully recovers geometrically faithful reconstructions and accurate breathing phases without breath-holds, outperforming single-CT baselines that fail to model deformation.

5. Significance and Impact

• Clinical Relevance: By achieving localization accuracy within the 3mm clinical tolerance, this method effectively solves the "CT-to-body divergence" problem caused by breathing. This could eliminate the need for difficult and uncomfortable breath-hold protocols during lung biopsies.
• Workflow Integration: Since paired inhale/exhale CTs are already standard for preoperative planning, the method requires only a modest additional acquisition (the second phase) to enable high-precision navigation.
• Benchmarking: The introduction of RESPIRE fills a critical gap in the field, providing the first dataset with dense ground truth for evaluating dynamic bronchoscopic reconstruction.
• Generalizability: The approach demonstrates that constraining reconstruction to a physiologically valid, low-dimensional subspace (derived from patient data) is superior to learning unconstrained deformations in scenarios with large camera displacements and coherent global motion.

Limitations & Future Work:

• Reliance on accurate airway segmentation (may require manual refinement).
• Current linear deformation model does not capture non-linear pressure-volume dynamics at intermediate phases.
• Assumes known camera pose; future work aims to jointly estimate pose and breathing phase for fully self-supervised navigation.