DeblurSplat: SfM-free 3D Gaussian Splatting with Event Camera for Robust Deblurring

Technical Summary of DeblurSplat: Traditional SfM-free 3D Gaussian Splatting with Event Camera for Robust Deblurring

Problem Statement
Existing 3D reconstruction and novel-view synthesis (NVS) methods, such as 3D Gaussian Splatting (3D-GS) and NeRF, typically assume high-quality inputs captured under ideal conditions. In real-world scenarios involving motion blur (caused by low light or long exposures), these methods suffer significant performance degradation. Motion blur smears edges and flattens textures, confusing feature detection and matching processes. Consequently, traditional Structure-from-Motion (SfM) pipelines (e.g., COLMAP) fail to estimate accurate camera poses or generate reliable initial point clouds. These pose errors propagate into the final reconstruction, resulting in distorted geometry, incomplete surfaces, and artifacts. While event cameras offer high dynamic range and no motion blur, existing event-based deblurring 3D reconstruction methods still rely on traditional SfM for initialization, making them susceptible to the same failure modes when blur is severe.

Methodology
The authors propose DeblurSplat, a traditional SfM-free framework for robust motion deblurring in 3D Gaussian Splatting. The pipeline integrates dense stereo learning and event camera data to bypass the need for accurate initial pose estimation from blurred images. The method consists of three core components:

1. SfM-free Initialization via DUSt3R:
   Instead of using COLMAP, the method leverages the pretrained dense stereo module DUSt3R. DUSt3R directly regresses 3D coordinates and confidence estimates from blurred images using a transformer-based architecture that captures global structural layouts. This approach avoids the accumulation of errors associated with inaccurate camera pose estimation in blurred conditions. The authors note that while motion blur acts as a low-pass filter degrading high-frequency details, DUSt3R preserves mid-to-low frequency geometric statistics (global layout and relative depth), providing a structurally coherent, albeit noisy, prior.

2. Confidence Balanced Sampling:
   To utilize the point clouds generated by DUSt3R effectively, the authors introduce a Confidence Balanced Sampling strategy. Naive strategies of using all points or only high-confidence points are suboptimal: high-confidence points often cluster in overlapping regions, while low-confidence points still encode essential spatial structure. The proposed method partitions the confidence range into intervals and samples points proportionally to their confidence within each interval. This ensures balanced spatial coverage, preventing over-concentration in high-confidence regions while preserving geometric completeness from lower-confidence areas.

3. Progressive Alignment with Event Streams:
   To address residual artifacts and refine the reconstruction, the framework incorporates event streams.

   • Latent Sharp Image Decoupling: Using the Event-based Double Integral (EDI) method, the system decouples motion-blurred RGB images into multiple latent sharp images based on the asynchronous event data.
   • Latent Motion Modeling: The exposure period is modeled as a sequence of latent images along a camera trajectory. The system parameterizes and optimizes camera poses (using Lie algebra interpolation in SE(3)) to recover the trajectory responsible for the blur formation.
   • Optimization: A progressive alignment strategy iteratively refines latent sharp appearances and camera motion. The loss function combines a standard 3D-GS reconstruction loss (aligning the average of latent renders with the blurred input) and an event-guided loss. The event-guided loss aligns the grayscale versions of rendered latent sharp images with the latent images derived from events, providing fine-grained supervision to guide the optimization toward geometrically consistent reconstruction.

Key Contributions

• DeblurSplat Framework: The first traditional SfM-free 3D-GS framework for robust deblurring, effectively integrating the extensive prior knowledge from DUSt3R and the high dynamic range information from event cameras.
• Confidence Balanced Sampling: A novel sampling strategy that leverages DUSt3R's confidence estimates to extract high-quality initial point clouds while maintaining necessary spatial coverage, avoiding the pitfalls of aggressive pruning or over-concentration.
• Progressive Alignment: A mechanism that combines event stream cues to estimate latent sharp images and latent camera poses, providing fine-grained guidance for scene reconstruction optimization.
• Comprehensive Evaluation: The construction of seven challenging synthetic "hard" scenes with complex lighting and large-scale camera motions, alongside a real-world dataset with controlled motion blur levels, enabling systematic evaluation beyond existing benchmarks.

Experimental Results
Extensive experiments were conducted on synthetic datasets (Normal and Hard scenes) and real-world datasets.

• Performance: DeblurSplat consistently outperforms state-of-the-art (SOTA) methods, including B-NeRF, B-3DGS, BAD-NeRF, BAD-GS, and other event-based approaches (E2GS, EDNeRF). In the "Hard" synthetic category, DeblurSplat achieved a PSNR of 26.94 compared to the next best (EDNeRF) at 24.20. In real-world scenarios, it achieved a PSNR of 25.93, significantly surpassing competitors.
• Efficiency: The method demonstrates superior initialization efficiency. While COLMAP initialization on sharp images takes significantly longer and often fails on blurred inputs, DeblurSplat achieves initialization approximately 2.96× faster on average in hard scenes.
• Ablation Studies: Experiments confirm that both Confidence Balanced Sampling and Progressive Alignment are critical; removing either component leads to a noticeable drop in PSNR, SSIM, and LPIPS metrics.

Significance and Claims
The paper claims that DeblurSplat constitutes the first attempt to leverage a dense stereo module (DUSt3R) specifically to tackle the challenge of motion deblurring in 3D reconstruction without relying on traditional SfM. By eliminating the dependence on SfM, the method improves reliability in scenarios where blur causes feature matching to fail. The integration of event cameras provides a high-dynamic-range semantic prior that guides the recovery of latent sharp images and camera poses, enabling high-fidelity novel view synthesis even under severe motion blur. The authors emphasize that their approach not only excels in generating high-fidelity novel views but also achieves significant rendering efficiency compared to existing SOTA deblur 3D-GS methods.