Technical Summary: Ghosts in the Point Clouds: De-glaring LiDAR in the Transient Domain

1. Problem Statement

The rapid transition of LiDAR technology from bulky, mechanically scanned systems to compact, solid-state, camera-like arrays (specifically those utilizing Single-Photon Avalanche Diode or SPAD arrays) introduces a critical failure mode: internal-multipath glare.

In these "camera-fied" sensors, light from bright or retroreflective surfaces (e.g., road signs, license plates) undergoes multiple internal reflections and scattering within the optical assembly before reaching the sensor. Unlike legacy scanning LiDARs, which avoided this via dedicated optical paths and sequential firing, modern dense pixel arrays allow this scattered light to spread across neighboring pixels. This phenomenon creates "ghosts in the point clouds"—phantom objects that obscure real scene geometry and pose safety-critical risks for autonomous systems.

Existing solutions are insufficient for three primary reasons:

1. Timing: Many approaches operate on the final point cloud, after artifacts have become entangled with geometric data, making separation difficult.
2. Pileup Distortion: High-intensity returns causing glare often induce severe photon pileup (non-linear distortion where early photons censor late ones), which standard linear de-glare methods fail to handle.
3. Generalization: Learning-based methods often rely on synthetic data and specific geometric assumptions (e.g., octagonal signs), struggling to generalize to complex scenes or diverse retroreflector types without extensive training data.

2. Methodology

The authors propose a training-free, physically grounded approach that operates in the transient domain (on low-level echo detections) prior to point-cloud formation. The pipeline consists of three main stages:

A. Transient Glare Modeling (TGSF)

The paper models internal glare as a linear, scene-independent operator acting on transient histograms.

• Transient Glare Spread Function (TGSF): Defined as $a(u, v, t, u', v', t')$, this operator describes how photons from a single pixel $(u', v')$ at time $t'$ redistribute across other pixels and time bins due to internal scattering.
• Simplification: In practice, the temporal delay caused by internal scattering is negligible compared to sensor resolution. Thus, the TGSF is approximated as a time-independent Glare Spread Function (GSF), $a(u, u')$, reducing the problem to a spatial convolution in the echo domain.

B. Pileup Correction

Before addressing glare, the method corrects for photon pileup, which distorts the intensity and time-of-flight (ToF) of the brightest returns.

• Moments-Based Correction: Instead of attempting to fully "undistort" raw histograms (which is computationally difficult and prone to saturation), the system extracts the first three moments (total counts, mean ToF, variance) of detected echoes.
• Lookup Tables (LUTs): Using a forward model of pileup physics, the authors generate LUTs that map measured variance and background noise to the true incident flux and mean shift. This allows the system to recover the true ToF and intensity of the "aggressor" echo, which is essential for accurate glare prediction.

C. Glare Prediction and Suppression

Once echoes are pileup-corrected, the algorithm determines the likelihood that a detection is a true return or a glare artifact.

1. Glare Prediction: Using the pre-calibrated GSF, the system predicts the expected glare contribution ($\bar{g}_{uk}$) at each pixel $u$ and echo $k$ based on the pileup-corrected intensities of all other pixels.
   $$\bar{g}_{uk} = \sum_{u'} a(u, u') \left[ \sum_{k'} o(t_{u'k'} - t_{uk}) y_{u'k'} \right]$$
   where $o(\delta t)$ accounts for temporal overlap between fitting windows.
2. Confidence Metric: A binomial likelihood metric is computed to assess if the measured photon count $Y$ significantly exceeds the expected glare count $N P_G$.
   $$C = \begin{cases} -\ln \mathbb{P}(Y; P_G, N) & \text{if } Y \geq N P_G \\ 0 & \text{if } Y < N P_G \end{cases}$$
   Echoes with $C=0$ are identified as glare; those with $C>0$ are retained as valid scene returns. The system selects the highest-confidence echo per pixel to form the final depth map.

3. Key Contributions

The paper makes four specific contributions:

1. Identification of Failure Mode: It explicitly identifies internal-multipath glare as a critical, frequent failure mode for next-generation solid-state LiDARs, distinct from the sparse artifacts of legacy systems.
2. Transient-Domain Formulation: It introduces a model that jointly accounts for internal multipath glare and non-linear photon pileup distortion, operating in the transient echo space rather than the final point cloud.
3. Training-Free Algorithm: It proposes an algorithm that leverages the known GSF to estimate the probability of glare-induced detections without requiring large-scale labeled datasets or synthetic training.
4. Pipeline Integration: It demonstrates a practical method that integrates seamlessly into standard LiDAR signal-processing pipelines (DSP) on unmodified commercial sensors, operating before point-cloud formation.

4. Experimental Results

The method was evaluated using a commercial SP-LiDAR (ADS6311) in various scenes containing diverse retroreflectors (stop signs, license plates, traffic cones, safety vests).

• Quantitative Performance: Compared to a baseline deep learning method [5] and a photographic de-glare approach [26], the proposed method achieved significantly lower Root Mean Square Error (RMSE) and higher $\delta_1$ accuracy (percentage of pixels within 1% of ground truth).
  • Example: In a scene with a stop sign, the baseline deep learning method achieved an RMSE of 0.48m, while the proposed method achieved 0.11m.
• Robustness to Complexity: Unlike the baseline, which struggled with complex geometries (e.g., traffic cones, safety vests) and multiple retroreflectors, the proposed method generalized effectively across all tested shapes and configurations.
• Pileup Handling: Experiments showed that photographic de-glare methods (applied to individual time slices) failed when pileup was severe because pileup shifts the direct return to earlier time bins, separating it from the glare signal in the time domain. The proposed echo-space method successfully handled this by correcting pileup before glare estimation.
• Preservation of True Signals: The method successfully suppressed glare while preserving true, weaker signals (e.g., a mannequin representing a child) that were otherwise obscured by "bloom" artifacts in standard pipelines.

5. Significance and Claims

The paper argues that internal glare is not merely a corner case but a fundamental challenge for the "camera-fication" of LiDAR. The significance of this work lies in:

• Shifting the Paradigm: Moving glare mitigation from post-processing (point cloud) to pre-processing (transient/echo domain), where the physics of the artifact is most tractable.
• Enabling Solid-State Architectures: By providing a software solution to a hardware-induced problem, the method reduces the need for the sub-optimal, bulky scanning architectures historically used to avoid glare. This could accelerate the adoption of high-resolution, low-cost, fully solid-state LiDARs.
• Modularity: The approach is designed to be a modular component of the photon-processing pipeline, compatible with existing pileup correction, denoising, and compression techniques.

The authors acknowledge limitations, noting that the method requires pre-calibration of the GSF for each sensor and has not yet been tested in outdoor scenes with intense solar background. However, they posit that algorithmic de-glare could become a standard component of 3D perception pipelines, analogous to denoising in conventional imaging.