Technical Summary: FlowCalib – LiDAR-to-Vehicle Miscalibration Detection using Scene Flows

1. Problem Statement

Accurate extrinsic sensor calibration is fundamental to the safety and reliability of autonomous driving. While Sensor-to-Sensor (S2S) calibration (e.g., LiDAR-to-Camera) is critical for object detection and fusion, Sensor-to-Vehicle (S2V) calibration ensures the correct spatial localization of objects relative to the vehicle's frame.

The paper highlights a critical gap in current research: existing online calibration methods primarily focus on correcting S2S errors by regressing transformation matrices between sensors. However, these methods often assume the error originates from a single source or apply unidirectional corrections (e.g., always correcting LiDAR points to match the camera). If the camera is miscalibrated but the LiDAR is not, forcing a correction on the LiDAR can degrade performance. Furthermore, detecting which individual sensor is miscalibrated (specifically the LiDAR-to-vehicle alignment) remains an under-addressed challenge. Angular misalignments (roll, pitch, yaw) in the LiDAR can cause objects to be perceived in incorrect lanes, leading to safety-critical planning errors or unnecessary braking.

2. Methodology: FlowCalib

The authors propose FlowCalib, a framework that detects LiDAR-to-vehicle miscalibration by analyzing the scene flow of static objects. The core premise is that angular misalignment introduces a systematic, distance-dependent bias in the perceived motion vectors of static points as the vehicle moves.

A. Data Preprocessing and Fault Injection

• Dataset: The nuScenes dataset is used as the baseline.
• Fault Injection: To train the model, angular errors are synthetically injected into the point clouds. A rotation matrix $R_{\alpha_{err}}$ is applied to the original point cloud $X_{org}$ to create a distorted cloud $X_{dist}$. Errors are sampled uniformly from $[-5.0, -0.5]^\circ \cup [0.5, 5.0]^\circ$.
• Preprocessing Steps:
  1. Ground Removal: Ground points are removed using RANSAC (following UNION) to prevent flat, featureless surfaces from confusing the flow estimation.
  2. Coordinate Transformation: Point clouds are transformed from the LiDAR frame to the vehicle frame to align the analysis with the S2V transformation context.
  3. Point Cloud Distillation: To ensure robust scene flow estimation, frames are selected to balance temporal information with the risk of incorrect point correspondences over large intervals.
  4. Dynamic Object Removal: Moving objects are filtered out using ground truth bounding boxes, ensuring the flow field represents only static scene elements.

B. Scene Flow Generation

The framework utilizes Neural Scene Flow Prior (NSFP) to estimate the 3D motion vector field between consecutive time steps ($t-1$ and $t$). NSFP minimizes the distance between transformed points and the target point cloud, incorporating a neural network as an implicit regularizer. This allows for scene-specific flow estimation without pre-training or labeled flow data.

C. Detection Architecture

The detection network employs a dual-branch architecture that fuses learned features with handcrafted geometric descriptors:

1. Global Flow Features (Learned):

   • A PointNet-based architecture processes unordered flow vectors.
   • It uses $1\times1$ convolutions, batch normalization, and ReLU activations to extract global signatures.
   • Features are aggregated using both max pooling (to capture dominant patterns) and mean pooling (for robustness), mitigating the impact of outliers.

2. Geometric Features (Handcrafted):

   • Magnitude: The Euclidean norm of flow vectors, capturing motion length. Statistics (mean $\mu_m$, std $\sigma_m$) are computed.
   • Angle: The angle $\psi$ between flow vectors and vehicle axes (pitch, roll, yaw) is calculated and aggregated into histograms ($h_{nbins}$) to capture the distribution of angular bias.
   • Rotation: The cross product between a point's radial position relative to the ego-vehicle and its flow vector is computed to capture rotational effects. Statistics (mean, std) of this distribution are used.
   • These features are concatenated and passed through a Multi-Layer Perceptron (MLP) to create an embedded vector.

3. Dual-Branch Detection Heads:

   • The concatenated global and geometric features are fed into two separate heads:
     • Global Head: A binary classifier determining if any misalignment is present.
     • Axis-Specific Head: A multi-output classifier (dimension 3) predicting the presence of misalignment for Roll ($\phi$), Pitch ($\theta$), and Yaw ($\psi$) independently.

3. Key Contributions

The paper claims the following contributions:

• First Framework: FlowCalib is the first framework specifically designed to detect LiDAR-to-vehicle miscalibration using scene flow.
• Sensor Independence: The method operates without additional sensors (IMU, GPS, or cameras), relying solely on LiDAR point cloud sequences.
• Granular Detection: It provides both a global misalignment decision and specific axis-level detection (roll, pitch, yaw).
• Benchmarking: It establishes a baseline for miscalibration detection on the nuScenes dataset.
• Open Source: The code is made available as open source.

4. Results

Experiments were conducted on the nuScenes dataset with injected angular errors ranging from $\pm 0.5^\circ$ to $\pm 5.0^\circ$.

• Global Detection Accuracy:
  • Overall accuracy: 81.16%.
  • High severity errors ($\pm 2^\circ$ to $\pm 5^\circ$): 90.27% accuracy.
  • Low severity errors ($\pm 0.5^\circ$ to $\pm 2^\circ$): 73.81% accuracy.
• Axis-Specific Performance:
  • Yaw ($\psi$): High detection rate (95.73% for single-axis errors).
  • Roll ($\phi$): High detection rate (92.86% for single-axis errors).
  • Pitch ($\theta$): Significantly lower detection rate (24.73% for single-axis errors).
  • Multi-Axis Errors: Performance improves drastically when multiple axes are perturbed simultaneously (e.g., 94.70% accuracy when all three axes are disturbed).

Analysis of Pitch Difficulty: The authors note that pitch errors are harder to detect because they induce uniform up/down movements across the scene, lacking the distinct spatial variations (left/right or forward/backward shifts depending on position) seen in roll and yaw errors.

5. Significance and Claims

The paper positions FlowCalib as a necessary evolution in autonomous vehicle safety systems. Its significance lies in:

• Addressing a Blind Spot: Moving beyond S2S correction to identify the root cause (individual sensor miscalibration), which is crucial for bidirectional correction strategies.
• Safety-Critical Application: By detecting S2V errors, the system can prevent objects from being mislocalized into adjacent lanes, thereby avoiding safety hazards and comfort issues (e.g., phantom braking).
• Robustness: The method leverages motion cues inherent in the scene flow, providing a robust alternative to methods requiring ground planes, calibration targets, or specific vehicle maneuvers.
• Interpretability: The use of geometric descriptors alongside learned features provides an interpretable basis for identifying extrinsic calibration errors.

The authors conclude that motion-induced distortions in the flow field provide a reliable and interpretable signal for identifying LiDAR-to-vehicle miscalibration, particularly for yaw and roll, establishing a new benchmark for this specific detection task.