Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Prediction

1. Problem Definition

The paper addresses Vision-based Risk Object Identification (Vision-ROI), a critical capability for Intelligent Driving Systems (IDS) that involves localizing potential hazards and estimating their risk levels.

• Current Limitations: Existing Vision-ROI approaches are largely deterministic. They rely on fixed decision thresholds and assume reliable perception and stable scene dynamics.
• The Core Issue: Real-world traffic is inherently uncertain due to occlusions, sensor noise, and incomplete observations. Deterministic methods fail to model this uncertainty, leading to:
  • Temporal Instability: Predictions that flicker between "risky" and "non-risky" states.
  • Boundary Misalignment: Premature or delayed risk detection (e.g., braking too early or too late).
  • Lack of Spatiotemporal Modeling: Current methods do not jointly model uncertainty across both space (location) and time (duration of risk).
  • Calibration Issues: Existing uncertainty estimation methods (Bayesian, Ensembles, etc.) often produce miscalibrated scores that do not reliably reflect predictive correctness.

2. Methodology: Conformal Risk Tube Prediction

The authors propose a unified framework called Conformal Risk Tube Prediction (CRTP) that models risk as a spatiotemporal tube rather than a single-frame decision.

A. Risk Tube Formulation

Instead of predicting a binary risk label for a single frame, the model predicts a Risk Tube $T$, which encloses a set of uncertain risk objects and the future time intervals $[t_{start}, t_{end}]$ during which they may become hazardous.

• Input: Past front-view image frames.
• Output: A tube representing the risk interval for each object. A smaller tube indicates high confidence; a larger tube indicates high uncertainty.

B. Base Model Architecture

• Feature Extraction: Uses an I3D backbone for global ego-features and RoIAlign for per-object features.
• Handling Occlusion: Introduces "phantom boxes" for occluded regions to capture hidden objects.
• Relation Modeling: Uses a Graph Convolutional Network (GCN) for spatial relations and an LSTM-like module for temporal dynamics.
• Initial Prediction: A linear layer predicts deterministic risk scores over an 8-step horizon.

C. Conformal Prediction (CP) Integration

To ensure statistical guarantees on coverage, the authors integrate Conformal Prediction:

1. Nonconformity Score: Measures the discrepancy between the predicted risk score and the ground truth.
2. Category-Aware Calibration: Vanilla CP is insufficient because different risk types (e.g., occlusion vs. collision) have distinct spatiotemporal characteristics. The authors train a classifier to identify the risk category and apply a dedicated conformal calibrator for each category.
3. Buffer Zones: The calibrated output creates a "buffer zone" around the decision boundary. If a prediction falls within this zone, it is treated as ambiguous, preventing oscillation (flickering) between risky and safe states.

D. Spatiotemporal Feature Alignment (STFA)

To improve the consistency of features across time and space for the same risk category, a Spatiotemporal Feature-Alignment Loss is introduced.

• It penalizes mismatches between the spatial similarity of objects in the same category and their temporal similarity (how their features evolve over time).
• This encourages the model to learn representations that are consistent with the physical evolution of hazards.

3. Key Contributions

1. Conformal Risk Tube Prediction Framework: A novel uncertainty-aware formulation that jointly models spatiotemporal risk uncertainty, providing coverage guarantees for true risks and calibrated risk scores.
2. Multiple Coexisting Risks (MCR) Dataset: A new dataset constructed in CARLA containing approximately 1,000 scenarios where multiple risk categories (Interaction, Collision, Obstacle, Occlusion) occur concurrently or sequentially. This fills a gap in existing benchmarks that typically focus on single-risk scenarios.
3. Category-Aware Calibration: A mechanism to handle the heterogeneity of risk types, ensuring reliable calibration across different hazard patterns.
4. Systematic Evaluation: Comprehensive analysis of uncertainty factors, including scenario variations, per-category behavior, and the propagation of perception errors.

4. Experimental Results

The method was evaluated against baselines including Rule-based, Hard Decision (HD), Bayesian Neural Networks (BNN), Kalman Filters (KF), Online Conformal Prediction (OCP), and Uncertainty Embedding (UE).

• Robustness to Spatiotemporal Variations (RQ1):

  • CRTP achieved the highest Coverage (0.851 in single-risk, 0.827 in multi-risk) while maintaining a moderate Tube Volume (indicating precise uncertainty estimation).
  • It significantly outperformed baselines in Temporal Consistency and Boundary Alignment, reducing fragmented predictions.
  • Performance remained robust even in complex Multi-Risk settings where other methods degraded significantly.

• Robustness to Perception Errors (RQ2):

  • When ground-truth bounding boxes were replaced with noisy perception detections, CRTP showed the smallest performance drop and the least inflation in Tube Volume compared to other methods.
  • This demonstrates the framework's ability to absorb detection noise without losing temporal fidelity.

• Downstream Impact (RQ3):

  • In a braking alert simulation, CRTP significantly reduced nuisance braking alerts (Average Brake Counts) and misaligned braking (Misaligned Brake Counts) compared to distance-based or other Vision-ROI methods.
  • It achieved the lowest misaligned brake counts (16.34 in single-risk, 20.68 in multi-risk), closely matching ground-truth braking behavior while minimizing false positives.

• Ablation Study:

  • Adding STFA improved coverage and reduced tube volume.
  • Adding Category-Aware Conformal Calibrators (CACC) further improved all metrics, confirming the necessity of handling risk categories separately.

5. Significance and Conclusion

This paper presents a paradigm shift in Vision-ROI from deterministic, frame-level decisions to uncertainty-aware, spatiotemporal risk modeling.

• Safety Criticality: By providing calibrated risk tubes with coverage guarantees, the system can make more reliable decisions in ambiguous scenarios, directly addressing safety-critical failures like premature or delayed braking.
• Practical Utility: The reduction in nuisance braking alerts demonstrates immediate value for real-world IDS deployment, balancing safety with driver comfort.
• Future Directions: The authors note that performance in occlusion scenarios remains a challenge and plan to extend the framework to other safety tasks like lane-change avoidance and junction yielding.

In summary, Conformal Risk Tube Prediction offers a principled, statistically grounded approach to handling the inherent uncertainty of driving environments, significantly improving the robustness and reliability of hazard detection systems.