💻 computer science

Using Ensemble Diffusion to Estimate Uncertainty for End-to-End Autonomous Driving

This paper introduces EnDfuser, an end-to-end autonomous driving system that utilizes ensemble diffusion to generate a distribution of candidate trajectories from fused perception data, thereby enabling uncertainty-aware decision-making that improves driving performance and interpretability on the LAV benchmark.

Original authors: Florian Wintel, Sigmund H. Høeg, Gabriel Kiss, Frank Lindseth

Published 2026-05-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Florian Wintel, Sigmund H. Høeg, Gabriel Kiss, Frank Lindseth

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a self-driving car to navigate a busy city. Most current systems work like a student who has memorized a single "best" route. When they see a situation, they instantly pick one path and stick to it, even if they are unsure. If that single choice is wrong, the car might crash.

This paper introduces a new system called EnDfuser (Ensemble Diffuser). Instead of forcing the car to pick just one path, EnDfuser acts like a cautious driver who imagines many possible futures at once.

Here is how it works, broken down into simple concepts:

1. The "Crystal Ball" Approach (Diffusion Models)

Traditional planning is like asking a friend, "Where should I turn?" and getting one answer: "Turn left."
EnDfuser uses a technology called a Diffusion Model. Think of this like a sculptor starting with a block of noisy, shapeless clay. The model starts with a bunch of random, messy ideas (noise) and slowly "denoises" them, refining them step-by-step until they become clear, smooth paths.

The magic trick here is that instead of making just one sculpture, EnDfuser makes 128 different sculptures (candidate trajectories) from the same starting point.

The Analogy: Imagine you are about to cross a street. A normal system looks and says, "I'll cross now." EnDfuser looks and thinks, "I could cross now, I could wait, I could cross slowly, or I could stop." It generates 128 different "what-if" scenarios in the blink of an eye.

2. Measuring Confidence (Uncertainty)

Because EnDfuser generates 128 different paths, the team can look at how much these paths agree with each other.

High Agreement: If all 128 paths look almost identical (e.g., they all say "turn left gently"), the car is confident.
Low Agreement: If the paths are all over the place (some say "turn left," some say "stop," some say "go straight"), the car is uncertain.

The paper uses the speed of these paths to measure this uncertainty. If the 128 paths suggest wildly different speeds, the system knows something is tricky about the situation.

3. The "Safety Brake" Rule

The researchers tested a very simple rule based on this uncertainty:

The Rule: "If the car is confused (high uncertainty about speed), hit the brakes."
The Result: By adding this simple "panic brake" only when the system is unsure, the car drove slightly better (a 1.7% improvement in their test scores) and had fewer collisions. It didn't need to be a complex AI to make this decision; it just needed to listen to its own confusion.

4. What Did They Learn? (The "Uncertainty Map")

By looking at where the car got confused, the researchers found that uncertainty spikes in specific places:

Intersections and Curves: This makes sense; these are complex areas where other cars might do unexpected things.
Dynamic Interactions: When the car is changing lanes or crossing paths with other moving vehicles, the "128 paths" diverge, signaling danger.
Data Errors: Sometimes the car gets confused because the training data it learned from was slightly messy (like a teacher giving a student a slightly wrong map). The system flagged these spots as "uncertain," effectively helping to find errors in the data itself.

Summary

The paper claims that by using Ensemble Diffusion, they created a self-driving system that doesn't just guess one future, but simulates many. By watching how much those simulations disagree, the car can tell when it is in a dangerous or confusing situation and slow down. This simple trick made the car safer and more reliable in their tests, proving that "knowing what you don't know" is a powerful tool for autonomous driving.

Technical Summary: EnDfuser – Ensemble Diffusion for Uncertainty Estimation in End-to-End Autonomous Driving

Problem Statement

End-to-end (E2E) autonomous driving (AD) systems are rapidly advancing, particularly within closed-loop simulation environments like CARLA. However, a significant limitation persists: many E2E planning systems either ignore uncertainty as part of the planning process or rely on specialized representations that lack generalizability. In the AD domain, uncertainty quantification (UQ) is critical because unreliable predictions can lead to catastrophic consequences. Sources of uncertainty include sensor noise, distribution shifts, and architectural shortcomings. While previous methods have attempted to address this using Bayesian approaches, Monte Carlo dropout, or ensembles, there is a need for a method that integrates uncertainty modeling directly into the trajectory planning module without requiring complex architectural changes or specialized discrete classifiers.

Methodology

The authors propose EnDfuser, an end-to-end driving system that utilizes a denoising diffusion probabilistic model (DDPM) as its trajectory planner. The system is designed to model the posterior distribution of trajectories, thereby providing a natural mechanism for uncertainty estimation through ensemble diffusion.

Architecture

EnDfuser builds upon the perception backbone of TransFuser++ (TF++), which fuses RGB images and LiDAR bird's-eye-view (BEV) features. However, it replaces TF++'s transformer+GRU planning module with a diffusion transformer.

Input Conditioning: The diffusion model is conditioned on enriched BEV features, which include the driving instruction (target point), current velocity, and the diffusion step $k$ .
Trajectory Representation: The output is a set of 8 waypoints spaced 250ms apart, representing a 2-second planning horizon.
Ensemble Generation: Instead of committing to a single point estimate, EnDfuser generates a distribution of candidate trajectories (specifically 128 candidates in the experiments) from a single perception frame. This is achieved via parallel batch sampling on a GPU.
Inference Efficiency: The system employs a DDIM (Denoising Diffusion Implicit Models) schedule, allowing for high-quality trajectory generation with only 2 denoising steps, ensuring real-time performance (approx. 29 FPS).

Uncertainty Quantification

The core innovation lies in leveraging the ensemble of generated trajectories to estimate uncertainty:

Transformation to Control Commands: The set of 128 trajectory candidates ( $T_t$ ) is transformed into a set of control commands ( $K_t$ ), specifically desired speed and yaw angle, using a logic derived from TF++'s PID controller.
Variance Calculation: The system calculates the variance of the predicted speed distribution ( $\hat{\sigma}^2_s$ ) across the 128 candidates. The authors identify speed variance as the primary indicator of uncertainty, noting that yaw is often dependent on speed in this specific task design.
Safety Rule: A simplistic, rule-based safety heuristic is introduced: if the speed variance $\hat{\sigma}^2_s$ exceeds a threshold $\lambda$ , the agent overrides the desired speed with 0 m/s (braking).

Key Contributions

The paper outlines three primary contributions:

EnDfuser Agent: The presentation of a simple E2E driving agent capable of modeling planning uncertainty in closed-loop CARLA scenarios using a diffusion-based planner.
Performance Improvement: Demonstration that a simple uncertainty-informed heuristic (the braking rule) increases the driving score of the agent by 1.7% on the LAV benchmark.
Distribution Analysis: Evidence that the posterior trajectory distribution can reveal the "long tail" of the driving distribution, specifically highlighting occurrences of potentially safety-critical situations and data biases (such as label noise).

Experimental Results

The system was evaluated on the LAV and Longest6 benchmarks within the CARLA simulator.

LAV Benchmark: The base EnDfuser configuration (without the safety rule) already outperformed the TF++ baseline in Driving Score (DS) and Infraction Score (IS). With the safety rule applied (threshold $\lambda = 0.4$ ), the DS improved further to 78.1 (compared to TF++'s 70.0), and vehicle collisions per kilometer decreased.
Longest6 Benchmark: While EnDfuser variants generally scored lower than TF++ on this more complex benchmark, the safety rule successfully reduced the vehicle collision rate from 1.1 to 1.02 collisions per kilometer. However, this did not translate to a higher overall DS due to a trade-off involving increased route timeouts.
Uncertainty Localization: Analysis of the speed variance map revealed that high uncertainty ("spikes") consistently correlated with intersections, bends, and interactions with dynamic agents.
Categorization of Uncertainty: Visual inspection of the 100 highest-variance frames showed that 83% occurred during interactions with other agents. The analysis also identified causal confusion (uncertainty without clear visual cause) and lateral label noise (where predictions extended beyond known target points due to training data limitations).

Significance and Claims

The authors position EnDfuser as a significant step toward uncertainty-aware decision-making in E2E driving policies.

Drop-in Replacement: The paper claims that ensemble diffusion can serve as a "drop-in replacement" for traditional point-estimate planning modules, effectively modeling the uncertainty of the posterior trajectory distribution.
Interpretability: By observing the full set of candidate trajectories, the system provides interpretability for uncertain, multimodal future spaces, allowing for the detection of safety-critical situations that might be missed by single-point estimators.
Data Mining: The method offers a potential tool for filtering training data, specifically to identify the "long tail" of the driving distribution and areas of high agent uncertainty, including instances of label noise.

The paper maintains a modest tone regarding its limitations, acknowledging that the current safety heuristic is simplistic and may not generalize well. It notes that the method does not currently distinguish between aleatoric and epistemic uncertainty and suggests that future work should explore more sophisticated measures (e.g., entropy) and dynamic safety heuristics.