⚡ electrical engineering

Quantifying Aleatoric and Epistemic Dynamics Uncertainty via Local Conformal Calibration

This paper introduces Local Uncertainty Conformal Calibration (LUCCa), a non-asymptotic, conformal prediction-based method that locally calibrates aleatoric uncertainty estimates to generate probabilistically valid prediction regions accounting for both aleatoric and epistemic uncertainties in robot dynamics, thereby enabling safe planning in novel environments.

Original authors: Luís Marques, Dmitry Berenson

Published 2026-09-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Luís Marques, Dmitry Berenson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots move through the world by predicting what will happen next. If a robot knows exactly how its wheels grip the floor or how its propellers push against the air, it can calculate a perfect path to a destination. But the real world is messy. Surfaces get slippery, winds shift, and the physics of a machine can change in ways its designers never anticipated. When a robot encounters a situation it has not seen before, its internal prediction of the future becomes a guess. To move safely, the robot must understand not just where it thinks it will be, but how uncertain that guess is. It needs to distinguish between two types of doubt. One type is random noise, like a wheel slipping on a patch of ice; this is unpredictable but happens within known limits. The other type is ignorance, like a robot trying to drive on a surface it has never touched before; here, the robot simply lacks the information to know what will happen. For years, engineers have struggled to measure this second type of uncertainty in a way that guarantees safety, often relying on assumptions that break down when the environment changes.

A team of researchers at the University of Michigan has developed a new method to solve this problem, allowing robots to build a safety net around their predictions even when they are in completely unfamiliar territory. They call their approach Local Uncertainty Conformal Calibration. The core idea is to take a robot's best guess about its future movement and then adjust the size of its "safety bubble" based on real-world data. Imagine a robot that has a model of how it moves, but that model might be wrong in certain areas. Before the robot starts a new task, the researchers have it run through a short calibration phase. During this phase, the robot tests its model against a small set of actual movements it performs on the ground. By comparing what the model predicted with what actually happened, the system learns how much to expand its safety bubble in different parts of the room. If the model is usually accurate in one corner, the bubble stays tight. If the model is shaky in another corner, perhaps because the floor is slippery, the bubble expands significantly to cover all possible outcomes.

The researchers tested this method on a simulated robot that moves like a car, capable of accelerating and turning. They placed this robot in environments where the physics suddenly changed, such as a section of the floor with much lower friction than the rest. In these tricky zones, the robot's standard model would assume it could stop or turn easily, leading it to plan paths that would result in a crash. The new method, however, recognized that the model was unreliable in those specific spots. It used the calibration data to inflate the uncertainty for those areas, forcing the planning algorithm to choose slower, safer routes that avoided the dangerous zones entirely. In a series of tests across four different environments, the robot using this new calibration method reached its goal without colliding with obstacles in most trials, though it did experience a small number of collisions in more complex scenarios like sharp turns. In contrast, a robot using the standard, uncalibrated model crashed in up to 80 percent of the attempts, because it failed to account for the fact that its own understanding of the world was incomplete.

What makes this approach particularly powerful is that it does not require the robot to know exactly how the physics have changed. It does not need to identify that the floor is wet or that the air is thick. It simply observes the mismatch between its prediction and reality and adjusts its confidence accordingly. The researchers proved mathematically that this method guarantees the robot's safety bubble will contain the true future position of the robot at least 90 percent of the time for the very first step of a plan, even with a very small amount of calibration data. This guarantee holds true regardless of whether the robot is moving through a familiar hallway or a chaotic, unpredictable space. By breaking the problem down into small, local regions, the system avoids the common pitfall of being overly cautious everywhere. Instead of assuming the worst-case scenario for the entire journey, it only expands the safety margin where the data shows it is truly necessary.

The study also looked at how this method performs over time as the robot plans a sequence of moves. While the mathematical proof is strongest for the very first step of a plan, the experiments showed that the method remained effective even as the robot looked further into the future. The system successfully managed the compounding uncertainty that usually plagues long-term planning. In the simulations, the robot using this method took slightly longer to reach its destination than the crashing robot that ignored uncertainty, but it arrived safely. The computational cost of adding this safety check was minimal, taking less than a millisecond per planning step on a standard computer processor. This suggests that the method could be implemented on real robots without slowing them down. The work demonstrates that by combining a robot's internal model with a simple, data-driven check, we can give machines the ability to recognize their own ignorance and act with appropriate caution, turning a potentially dangerous guess into a provably safe plan.

Technical Summary: Quantifying Aleatoric and Epistemic Dynamics Uncertainty via Local Conformal Calibration

1. Problem Statement

Robotic control systems often rely on approximations of dynamics (analytical, simulated, or learned) that may be inaccurate when deployed in novel environments. While many methods exist to quantify aleatoric uncertainty (stochasticity inherent in the system), these estimates are insufficient for novel environments where the underlying physical interactions change, inducing epistemic uncertainty (uncertainty due to lack of data or model mismatch).

The core challenge is to provide provable, non-asymptotic safety guarantees for motion planning when:

The true dynamics function $f$ is unknown and potentially discontinuous.
The approximate model $\tilde{f}$ may be arbitrarily wrong in the deployment domain ( $P_{test}$ ) compared to the training domain ( $P_{train}$ ).
The system must satisfy a safety constraint $P(Y_\tau \in C) \geq 1-\alpha$ over a planning horizon, where $C$ is a safe set.

Existing approaches often rely on strong assumptions (e.g., Gaussian error distributions), provide only asymptotic bounds, or use global calibration factors that are overly conservative and uninformative for high-dimensional state-action spaces.

2. Methodology: Local Uncertainty Conformal Calibration (LUCCa)

The authors propose LUCCa, a conformal prediction-based framework that calibrates the aleatoric uncertainty estimates of a dynamics predictor to account for epistemic uncertainty. The method operates in two phases: an offline calibration phase and an online planning phase.

2.1 Core Concepts

Split Conformal Prediction (SplitCP): The method utilizes SplitCP to construct prediction regions that are valid in a frequentist sense without assuming the structure of the predictor or the data distribution.
Mahalanobis Distance as Non-conformity Score: Since the dynamics predictor $\tilde{f}$ outputs a Multivariate Normal (MVN) distribution $\mathcal{N}(\hat{\mu}, \hat{\Sigma})$ , the non-conformity score $r$ is defined as the Mahalanobis distance between the predicted distribution and the true observed state:
$R^{(i)} = d_M(\tilde{f}(\bar{X}^{(i)}), Y^{(i)}) = \sqrt{(Y^{(i)} - \hat{\mu})^\top \hat{\Sigma}^{-1} (Y^{(i)} - \hat{\mu})}$
Theoretical analysis shows that if the model were perfect, the squared Mahalanobis distance follows a Chi-squared distribution ( $\chi^2_{dim(S)}$ ).
Local Calibration (LOCART): Instead of applying a single global scaling factor (which leads to conservative, large prediction regions), LUCCa employs the LOCART method. It partitions the state-action space using a decision tree regressor trained on calibration residuals.
- Partitioning: The calibration dataset is split. One part trains a decision tree to map state-action pairs $(s, u)$ to leaf nodes representing regions of similar model error.
- Local Scaling: For each leaf node $k$ , a specific conformal scaling factor $\xi_k$ is computed based on the residuals of data points falling into that partition.
- Calibration: During online planning, the predicted covariance $\hat{\Sigma}$ is scaled locally: $\hat{\Sigma}_{cal} = \xi_k \hat{\Sigma}$ . This creates hyperellipsoidal prediction regions that adapt to the local difficulty of the dynamics.

2.2 Algorithm Flow

Offline Calibration:
- Collect a calibration dataset $D_{cal}$ of transitions $(s_t, u_t, s_{t+1})$ from the deployment environment (or a proxy).
- Compute residuals $R^{(i)}$ for all calibration points.
- Fit a decision tree (LOCART) to partition the state-action space and compute local scaling factors $\xi_k$ for each partition.
Online Planning (MPC):
- Given a current state, the planner (e.g., MPPI) samples trajectories.
- For each step in the rollout, the approximate dynamics $\tilde{f}$ propagate the state distribution.
- The local scaling factor $\xi_k$ (determined by the current state-action pair's location in the decision tree) is applied to the propagated covariance.
- The planner optimizes a cost function that includes a penalty for the trace of the calibrated uncertainty (encouraging paths through regions of lower model error) and collision avoidance with the calibrated prediction regions.

3. Key Contributions

Algorithm Proposal: Introduction of LUCCa, an algorithm that provides probabilistically valid prediction regions for robot future states by calibrating aleatoric uncertainty estimates to account for epistemic uncertainty.
Theoretical Guarantees:
- Proof of validity for any finite set of calibration data, any predictor outputting a multivariate normal uncertainty, and any unknown true dynamics function.
- Demonstration that the method achieves local coverage (conditional on state-action partitions) with finite samples, and achieves asymptotic conditional coverage under mild assumptions.
- Proof that for the first planning step, the calibrated region contains the true state with probability $\geq 1-\alpha$ . For subsequent steps, guarantees hold if the approximate dynamics are linear and specific assumptions about the controller are met.
Local Adaptivity: The use of state-action dependent scaling factors avoids the over-conservatism of global calibration, allowing the planner to distinguish between regions of high and low model accuracy.
Empirical Validation: Demonstration of the method's effectiveness on a double-integrator system with significant dynamics shifts (e.g., friction changes), showing improved safety and success rates compared to uncalibrated baselines.

4. Experimental Results

The authors evaluated LUCCa on a double-integrator system in four environments containing regions of accurate dynamics and regions of significant model mismatch (simulating lower friction).

Empirical Coverage: In the "Corridor" environment, LUCCa maintained empirical coverage above the 90% threshold (for $\alpha=0.1$ ) across all time steps, even in regions of high model error. In contrast, the uncalibrated baseline significantly underestimated uncertainty in mismatched regions, leading to coverage dropping well below 90% as the horizon extended.
Motion Planning Performance: Using Model Predictive Control (MPC) with MPPI:
- Safety: LUCCa achieved a 0% collision rate in the Corridor and Passage environments and 3% in the L-Turn, compared to 30%, 83%, and 80% respectively for the uncalibrated baseline.
- Success Rate: LUCCa achieved 100% success in Corridor and Passage, compared to 70% and 17% for the baseline.
- Efficiency: While the baseline sometimes reached goals faster (by ignoring uncertainty and taking risky paths), it frequently collided. LUCCa's cost function, which penalized high uncertainty, caused the robot to slow down in high-error regions, avoiding collisions.
Computational Overhead: The local calibration step (querying the decision tree and scaling covariance) added approximately 0.3 ms per planning step (a 32% increase in step duration), which the authors deem negligible for practical deployment.

5. Significance and Claims

The paper claims that LUCCa addresses a fundamental limitation in data-driven robotics: the inability to safely deploy learned or analytical models in novel environments where the dynamics may shift.

Theoretical Rigor: Unlike heuristic uncertainty quantification methods, LUCCa provides non-asymptotic, finite-sample guarantees without assuming the form of the true dynamics or the distribution of errors.
Practical Utility: By moving from global to local calibration, the method generates prediction regions that are "sharp" (tight) in regions of high model accuracy and "adaptive" (larger) in regions of high epistemic uncertainty. This allows planners to make informed trade-offs between exploration and safety.
Safety-Critical Planning: The method enables the construction of probabilistically-safe plans for systems with unknown stochastic dynamics, ensuring that trajectories remain within safe sets with a user-specified probability, even when the model is arbitrarily wrong in uncharacterized domains.

The authors conclude that while the multi-step safety guarantees are theoretically proven for linear dynamics under specific controller assumptions, empirical results suggest the method provides robust local coverage even for more complex planning scenarios.