📊 statistics

Robust Risk Under Evolving Uncertainty: A Wasserstein Counterpart of the Entropic Value-at-Risk

This paper introduces the Wasserstein entropic value-at-risk, a robust risk measure that overcomes the limitations of traditional entropic value-at-risk by using optimal transport balls to account for catastrophic scenarios deemed impossible by nominal models, thereby enabling a dynamic decision-making framework that transitions from caution to confidence as environmental uncertainty decreases.

Original authors: Deep Kumar Ganguly, Jan Křetínský

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Deep Kumar Ganguly, Jan Křetínský

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of autonomous machines, from delivery drones to self-driving cars, there is a fundamental tension between speed and safety. An agent, or machine, must make decisions while it is still learning about its surroundings. When it knows very little, it should be extremely cautious, avoiding any path that might lead to disaster. But once it has gathered enough evidence to understand the environment, it should become bold enough to act efficiently. The challenge lies in creating a system that can smoothly transition from extreme caution to confident action as its knowledge grows, without ever making a fatal mistake during the learning process. This requires a way to measure risk that changes dynamically: the machine's own uncertainty about the world should dictate how much it fears the unknown.

Researchers at the Technical University of Munich and Masaryk University have developed a new method to solve this problem, offering a safer way for machines to navigate uncertain environments. They focused on a specific type of risk calculation used to protect against worst-case scenarios. Previous methods relied on a mathematical concept that, while useful, had a critical flaw: if a machine's current model of the world said a disaster was impossible, the safety system would completely ignore that disaster, even if it was physically possible. The new approach replaces this old method with a different mathematical tool that allows the machine to consider catastrophic outcomes that its current model deems unlikely, but not impossible. This ensures that the machine remains vigilant against rare but devastating events until it is absolutely certain they cannot happen.

The researchers built their solution around a concept called the "entropic value-at-risk," which is a standard way to calculate the worst likely loss a system might face. This standard method works by imagining a circle of possible alternative realities around the machine's current best guess. If the machine is unsure, the circle is large, and it considers many different possibilities. As the machine learns, the circle shrinks. However, the shape of this circle in the old method was rigid. It was constructed using a rule that meant if the machine's best guess assigned zero probability to a specific disaster, no amount of uncertainty could ever bring that disaster back into consideration. The disaster was effectively invisible to the safety system, creating a blind spot that could lead to fatal errors when the machine was overconfident.

To fix this, the team introduced a new measure they call the "Wasserstein entropic value-at-risk." Instead of using the rigid circle of the old method, they used a different geometric shape based on the physical distance between outcomes. Imagine a machine trying to cross a canyon. If the wind is calm, the machine might guess the air is still. If a sudden gust hits, the machine needs to know that a storm is possible, even if its current data says the air is calm. The old method would say, "Since my data says calm, a storm is impossible, so I will ignore it." The new method says, "A storm is far away from my current guess, but it is reachable. I will pay a price to consider it, proportional to how far away it is." This allows the safety system to account for disasters that are physically possible but currently seem unlikely, ensuring the machine hedges against them until it is sure they are not happening.

The researchers proved that this new method is mathematically sound and fits perfectly into the existing hierarchy of risk measures. They showed that it behaves exactly as a safe system should: it is conservative when the machine is ignorant and becomes less conservative as the machine learns. Crucially, they demonstrated that this new measure strictly accounts for those "zero-probability" catastrophes that the old method ignored. In their tests, when they introduced a disaster scenario that the machine's model said had zero chance of occurring, the old method's risk calculation did not change at all, no matter how dangerous the scenario became. The new method, however, immediately increased its caution level, reflecting the true danger of the situation.

To make this practical for real-time decision-making, the team linked the size of the safety margin directly to the machine's own "belief entropy," a measure of how confused or uncertain the machine is. When the machine is confused, the safety margin is wide, forcing it to act slowly and carefully. As the machine gathers more data and its belief sharpens, the margin automatically shrinks, allowing it to move faster. This creates a closed-loop system where the machine's caution evolves naturally with its knowledge. They tested this on a simulated drone crossing a canyon. The drone started by hovering and creeping forward, the safest but slowest option. As it gathered evidence that the wind was calm, its internal uncertainty dropped, the safety margin tightened, and it switched to a fast cruising policy. The transition happened automatically, without a hard-coded switch, and the system maintained a guaranteed safety bound throughout the entire process.

The results of their simulations confirmed that the new method works exactly as the theory predicted. The machine successfully avoided catastrophic failures that the old method would have missed, while still being able to move efficiently once it was confident. The researchers verified that their mathematical formulas matched the results of complex computer simulations to a very high degree of precision. They also showed that the system converges quickly to a stable solution, meaning it can be used in real-time applications. By replacing a flawed geometric assumption with a more robust one, the team has provided a way for autonomous agents to be both safe and efficient, ensuring they remain cautious when they should be and bold when it is safe to be so. This work offers a concrete path forward for building machines that can learn and adapt in the real world without sacrificing safety for speed.

Technical Summary: Robust Risk Under Evolving Uncertainty

1. Problem Statement

The paper addresses the challenge of decision-making for an agent learning its environment under uncertainty. The core problem is designing a risk attitude that evolves with the agent's epistemic state: the agent should be maximally cautious when ignorant (high uncertainty) and become bolder as its belief sharpens (low uncertainty).

Existing approaches face a dichotomy:

Fixed Robust Control: Assumes the worst case forever, leading to overly conservative behavior that fails to adapt even when the environment is clearly safe.
Expected-Value/Bayesian Planning: Commits to a policy before beliefs are sharp, risking catastrophic failure if the initial guess about the hidden environment type is wrong.

The paper identifies a specific limitation in the Entropic Value-at-Risk (EVaR), a popular coherent risk measure. While EVaR offers a robust-optimization identity where a confidence level $\alpha$ defines a relative-entropy (KL) ball of alternative models, this geometry has a "blind spot." If the nominal model $P$ assigns zero probability to a catastrophic outcome, the KL-ball cannot include any distribution assigning positive probability to that outcome (since KL divergence is infinite off the support of $P$ ). Consequently, as an agent learns and its nominal model concentrates away from rare disasters, the EVaR safety check becomes vacuous precisely when overconfidence is most dangerous.

2. Methodology

The authors propose replacing the relative-entropy ball with a 1-Wasserstein ball (optimal transport) to define a new risk measure: the Wasserstein Entropic Value-at-Risk (WEVaR).

Core Definitions

WEVaR: Defined as the supremum of expected loss over a Wasserstein ball of radius $\varepsilon$ :
$\text{WEVaR}_\varepsilon(X) = \sup \{ \mathbb{E}_Q[X] : W_1(Q, P) \le \varepsilon \}$
Unlike the KL-ball, the Wasserstein ball allows the adversary to shift mass to outcomes with zero nominal probability, provided the "transport cost" (distance) is within the radius $\varepsilon$ .
Variational Dual: The paper derives a variational dual for WEVaR that mirrors the entropic formula. For a bounded loss $X$ and radius $\varepsilon$ :
$\text{WEVaR}_\varepsilon(X) = \inf_{\lambda \ge 0} \{ \lambda \varepsilon + \mathbb{E}_P [X^c_\lambda] \}$
where $X^c_\lambda(s) = \max_{s'} (X(s') - \lambda d(s', s))$ is the $c$ -transform. This transforms the problem into a one-dimensional convex program.
Closed-Form Expression: Below a saturation threshold, the measure admits an exact closed form:
$\text{WEVaR}_\varepsilon(X) = \mathbb{E}_P[X] + \varepsilon \cdot \text{Lip}_d(X)$
where $\text{Lip}_d(X)$ is the Lipschitz constant of the loss function with respect to the ground metric. This equality holds exactly for $\varepsilon$ below the threshold (in particular, exactly for two-point supports).

Dynamic Programming Operator

The authors integrate this risk measure into a dynamic programming framework where the ambiguity radius $\varepsilon$ is driven by the agent's belief entropy $H(b)$ .

Radius: $\varepsilon(b) = \beta H(b)$ , where $\beta$ is a sensitivity parameter.
Mechanism: As the agent gathers data, the belief $b$ sharpens, $H(b) \to 0$ , and the ambiguity ball contracts. This causes the risk attitude to slide from worst-case (high entropy) to risk-neutral (zero entropy).
Operator: The resulting robust Bellman operator has a closed-form inner update:
$(\mathcal{T}V)(s, b) = \max_a \left[ r(s, a) + \gamma \left( \mathbb{E}_{\bar{P}_b}[V] - \beta H(b) \cdot \text{Lip}_d(V) \right) \right]$
This eliminates the need for solving a linear program at every step, reducing computational cost to $O(|S|^2)$ .

3. Key Contributions

Definition and Duality of WEVaR: The paper defines WEVaR and proves a Kantorovich–Rubinstein variational dual. This dual is convex, well-posed, and structurally mirrors the entropic formula (replacing inverse temperature with transport price).
Risk Hierarchy and Coherence: It is proven that WEVaR is a coherent risk measure. The authors establish a "sandwich" relationship where EVaR is bounded by WEVaR under a transport-entropy inequality. Crucially, they prove that WEVaR strictly accounts for catastrophes with zero nominal probability (which EVaR ignores) once the transport radius exceeds the distance to those states.
Numerical Verification: Using Gurobi, the authors verify strong duality between the primal and dual formulations for both EVaR and WEVaR. They demonstrate that while EVaR remains unchanged when a zero-probability disaster's loss increases, WEVaR increases linearly with the loss, confirming the theoretical "blind spot" correction.
Evolving Uncertainty Framework: The paper constructs a closed-form robust dynamic programming operator where the risk radius is a function of belief entropy. This yields a "safety sandwich" (bounded between a maximally cautious floor and a type-aware oracle) and a computable "safety switch" mechanism.

4. Results

Geometric Behavior: In a five-state metric space, the Wasserstein ball (blue) is shown to reach "Fail" vertices even when the nominal belief places zero mass on them, whereas the KL ball (red) is trapped on the support edge.
Quantitative Gap: In a numerical experiment where a zero-nominal-probability disaster loss increased from 50 to 1000, EVaR changed by 0, while WEVaR changed by 807.5, validating the theoretical advantage in handling unreachable-but-possible catastrophes.
Convergence: Value iteration using the closed-form operator converges geometrically with rate $\gamma$ . The adaptive value remains strictly within the safety bounds (sandwich) at every iteration, not just at convergence.
Safety Switch: In an "Ambiguous Bridge" scenario, the agent exhibits a "creep-then-cruise" behavior. It hovers (cautious) until its belief that the environment is benign crosses a specific threshold ( $\approx 0.983$ ), at which point it switches to a fast policy. This threshold is derived purely from reward asymmetry and is independent of the sensitivity parameter $\beta$ .

5. Significance and Claims

The paper claims that WEVaR provides a necessary geometric correction to entropic risk measures for safety-critical learning. By utilizing optimal transport geometry, the method ensures that an agent's safety checks do not become vacuous as it gains confidence.

The primary significance lies in the closed-form robust dynamic programming operator. By linking the ambiguity radius directly to belief entropy, the authors provide a mechanism where caution naturally contracts as evidence accumulates. This allows for:

Certified Safety: A "safety sandwich" that guarantees the agent's value stays within safe bounds throughout the learning process.
Computational Efficiency: The closed-form update avoids the computational burden of solving robust optimization problems at every step, making the approach scalable to larger state spaces (via Lipschitz critics).
Adaptive Risk: A principled way to transition from robust control to expected-value control without manual tuning of the risk attitude, driven entirely by the agent's epistemic state.

The authors position this work as a bridge between relative-entropy and optimal-transport robustness, suggesting that scaling this operator to continuous spaces via Lipschitz critics and applying it to active information gathering are natural next steps.