⚡ electrical engineering

Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics

This paper introduces the Uncertainty-Aware Predictive Safety Filter (UPSi), a novel framework that integrates probabilistic ensemble neural networks with rigorous reachable set-based verification to provide scalable, safety-guaranteed exploration for model-based reinforcement learning without sacrificing performance.

Original authors: Bernd Frauenknecht, Lukas Kesper, Daniel Mayfrank, Henrik Hose, Sebastian Trimpe

Published 2026-08-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bernd Frauenknecht, Lukas Kesper, Daniel Mayfrank, Henrik Hose, Sebastian Trimpe

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to play a video game, but with a twist: the robot learns by trying things out, sometimes making wild guesses to see what happens. This is called Reinforcement Learning, and it's how machines get really good at complex tasks like driving cars or playing chess. But there's a huge problem: if the robot tries a "wild guess" that is too crazy, it might crash the car or break the game. We need a way to let the robot explore and learn, but stop it right before it does something dangerous.

To solve this, scientists use something called a "Predictive Safety Filter." Think of it like a super-smart guardian angel or a strict coach standing next to the robot. Before the robot makes a move, the coach runs a quick simulation in their head: "If you jump there, will you hit the wall?" If the answer is "maybe," the coach steps in and changes the robot's move to a safer one. Traditionally, these coaches were very cautious and needed a perfect textbook description of the world to work. But the real world is messy and complicated, so those old coaches often got stuck or couldn't handle complex games. This paper introduces a new kind of coach that is both super smart and very careful, allowing robots to learn faster without getting hurt.

The Problem: The "Black Box" Coach

In the world of machine learning, there's a popular tool called a "Probabilistic Ensemble" (PE). Imagine you have a team of five different experts (neural networks) trying to guess what will happen next in a game. Instead of trusting just one, you ask all five and look at their answers. If they all agree, you feel confident. If they all disagree, you know you are in a "foggy" area where the model doesn't know much. This is great for learning complex things, but it has a flaw: sometimes the team gets overconfident in the fog. They might say, "We are 100% sure you won't crash," even though they are actually guessing wildly.

Previous attempts to use these "expert teams" as safety coaches failed because they didn't have a rigorous way to check if the team was actually telling the truth or just hallucinating. They were like a coach who says, "I think you're safe," without actually checking the map, leading to accidents when the robot tried something too risky.

The Solution: UPSi (The "Oop-See" Filter)

The authors of this paper introduce a new system called UPSi (pronounced "oop-see"), which stands for Uncertainty-Aware Predictive Safety Filter.

Think of UPSi as a safety coach that doesn't just ask the experts for an answer; it also checks their confidence levels. It uses a clever mathematical trick to draw a "safety bubble" around the robot's possible future paths.

The Safety Bubble: Instead of guessing a single future path, UPSi calculates a whole cloud of possible places the robot could end up. It makes sure this entire cloud stays inside the "safe zone" (like staying on the track).
The Certainty Check: This is the magic part. UPSi has a special rule: "We only trust the experts if they are in a 'Certain Set'." If the robot tries to move into an area where the experts are confused (high uncertainty), UPSi says, "Nope, I don't know enough about this area to guarantee safety. Let's not go there." This prevents the robot from tricking the model into thinking a dangerous move is safe just because the model is guessing.

How It Works in Practice

The researchers tested UPSi in three different simulated worlds:

Pendulum: A robot arm trying to swing up without hitting a forbidden zone.
Cartpole: A classic balancing act where a pole must stay upright on a moving cart.
Drone: A flying robot trying to reach a target height without crashing.

In these tests, they compared UPSi against other safety filters. The results were clear:

Fewer Crashes: UPSi stopped the robot from making dangerous moves much more often than the previous methods. In the Cartpole test, the old method failed (crashed) 7.15% of the time, while UPSi only failed 0.75% of the time.
Just as Good at Learning: Even though UPSi was being extra careful, the robot still learned to play the game just as well as it did without the filter. It didn't slow down the learning process.

The "What If" and The "How Sure"

The authors are very careful about what they claim. They didn't just guess that UPSi works; they proved it mathematically that their "safety bubbles" (called reachable sets) are big enough to catch every possible outcome, even if the robot's model of the world is slightly wrong. They showed that if the robot stays inside these bubbles, it is mathematically guaranteed to stay safe.

However, they also admit that their current version is a bit slow for real-time use in very complex situations (like a fast drone). In their simulations, the filter took between 0.04 seconds and 25 seconds to make a decision, depending on the complexity. While this is fast enough for the tests they ran, they note that making it fast enough for real-world, split-second decisions is a challenge for the future.

The Takeaway

This paper bridges a gap between two worlds: the flexible, powerful learning of modern AI and the strict, reliable safety of traditional engineering. UPSi shows that we can use powerful, data-hungry AI models to teach robots complex skills, as long as we wrap them in a safety filter that knows when it doesn't know. It's a step toward robots that can explore the unknown without breaking the world.

Technical Summary: Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics

Problem Statement

Deep Reinforcement Learning (RL) faces significant challenges regarding exploration safety. While Predictive Safety Filters (PSFs) offer rigorous safety guarantees by leveraging Model Predictive Control (MPC) concepts, their practical applicability is often limited by their reliance on first-principles models or Gaussian Processes (GPs). These traditional model classes either demand substantial prior domain knowledge or fail to scale effectively with large datasets and high-dimensional systems. Conversely, modern Model-Based Reinforcement Learning (MBRL) methods utilize Probabilistic Ensemble (PE) neural networks to learn complex, high-dimensional dynamics from data with minimal prior knowledge. However, existing attempts to integrate PE models into PSFs lack rigorous uncertainty quantification, leading to heuristic safety checks that are prone to model exploitation under high epistemic uncertainty.

Methodology

The authors propose the Uncertainty-Aware Predictive Safety Filter (UPSi), a framework that bridges the scalability of PE-based MBRL with the rigorous safety guarantees of MPC. UPSi operates within a Dyna-style MBRL architecture (e.g., MBPO) and formulates future outcomes as reachable sets to enforce constraint satisfaction.

Core Components

Probabilistic Ensemble Dynamics: UPSi utilizes a PE model consisting of $E$ neural networks to approximate system dynamics. The model separates aleatoric uncertainty (irreducible process noise) from epistemic uncertainty (lack of data) via ensemble disagreement.
Certain Set ( $C_j$ ): To prevent model exploitation, UPSi defines a "certain set" $C_j \subseteq S \times A$ . Within this set, the model's predictive uncertainty is sufficiently low. The set is determined using a Kalman gain metric derived from sensor fusion principles, ensuring that predictions are only trusted where the model has sufficient data support.
Reachable Set Construction:
- The method overapproximates the Robust Reachable Sets (RRS) using ellipsoidal tubes propagated along a nominal trajectory.
- The tube shape matrix $P_{n|t}$ is updated by propagating the aleatoric variance and accounting for linearization errors.
- Crucially, the authors derive a rigorous upper bound on the linearization error ( $e_{max}$ ) caused by approximating nonlinear dynamics with first-order Taylor expansions. This error is explicitly added to the shape matrix propagation to ensure the reachable set is a valid overapproximation of the true system behavior.
Safety Filter Formulation: The safety filter solves a constrained optimization problem at each time step. It seeks an action sequence that minimizes deviation from the RL agent's proposed action while satisfying:
- State and input constraints ( $S, A$ ).
- Terminal set constraints ( $T_j$ ).
- Certainty Constraints: All predicted state-action pairs along the trajectory must remain within the certain set $C_j$ . This is enforced by tightening constraints based on the Lipschitz continuity of the uncertainty measure.
Terminal Set Expansion: To reduce conservatism, the terminal set $T_j$ is expanded iteratively using a data-driven approach. Feasible trajectories from previous episodes are used to approximate a larger controllable set, allowing the agent to explore further while maintaining safety.

Key Contributions

The paper presents three primary contributions:

Rigorous Reachable Sets from PE Models: The authors derive a method to compute provably overapproximating reachable sets from probabilistic neural network models. This involves separating aleatoric and epistemic uncertainty and explicitly bounding linearization errors to ensure the computed sets contain all possible system trajectories.
Explicit Certainty Constraint: A novel constraint is introduced to the PSF formulation that prevents the agent from exploiting model inaccuracies. By restricting the planning horizon to the "certain set," the filter ensures safety even when the model is uncertain.
Empirical Validation and Practical Integration: The authors validate the theoretical overapproximation and integrate a simplified version of UPSi into the MBPO framework. They demonstrate that this approach significantly improves exploration safety compared to prior neural-network-based PSFs (specifically XMPSC) while maintaining performance comparable to standard MBPO.

Experimental Results

The evaluation was conducted on three standard safe RL benchmarks: Pendulum, Cartpole, and Drone.

Reachable Set Verification: In the Cartpole environment, the proposed ellipsoidal tube construction was shown to consistently overapproximate the ground truth reachable sets (verified via 100,000 simulated trajectories), confirming the theoretical bounds.
Safety Performance: When integrated into MBPO, UPSi demonstrated a substantial reduction in constraint violations compared to XMPSC (a prior PE-based safety filter). For instance, in the Cartpole environment, the infeasibility rate dropped from 7.15% (XMPSC) to 0.75% (UPSi).
Performance: Despite the added safety constraints, UPSi achieved control returns on par with standard MBPO and SAC baselines.
Filtering Behavior: UPSi intervened more frequently than XMPSC but found feasible solutions more often, indicating a better balance between safety enforcement and exploration capability.

Significance and Claims

The paper claims to successfully bridge the gap between the scalability of modern MBRL methods and the rigorous safety guarantees of predictive safety filters. By enabling the use of flexible, data-driven PE models within a rigorous MPC framework, UPSi extends the applicability of safety filters to a broader class of high-dimensional, nonlinear systems that were previously difficult to model using first-principles or GPs.

The authors acknowledge limitations, noting that the practical implementation relies on simplifications (e.g., simplified tube propagation, no ancillary controller in experiments) that break formal guarantees, and that real-time feasibility for high-dimensional systems (like the Drone) remains a challenge. However, the work establishes a foundational approach for uncertainty-aware safety in learning-based control, suggesting that future improvements in real-time feasibility could enable deployment on more complex hardware.