⚡ electrical engineering

MADR: MPC-guided Adversarial DeepReach

MADR introduces a novel framework that combines Model Predictive Control guidance with adversarial deep learning to overcome the limitations of traditional Hamilton-Jacobi reachability and physics-informed deep learning, enabling robust, high-dimensional, two-player zero-sum game solutions with superior convergence and performance on both simulated and real robotic systems.

Original authors: Ryan Teoh, Sander Tonkens, William Sharpless, Aijia Yang, Zeyuan Feng, Somil Bansal, Sylvia Herbert

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ryan Teoh, Sander Tonkens, William Sharpless, Aijia Yang, Zeyuan Feng, Somil Bansal, Sylvia Herbert

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where robots must navigate not just static obstacles like walls or trees, but also unpredictable forces and even other agents actively trying to stop them. This is the reality for autonomous systems, from drones flying through stormy weather to robots moving in crowded spaces. To keep these machines safe, engineers rely on a mathematical framework that asks a simple but profound question: given a starting point and a set of rules, can the robot guarantee it will reach its goal without crashing, even if everything goes wrong? This framework, known as reachability analysis, calculates a "safe zone" for the robot. If the robot stays within this zone, it is mathematically guaranteed to survive any worst-case scenario, whether that scenario is a sudden gust of wind or a rival robot trying to intercept it. However, for decades, this powerful tool has been stuck in a cage of its own making. The mathematics required to map out these safe zones become impossibly complex as the number of variables increases. A robot with just a few moving parts is easy to model, but a drone with many sensors and moving parts creates a problem so vast that traditional computers cannot solve it in a reasonable time. This limitation, often called the curse of dimensionality, has kept the most rigorous safety guarantees out of reach for real-world, high-dimensional machines.

A team of researchers has now broken through this barrier with a new approach called MADR, which stands for MPC-guided Adversarial DeepReach. The core idea is to teach a computer to learn the shape of these safe zones by combining two different methods. The first is a deep learning technique that tries to solve the underlying physics equations on its own, but it often struggles to converge on the right answer quickly or accurately. The second is a planning tool used by robots to decide their next move, which can simulate thousands of possible futures in seconds. The researchers realized that if they used the planning tool to generate high-quality examples of how a robot and an adversary would interact, they could use those examples to guide the learning process. Instead of letting the computer guess blindly, they fed it the results of these simulated battles, where one agent tries to stay safe and the other tries to force a collision. This guidance allowed the system to learn the safe zones much faster and with far greater precision than before.

The team tested this method on a wide variety of challenging scenarios, ranging from simple two-dimensional games to complex, high-dimensional simulations involving drones and humanoid robots. In one experiment, they modeled a drone flying at high speed toward a pillar while facing strong, unpredictable wind. The new method successfully kept the drone safe in nearly all cases, whereas previous approaches failed to avoid the obstacle when the wind was particularly fierce. In another test, they simulated a game of tag between two robots, where one tries to catch the other. The system learned strategies that were nearly as good as the theoretical best possible solution, a benchmark that had previously been impossible to reach for such complex systems. The researchers also took their work out of the computer and into the real world. They equipped small drones and ground robots with their new safety software and watched them play games of pursuit and evasion in a motion-capture arena. Even when the robots were flying for minutes at a time, far longer than the short simulations used to train them, the system held up. The robots successfully avoided capture or caught their targets, demonstrating that the safety guarantees learned in a few seconds of training could hold true over much longer periods.

What makes this achievement particularly significant is how the system handles the "adversary." In many safety systems, the computer assumes the worst-case scenario is a random, chaotic disturbance. But in a game of tag or a military drone encounter, the opponent is intelligent and deliberate. The new framework treats the opponent as a strategic player who is actively trying to minimize the robot's safety. By training the system to anticipate this specific type of intelligent opposition, the resulting safety zones are more robust. The researchers found that when they compared their method against other state-of-the-art techniques, their approach consistently produced safer paths and more accurate predictions of where a robot could go. In the hardware tests, the robots were able to execute complex maneuvers, such as a drone diving to avoid a human operator or a ground robot cornering an opponent, all while adhering to the strict safety boundaries calculated by the new algorithm.

The success of this work suggests that we are moving closer to a future where autonomous systems can operate safely in dynamic, unpredictable environments without needing to be overly cautious or slow. By bridging the gap between rigorous mathematical theory and practical, data-driven learning, the researchers have shown that it is possible to scale safety guarantees to the complex machines of tomorrow. The system does not just guess where it is safe; it learns from the worst possible interactions to build a shield that holds up even when the rules of the game are being played by an opponent. This is not a magic trick that solves every problem instantly, but a concrete step forward that allows engineers to trust robots with more difficult tasks, knowing that the mathematics behind their safety has been tested against the very worst that the world can throw at them.

Technical Summary: MADR: MPC-guided Adversarial DeepReach

Problem Statement

Hamilton-Jacobi (HJ) Reachability provides a rigorous framework for generating safe value functions and policies in zero-sum differential games, effectively modeling interactions between an ego robot and adversarial entities (passive disturbances or active opponents). However, classical grid-based HJ methods suffer from the "curse of dimensionality," limiting their application to systems with fewer than six states. While physics-informed deep learning (e.g., DeepReach) offers scalability by replacing grids with function approximators, these methods often struggle with slow convergence and inaccurate solutions in high-dimensional, adversarial settings due to weak Partial Differential Equation (PDE) gradients and the complexities of self-supervised learning. Existing attempts to improve convergence via supervised learning have largely been restricted to single-player problems or simple games, failing to robustly address general two-player, zero-sum differential games.

Methodology

The authors propose MADR (MPC-guided Adversarial DeepReach), a framework designed to robustly approximate the value function for two-player, zero-sum differential games. The core insight is to enrich the self-supervised learning of the HJ-Isaacs Variational Inequality (HJI-VI) with supervision derived from Sampling-based Model Predictive Control (SB-MPC) rollouts.

Key Algorithmic Components

Value-Only Approach: Unlike actor-critic reinforcement learning methods that learn policies and value functions simultaneously (often leading to instability), MADR focuses on learning a single, high-quality value function. Optimal policies for both players emerge directly from the gradients of this learned value function.
Adversarial MPC Rollouts: To generate supervised data, the framework runs parallel SB-MPC processes:
- Control Perspective: The agent optimizes its control sequence while the disturbance follows a policy derived from the current value function gradient (minimizing safety).
- Disturbance Perspective: The disturbance optimizes its sequence while the agent follows a policy derived from the value function gradient (maximizing safety).
- These rollouts generate a dataset of state-action-cost tuples, which serve as a supervisory signal to correct the value network.
Loss Function: The training combines the standard PDE-based loss (satisfying the HJI-VI) with a supervised loss ( $L_{MPC}$ ) that minimizes the difference between the learned value function and the costs observed in the MPC rollouts.
MADR-FOLLOW Filter: To address suboptimality in long-horizon games where the approximated solution quality may erode, the authors introduce a specialized filter for the pursuer (Player II). If the pursuer cannot immediately "catch" the evader, this policy switches to a "following" strategy that minimizes the distance to the evader rather than attempting an immediate capture, ensuring the pursuer remains close to the optimal trajectory.

Key Contributions

General Framework for Zero-Sum Games: MADR extends supervised learning for HJ reachability beyond single-player or linear cases to general two-player, zero-sum differential games with nonlinear dynamics.
Robust Value Approximation: By leveraging worst-case control-disturbance interactions as expert guidance, the method significantly accelerates convergence and improves the fidelity of the learned value function compared to self-supervised baselines.
Policy Generation: The framework yields optimal strategies for both players in zero-sum games and safe policies for worst-case robustness without requiring separate actor networks.
Scalability: The approach successfully scales to high-dimensional systems (up to 20 states) where grid-based methods are intractable.

Experimental Results

The authors validated MADR through extensive simulations and hardware experiments across diverse robotic platforms:

13D Quadrotor (Wind Disturbances): In a high-dimensional simulation with wind disturbances, MADR achieved a 98.9% safe rate, significantly outperforming the ISAACS baseline (86.6%). MADR demonstrated superior accuracy in value estimation, rarely overestimating safety margins.
6D Dubins Pursuit-Evasion: Compared against Dynamic Programming (DP) ground truth, MADR achieved an Intersection-over-Union (IOU) of 0.997 with the true backward reachable tube, outperforming Vanilla DeepReach (0.969) and ISAACS (0.928). In policy matchup tables, MADR policies performed near-optimally against various opponents.
20D Drone Pursuit-Evasion: In a 20-dimensional drone game, the MADR evader successfully avoided capture from all competing policies except its own pursuer, while the MADR pursuer performed comparably to MPC.
Hardware Demonstrations:
- TurtleBots (6D): In 500-second real-world trials, MADR policies achieved capture times comparable to DP optimal solutions, significantly outperforming Vanilla DeepReach.
- Crazyflie Drones (20D): Real-world drone-vs-drone experiments confirmed the effectiveness of the MADR-FOLLOW strategy, which achieved a higher capture rate (26.4%) compared to the standard MADR policy (11.6%) over 3-minute trajectories.
- Humanoid vs. Drone: Experiments involving a teleoperated Unitree G1 humanoid demonstrated the framework's robustness in mixed-competence scenarios, with the drone successfully executing pursuit and evasion maneuvers against a human-driven agent.

Significance and Claims

The paper claims that MADR bridges the gap between the rigorous, theory-driven framework of HJ reachability and the practical application of optimal control in real-world, adversarial environments. By integrating MPC-guided supervision, the method overcomes the infeasibility of grid-based methods and the convergence issues of pure self-supervised learning. The authors emphasize that their approach enables reliable decision-making under uncertainty for safety-critical autonomous systems. They position MADR not as a replacement for theoretical guarantees, but as a practical learning framework that preserves the safety properties of HJ reachability while scaling to the complex, high-dimensional dynamics found in modern robotics. Future work is identified as focusing on establishing stronger theoretical grounding, including convergence analysis and formal safety guarantees.