⚡ electrical engineering

Neural Backward Reach-Avoid Tubes with MPC Supervision for High-Dimensional Systems: An Application to Safe Spacecraft Docking

This paper proposes a learning-based Backward Reach-Avoid Tube framework that integrates Hamilton-Jacobi structure with MPC supervision to enable safe, high-dimensional spacecraft docking by training a neural value function offline and deploying it via gradient-driven and terminal MPC controllers online, achieving superior success rates and efficiency compared to existing methods.

Original authors: Santiago Thorup, Luca Castelletto, Zeyuan Feng, Somil Bansal

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Santiago Thorup, Luca Castelletto, Zeyuan Feng, Somil Bansal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guide a tiny, delicate drone (the "chaser") into a moving, spinning docking port on a massive space station (the "target"). You have to do this perfectly: you must get close enough to lock on, but if you get too close or hit the wrong angle, you crash.

The problem is that space physics is incredibly complex. The drone moves forward, backward, left, right, spins, and tilts all at once. This creates a "13-dimensional" puzzle. Trying to solve this puzzle using traditional math is like trying to map every single grain of sand on a beach to find a path; the computer simply runs out of memory and time before it can finish.

This paper introduces a new way to teach a computer how to solve this puzzle safely and quickly. Here is how they did it, explained simply:

The Problem: The "Curse of Dimensions"

Think of the space around the target as a giant, multi-layered maze.

Old Way (Grid Solvers): Imagine trying to map this maze by drawing a grid on every single inch of the floor. In a small room (low dimensions), this works. But in a 13-dimensional space, the number of grid points becomes so huge (more than the number of atoms in the universe) that no computer can handle it.
The Learning Way (Neural Networks): Instead of drawing a grid, the authors trained a "neural network" (a type of AI brain) to learn the shape of the safe path. However, teaching this AI to avoid crashing and reach the goal at the same time is tricky. The AI often gets confused, thinking it's safe when it's actually about to crash, or it gets stuck because the math is too flat and gives no clues on which way to turn.

The Solution: The "MPC Supervisor"

To fix the confused AI, the authors added a "Supervisor" to the training process.

The Teacher (MPC): They used a powerful, slow computer algorithm called Model Predictive Control (MPC) to act as a teacher. This teacher can solve the puzzle perfectly but is too slow to use in real-time.
The Student (Neural Network): The AI student tries to learn the path.
The Curriculum: Instead of asking the student to solve the whole 13D maze on day one, they used a "curriculum."
- First, they let the student practice on short, easy paths.
- The teacher (MPC) checks the student's work. If the student is doing well, the teacher says, "Okay, now try a slightly longer path."
- If the student makes a mistake, the teacher doesn't just say "Wrong." It generates a specific, correct path for that exact spot to show the student what to do.
- This creates a feedback loop: as the student gets smarter, the teacher's examples get more precise, helping the student learn faster and more accurately.

The Result: A "Safe Tube"

Once the AI is trained, it creates a Backward Reach-Avoid Tube (BRAT).

The Metaphor: Imagine a glowing, invisible tunnel surrounding the target.
- Inside the tunnel: You are guaranteed to reach the docking port safely without crashing.
- Outside the tunnel: You are in danger.
The AI doesn't just know where the tunnel is; it knows exactly which way to steer to get into the tunnel and stay there.

How It Works in Real-Time (The Online Phase)

When the actual spacecraft is flying, it uses two modes:

The "Get to the Tunnel" Mode: If the ship is far away (outside the tunnel), the AI steers it toward the glowing tunnel entrance.
The "Docking" Mode: Once inside the tunnel, the AI switches to a precision mode to gently guide the ship into the port.
The Safety Net: Even if the AI makes a mistake or the ship wanders too close to the edge, a "safety filter" acts like an emergency brake, instantly overriding the AI to push the ship away from a collision if it gets too close to the target body.

Why This Matters

The authors tested this on two scenarios:

A 6-Dimensional Test: They compared their AI against the "perfect" grid-based solution. Their AI was just as safe but docked faster because it didn't suffer from the "pixelation" errors of the grid method.
The Full 13-Dimensional Test: This is the real deal. The grid method couldn't even run. Their AI succeeded in docking 98.88% of the time with almost zero collisions.
- Speed: The AI makes a decision in 16 milliseconds.
- Comparison: The old "perfect" optimization methods took 12 seconds to make a decision (too slow for real-time). Other learning methods failed almost 100% of the time.

Summary

The paper presents a way to teach an AI to dock a spacecraft in high-dimensional space by using a slow, perfect "teacher" to train a fast, smart "student." The result is a system that is safe, fast, and capable of solving problems that were previously impossible for computers to handle. It ensures the spacecraft can find its way into a tiny, moving target without crashing, even in the complex physics of orbit.

Technical Summary: Neural Backward Reach-Avoid Tubes with MPC Supervision for High-Dimensional Systems

Problem Statement

Autonomous spacecraft docking requires control policies that guarantee both collision avoidance and target reachability under tightly coupled, high-dimensional translational-rotational dynamics. While Hamilton–Jacobi (HJ) reachability provides formal guarantees for such "reach-avoid" problems, classical grid-based solvers are limited by the curse of dimensionality, typically restricting practical application to systems with five or six dimensions. Spacecraft docking, involving full 6-degree-of-freedom (6-DOF) dynamics, results in a 13-dimensional state space, rendering grid-based methods computationally intractable.

Existing learning-based approaches, such as DeepReach, attempt to approximate HJ value functions using neural networks to scale to higher dimensions. However, these methods struggle specifically in reach-avoid settings where the goal set (docking port) and failure set (collision zone) are tightly coupled. The nested min-max structure of the reach-avoid variational inequality (VI) produces vanishing gradients in large regions of the state space, leading to unstable training and suboptimal policies. Furthermore, purely PDE-based training often fails to converge on long horizons required for docking tasks.

Methodology

The authors propose a learning-based Backward Reach-Avoid Tube (BRAT) framework that integrates HJ structure with Model Predictive Control (MPC) supervision to address these challenges. The approach consists of an offline training phase and an online deployment phase.

Offline Training: MPC-Supervised Curriculum Learning

The core innovation is a training strategy that augments standard PDE residual minimization with curriculum-driven MPC supervision.

Neural Architecture: The HJ value function is approximated using a SIREN (Sinusoidal Representation Networks) neural network with an exact-boundary-condition parameterization to ensure boundary constraints are satisfied by construction.
Hybrid Loss Function: The network is trained by minimizing a combined loss:
- PDE Loss: Penalizes violations of the reach-avoid variational inequality.
- MPC Supervision Loss: Uses value targets generated by an MPC solver. These labels provide direct supervision in regions where the PDE residual signal is weak (e.g., far from the active boundary).
Train-Verify-Refine Cycle: To handle the temporal challenge of learning long-horizon value functions, the authors introduce an active feedback loop. Training proceeds in a curriculum that gradually expands the time horizon. At specific checkpoints, training pauses to validate the current value function against MPC cost estimates on a held-out set. If validation fails, training continues at the current horizon. Crucially, MPC labels are regenerated at each checkpoint using the current learned policy as a "warm start," creating a bootstrapping loop where improved network quality leads to better supervision labels.

Online Deployment: Dual-Controller Architecture

The learned value function is deployed via two real-time controllers, both backed by a least-restrictive safety filter (an avoid-only BRT) to ensure safety even when the system is outside the learned BRAT.

BRAT Gradient Controller:
- Convergence Phase: When the state is outside the BRAT ( $V > 0$ ), the controller uses the value function gradient to drive the system toward the BRAT interior using bang-bang control.
- Precision Phase: Once inside the BRAT ( $V \leq 0$ ), the controller performs a minimum-time search to find the tightest time slice where the state remains safe, applying bang-bang control for precise terminal guidance.
Terminal MPC Controller: This controller utilizes short-horizon MPC with the learned value function serving as a differentiable terminal cost, combining the tractability of short-horizon planning with the long-horizon reach-avoid information encoded in the neural network.

Key Contributions

Extension to Reach-Avoid Problems: The authors extend MPC-supervised DeepReach to reach-avoid problems, introducing a curriculum-based training strategy with a train-verify-refine loop to enable stable learning in high-dimensional systems.
Novel Controller Suite: Development of two real-time controllers (BRAT-gradient and value-function-augmented Terminal MPC) that leverage the learned value function for safe docking.
High-Dimensional Validation: Evaluation on spacecraft docking models of increasing fidelity, including a 6D planar system (validated against grid-based ground truth) and the full 13D orbital system.
Performance Superiority: Demonstration that the proposed method outperforms existing MPC, Vanilla DeepReach, and Reinforcement Learning (RL) baselines in both success rate and computational efficiency.

Results

The framework was evaluated on a 6D planar docking problem and a full 13D orbital docking problem.

6D Planar Docking:
- Accuracy: The learned BRAT closely matched the grid-based ground truth, achieving an 81.12% True Positive Rate (TPR) and a low 0.15% False Positive Rate (FPR) against the grid solution.
- Performance: The proposed BRAT controller achieved a 99.6% docking success rate with 0% collisions, matching the grid-based ground truth (99.8% success).
- Efficiency: The BRAT controller operated in real-time with a mean computation time of 10.74 ms per step, significantly faster than standard MPC (5880 ms) and Terminal MPC (2246 ms).
- Optimality: The learned controller achieved slightly faster mean docking times (29.17s) compared to the grid-based solution (30.18s), attributed to the neural network's ability to mitigate discretization errors inherent in grid-based solvers.
13D Orbital Docking:
- Scalability: The framework successfully scaled to the 13D system, a regime where grid-based solvers are intractable.
- Performance: The BRAT controller achieved a 98.88% docking success rate with a 0.58% collision rate across 10,000 uniformly sampled initial conditions.
- Comparison: Baselines failed significantly in 13D. Standard MPC had a 27% collision rate and 73% timeout rate. Vanilla DeepReach and RL baselines failed almost entirely (99.4% timeout). Terminal MPC achieved 83% success but required ~12.7 seconds per control step, rendering it impractical for real-time onboard execution. The proposed BRAT controller maintained a 16.00 ms per-step execution time.

Significance and Claims

The paper claims that this work addresses the critical gap in applying formal reach-avoid guarantees to high-dimensional, coupled dynamical systems like spacecraft docking. By tightly integrating MPC supervision with PDE-based learning, the authors overcome the instability of purely PDE-driven training in reach-avoid settings.

The significance lies in the ability to synthesize controllers that are:

Formally Safe: Capable of providing reach-avoid guarantees via the HJ framework.
Scalable: Effective in 13-dimensional state spaces where traditional grid methods fail.
Computationally Efficient: Capable of real-time execution on onboard hardware, unlike optimization-based MPC which is too slow for high-dimensional problems.
Robust: Capable of handling the tight coupling between goal and failure sets that typically confounds learning-based approaches.

The authors note that the current formulation assumes a disturbance-free environment and a stationary target. Future work is identified as extending the framework to include bounded disturbances via differential game formulations and handling tumbling targets with time-varying goal sets.