⚡ electrical engineering

TRUST-UP: Trustworthy Reinforcement learning Using Safe Techniques for UAV Pursuit

This paper introduces TRUST-UP, a trustworthy reinforcement learning framework for UAV pursuit that combines Control Barrier Function-based safety filters with a transparent switching strategy to ensure provably safe and certifiable autonomous flight in congested urban environments.

Original authors: Yaosheng Deng, Mengtao Lyu, Junjie Gao, Jiaping Xiao, Mir Feroskhan

Published 2026-06-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yaosheng Deng, Mengtao Lyu, Junjie Gao, Jiaping Xiao, Mir Feroskhan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot drone to play a game of "tag" in a busy city park filled with people, other drones, and trees. You want the drone to be fast, smart, and able to chase a moving target.

The Problem: The "Technically Safe" but "Scary" Drone
Standard AI (Reinforcement Learning) is great at learning how to move quickly. However, it has a major flaw: it only cares about mathematical safety.

The Math View: If the drone swerves at the last second to avoid hitting a person's head, the math says, "Great job! No collision occurred."
The Human View: That same last-second swerve feels terrifying. It invades the person's personal bubble, making them feel unsafe and untrusting.

The paper argues that for drones to be allowed to fly around people, they need to be not just technically safe, but perceptually safe. They need to respect a "Trust Radius"—a larger, invisible bubble around a person that makes them feel comfortable, not just physically unharmed.

The Solution: TRUST-UP
The authors created a system called TRUST-UP (Trustworthy Reinforcement learning Using Safe Techniques for UAV Pursuit). Think of it as a "Smart Safety Co-Pilot" that sits between the AI's brain and the drone's motors.

Here is how it works, using simple analogies:

1. The "Wild" Pilot (The RL Model)

First, they train a standard AI (using a method called Soft Actor-Critic) to be a great pilot. This AI learns how to chase a target efficiently. However, like a reckless race car driver, it might try to make dangerous, jerky moves to win the race. It doesn't inherently understand human comfort zones.

2. The "Strict" Safety Filter (The CBFs)

This is the core innovation. Before the AI's command reaches the drone's motors, it passes through a Safety Filter.

The Analogy: Imagine the AI is a child running in a playground, and the Safety Filter is a strict but fair parent holding a leash.
How it works: The filter uses mathematical rules called Control Barrier Functions (CBFs). These rules act like invisible, flexible walls.
- Wall 1 (Collision): You cannot hit the person or the tree.
- Wall 2 (Sensing): You must stay close enough to see your target (like a dog on a leash that can't run too far away).
- Wall 3 (Engine Limits): You cannot push the drone's engine beyond its physical limits.

If the "Wild Pilot" tries to run into a wall, the "Parent" (the filter) instantly grabs the leash and redirects the drone to a safe path. The paper proves mathematically that this filter will always find a safe path, even if the wind is blowing or the target is moving erratically.

3. The "Switch" (Transparency)

The system has a special "Switch" that decides who is in control at any given moment.

Green Light: If the AI's plan is already safe and respects the "Trust Radius," the Switch lets the AI drive freely.
Red Light: If the AI tries to do something unsafe, the Switch instantly takes over, calculates the safest possible move, and executes it.
Why this matters: This makes the system "transparent." We can look at the code and say, "We know exactly why the drone stopped or turned," which is crucial for getting official approval (certification) to fly in cities.

4. The "Virtual Engine" Trick

The paper mentions a clever trick where they add a "virtual engine" to the math.

The Analogy: Imagine a car that can only accelerate so fast. If you suddenly demand a sharp turn, the car might skid. The authors' method adds a "virtual gear" that smooths out these sudden demands, ensuring the drone doesn't jerk around, which keeps the "Trust Radius" intact even during emergencies.

The Results: What Happened in the Simulations?

The authors tested this in a computer simulation with two drones chasing two targets around obstacles.

The "Wild" AI (SAC only): It crashed into obstacles, lost sight of its targets, and made erratic movements. It failed to keep the "Trust Radius."
The TRUST-UP System: The drones successfully chased their targets, avoided all obstacles, stayed within the required distance to "see" the target, and never violated the safety limits. Even when the targets made sudden "figure-8" turns or when wind blew, the TRUST-UP drones stayed calm and safe.

The Bottom Line

The paper claims that TRUST-UP bridges the gap between a fast, smart AI and the need for human trust. It takes a potentially dangerous AI and wraps it in a mathematically proven "safety suit" that guarantees the drone will never do anything that feels unsafe to a human, making it ready for real-world use in crowded urban skies.

Technical Summary: TRUST-UP

Problem Statement

The paper addresses the critical gap between the high performance of Reinforcement Learning (RL) in autonomous aerial vehicle (UAV) pursuit and the stringent safety and transparency requirements necessary for aviation certification, particularly in low-altitude urban environments with human presence.

While RL enables agile flight control, its "black-box" nature and lack of deterministic safety guarantees hinder airworthiness certification. A specific challenge identified is the concept of "perceived safety" or "trust radius." Unlike inanimate obstacles, humans require expanded safety margins based on psychological comfort and proxemics. Standard RL policies often prioritize technical collision avoidance (e.g., narrowly missing a pedestrian at high speed), which may technically satisfy collision constraints but violate human trust by invading personal space. Existing safe RL methods, such as reward shaping or static shielding, often fail to provide the deterministic guarantees required for human psychological trust or struggle with feasibility when enforcing complex, human-aware constraints alongside thrust limitations and sensing ranges.

Methodology: The TRUST-UP Framework

The authors propose TRUST-UP (Trustworthy Reinforcement learning Using Safe Techniques for UAV Pursuit), a framework that integrates a model-free RL policy with a formal safety filter based on Control Barrier Functions (CBFs). The system is designed to transform unsafe RL actions into provably safe flight commands while maintaining operational transparency.

1. System Augmentation and Transformation

To address the feasibility issues common in thrust-constrained systems, the authors augment the UAV dynamics by introducing virtual control inputs. This transforms the original first-order, thrust-constrained problem into a second-order, output-constrained problem. This transformation incorporates actuator inertia into the safety constraints, alleviating overload problems caused by abrupt input changes in traditional first-order CBFs and enhancing the feasibility of the resulting Quadratic Program (QP).

2. Adaptive Safety Filter Construction

The core of TRUST-UP is a safety filter constructed from three adaptive CBFs, formulated as a QP problem. These constraints ensure the forward invariance of three specific safety sets:

Input Constraint ( $C_{u,i}$ ): A time-varying constraint that dynamically adapts the maximum thrust allowance based on the relative position between the pursuer and the target. This allows for relaxed bounds during emergency maneuvers while strictly regulating erratic commands during nominal flight.
Collision Avoidance ( $C_{c,i}$ ): Ensures a minimum safe distance between the pursuer and all other agents (targets, other UAVs) and static obstacles.
Sensing Range ( $C_{s,i}$ ): Ensures the pursuer remains within a maximum distance to maintain sensor coverage of its target.

The filter accounts for system uncertainties (e.g., wind, sensor noise) by utilizing adaptive laws to estimate unknown parameters, ensuring robustness against disturbances.

3. Transparent Switching Strategy

The framework employs a hybrid control law with a switching strategy:

Region $R_1$ : If the nominal RL action ( $\pi_i$ ) satisfies all safety constraints (i.e., it lies within the admissible set), the system executes the RL action directly.
Region $R_2$ : If the RL action violates any constraint, the safety filter is activated. The system solves the CBF-QP to find the closest safe control input ( $v^*_i$ ) to the nominal action that satisfies all constraints.

The authors formally prove that this switching strategy ensures the QP problem satisfies the Karush-Kuhn-Tucker (KKT) conditions, guaranteeing the existence of a unique, Lipschitz continuous solution that maintains safety set invariance.

Key Contributions

Thrust-Constrained CBF Design: The paper introduces a method to handle thrust constraints by augmenting the system with virtual inputs. This approach enforces operational flight envelope limits while adaptively relaxing upper bounds during emergency maneuvers, balancing mission performance with verifiable safety.
Adaptive Safety Filter: The design of two position-based CBFs (for collision avoidance and sensing) combined with the thrust-constrained CBF forms a safety filter with guaranteed feasibility. The adaptive nature of these constraints allows UAVs to maintain safe operations even under atmospheric disturbances.
TRUST-UP Algorithm with Proven Feasibility: The proposal of a transparent switching strategy that determines the safety of RL outputs. The authors provide a formal proof that the algorithm satisfies KKT conditions for all safety constraints, ensuring compliance with aviation trustworthiness requirements for certified deployment.

Simulation Results

The authors validated TRUST-UP through numerical simulations involving two pursuit UAVs tracking targets in environments with static obstacles and unknown disturbances. Two scenarios were tested:

Circular Maneuvers: Targets performed large circular maneuvers.
Figure-8 Maneuvers: Targets performed complex "figure-8" trajectories with varying curvature.

Comparative Analysis:

SAC-Only vs. TRUST-UP: The baseline Soft Actor-Critic (SAC) algorithm, trained without the safety filter, failed to guarantee safety. In simulations, the SAC-only agents collided with obstacles and lost sensing of their targets (exceeding the sensing radius).
Safety Guarantees: TRUST-UP successfully maintained all safety constraints (collision avoidance, sensing range, and input limits) in both scenarios, even when targets performed aggressive evasive maneuvers.
Control Stability: The control inputs generated by TRUST-UP remained within prescribed bounds without high-frequency oscillations, whereas the SAC-only approach exhibited unstable tracking behavior under disturbances.
Computational Efficiency: The TRUST-UP CBF-QP implementation demonstrated improved online computational efficiency, reducing the mean QP solving time by approximately 14.1% compared to a standard input-constrained CBF-QP technique.

Significance and Claims

The paper claims that TRUST-UP addresses the fundamental deficiency in deploying autonomous systems in human environments: the disconnect between technical safety and perceived trustworthiness. By converting psychological safety zones (trust radii) into deterministic, hard constraints within a transparent safety filter, the framework bridges the gap between AI performance and aviation certification standards.

The authors emphasize that their work contributes toward certifiable and explainable AI frameworks for low-altitude aviation. Unlike methods that rely on reward shaping (soft penalties), TRUST-UP provides deterministic guarantees. The transparent switching strategy ensures that safety decisions are interpretable, satisfying the requirements for airworthiness certification in safety-critical aerial operations. The paper concludes that this approach is a necessary step toward trustworthy autonomous flight systems for future urban air mobility, particularly where human presence and psychological comfort are critical factors.