🤖 machine learning

ASGARD: Action-Space Guard for UAV Resilience via Reinforcement Learning

The paper introduces ASGARD, a two-phase teacher-student framework that enhances the resilience of Reinforcement Learning-based UAV controllers against runtime action-space attacks by training a monitor to generate corrected action commands using only physical state history.

Original authors: Mohsen Salehi, Karthik Pattabiraman

Published 2026-09-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohsen Salehi, Karthik Pattabiraman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Drones are no longer just remote-controlled toys; they are sophisticated machines that navigate the sky using software to interpret the world and decide where to fly. At the heart of this autonomy lies a type of artificial intelligence called reinforcement learning, where a computer program learns to control a vehicle by trial and error, much like a child learning to ride a bicycle. The system observes its surroundings, tries a movement, and receives feedback on whether that movement brought it closer to its goal. Over time, it builds a strategy to fly safely and efficiently. However, this reliance on software creates a vulnerability. Just as a human pilot can be tricked by a false signal, a drone's computer can be deceived. While researchers have long worried about hackers feeding false data into the drone's sensors, a more subtle and dangerous threat has emerged: attacks that happen after the drone has already decided what to do.

Imagine a drone has calculated that it needs to tilt slightly to the left to stay on course. The computer sends this instruction to the motors. In a specific type of cyberattack, an intruder intercepts that instruction in the split second between the computer's decision and the motor's action, altering the command before it is executed. The drone's brain thinks it is flying correctly, but its body is being forced to move in a different, dangerous direction. This is known as an action-space attack. Because the attack happens after the decision is made, traditional safety systems that check the drone's sensors or the computer's logic often miss it entirely. The drone appears to be thinking clearly, even as it is being hijacked.

To address this invisible threat, researchers at the University of British Columbia have developed a new defense system called ASGARD. The system is designed to act as a guardian for the drone's commands, ensuring that what the motors receive is exactly what the computer intended, even if an attacker tries to tamper with the message. The researchers built this system using a two-stage training process that mimics how a master teacher might train a student. In the first stage, the "teacher" system learns to fly while having access to secret information about the attacks, such as knowing exactly when and how an intruder is trying to mess with the controls. This privileged knowledge allows the teacher to understand the difference between a safe command and a corrupted one. It learns to generate a hidden signal that captures the true intent of the flight, regardless of the noise in the environment.

Once the teacher has learned this skill, it passes the knowledge to a "student" system that will actually fly the drone in the real world. The student does not have access to the secret information about the attacks; it only sees the history of the drone's physical movements, such as its position, speed, and orientation. Through careful training, the student learns to reconstruct the teacher's hidden signal using only these observable facts. This allows the student to recognize when a command has been altered, even without knowing the specific details of the attack. The system includes a lightweight monitoring component that sits between the decision-making software and the motors. This monitor constantly checks the outgoing commands against the hidden signal it has reconstructed. If it detects a discrepancy, it instantly corrects the command before it reaches the motors, effectively neutralizing the attack in real time.

The researchers tested this approach in a simulated environment where a drone had to fly through a series of checkpoints. They subjected the drone to various types of attacks, including those that tried to force it to pitch forward, roll sideways, or change its speed. In these tests, a standard drone controller without this protection crashed in nearly half of the scenarios. Even a previous, advanced defense system designed to handle sensor attacks failed completely when faced with these action-space intrusions, crashing in every single attempt when all four control channels were attacked at once. In contrast, the ASGARD system kept the drone flying safely. When only one control channel was attacked, the system successfully completed 95% of the missions without a single crash. Even in the most difficult scenario, where all four control channels were attacked simultaneously, the system managed to complete two-thirds of the missions, a feat the other systems could not achieve.

The system also proved to be surprisingly adaptable. The researchers trained the drone to defend against attacks on just one specific control, such as the pitch, and then tested it against attacks on the other controls it had never seen before. The system generalized well, successfully defending against these unseen threats and completing the majority of missions. This suggests that the system learned a fundamental understanding of how to detect corruption in the control flow, rather than just memorizing specific attack patterns. Furthermore, the system remained effective against stealthy attacks that slowly increased the interference over time, a tactic that often catches other defenses off guard. By continuously tracking the drone's history and adjusting its corrections as the disturbance grew, the system prevented the drone from drifting off course.

The results indicate that this two-phase approach offers a robust solution to a problem that has left autonomous drones vulnerable. By placing a smart, corrective layer between the decision-making software and the physical motors, the system ensures that the drone executes its intended path, even when an attacker tries to hijack the command line. While the study was conducted in simulation, the findings suggest a clear path toward making autonomous flight safer in an increasingly connected and potentially hostile world. The work demonstrates that resilience against cyberattacks does not require stopping the drone or landing it in an emergency; instead, it can be achieved by actively repairing the commands in real time, allowing the machine to continue its mission with confidence.

Technical Summary: ASGARD – Action-Space Guard for UAV Resilience via Reinforcement Learning

Problem Statement

Reinforcement Learning (RL) has become a standard for Unmanned Aerial Vehicle (UAV) navigation and control due to its adaptability to complex environments. However, RL-based controllers are vulnerable to action-space attacks. Unlike physical attacks (e.g., GPS spoofing) that corrupt sensor inputs, action-space attacks occur after the control policy generates an action command but before the actuators execute it. Attackers can overwrite these commands in memory (e.g., via data-only attacks or memory corruption) to inject biases into control channels such as pitch, roll, thrust, or gain.

Existing defenses, such as robust-RL methods (e.g., ARMOR, RARL), primarily focus on hardening the policy against input perturbations or training against adversarial inputs. Once deployed, these systems forward the policy's output directly to the actuators without a mechanism to intervene if the action has been corrupted in transit. Consequently, even if the policy generates a "safe" command, the UAV executes a corrupted one, leading to mission failure or crashes.

Methodology: The ASGARD Pipeline

The authors propose ASGARD (Action-Space GuARD), a two-phase teacher–student pipeline designed to recover corrupted action commands at runtime without requiring privileged information during deployment. The system consists of three core components: an encoder, a control policy, and a monitor.

1. Teacher Phase (Training with Privileged Information)

In this phase, the system is trained with access to privileged information regarding the attack (e.g., the target action channel, attack bias, and duration), which is unavailable at runtime.

Teacher Encoder: Implemented as a Variational Autoencoder (VAE), it takes the UAV's physical state ( $o_t$ ) and the attack context ( $x_t$ ) to produce an action-attack-aware latent representation ( $\bar{l}_t$ ).
Control Policy: Trained using Proximal Policy Optimization (PPO) using $\bar{l}_t$ as input to generate intended actions ( $\bar{a}_t$ ).
Teacher Monitor: A lightweight Multilayer Perceptron (MLP) placed between the policy and actuators. It observes the latent ( $\bar{l}_t$ ), the potentially corrupted action ( $\bar{a}'_t$ ), and an expected next state ( $\hat{s}_{t+1}$ ) predicted by an auxiliary head. The monitor is trained via supervised regression to output a corrected action ( $\tilde{\bar{a}}_t$ ) that matches the uncorrupted intent, effectively learning to repair corrupted commands before they reach the hardware.

2. Student Phase (Runtime Deployment)

This phase adapts the system for onboard execution where privileged attack information is absent.

Student Encoder: Implemented as a Temporal VAE using a Long Short-Term Memory (LSTM) network. It is trained via supervised learning to approximate the teacher's latent representation using only the history of physical states and previously executed actions ( $H_t$ ).
Student Monitor: Trained to mimic the teacher monitor's corrections using the student latent ( $l_t$ ) instead of the privileged latent.
Execution: At runtime, the Student Encoder, Control Policy, and Student Monitor operate together. The monitor actively intercepts every action command, correcting it if necessary, and forwards the safe command to the actuators.

Key Contributions

Runtime Recovery Mechanism: Unlike prior action-space defenses that retrain policies to be robust against perturbations, ASGARD introduces a mechanism to recover corrupted actions at runtime. It actively corrects commands rather than merely detecting attacks or halting the system.
Two-Phase Teacher–Student Pipeline: The authors propose a training scheme where a teacher with privileged attack knowledge supervises a student that relies solely on physical state history. This allows the system to learn attack-aware representations without requiring real-time attack data during deployment.
Active Correction Monitor: The introduction of a lightweight MLP monitor that sits between the policy and actuators. This component learns to distinguish between safe and corrupted actions, forwarding safe commands unchanged while repairing corrupted ones, ensuring the UAV remains under safe control signals rather than falling back to degraded modes.

Experimental Results

The authors evaluated ASGARD on a quadcopter waypoint-following task using the gym-pybullet simulator, comparing it against a baseline RL controller and the state-of-the-art ARMOR defense.

Single-Channel Attacks: Under attacks targeting individual channels, ASGARD achieved high mission success rates with zero crashes. Specifically, it achieved 95% success for Pitch, 94% for Gain, 94% for Roll, and 97% for Thrust, with 0% crash rates across all four single-channel scenarios. In contrast, the baseline RL controller succeeded in ~40% of missions (47% crash rate), and ARMOR succeeded in ~50% (36% crash rate).
Combined Attacks: When all four action channels were corrupted simultaneously (a worst-case scenario), both baseline RL and ARMOR failed 100% of missions. ASGARD maintained a 67% success rate with only a 10% crash rate.
Zero-Shot Generalization: ASGARD demonstrated strong generalization to unseen attack channels. When trained on a single attack type (e.g., Pitch) and tested on others (Roll, Thrust, Gain), it achieved 2–3 times the success rate of ARMOR and 5–6 times lower crash rates.
Stealthy Attacks: Against stealthy attacks that gradually accumulate bias, ASGARD maintained 100% success, whereas ARMOR failed completely. The authors attribute this to ASGARD's ability to track bias accumulation via the latent history, whereas ARMOR lacks a runtime correction stage.
Training Efficiency: The teacher–student pipeline converged faster (approx. 5x) and with less variance than ARMOR under adversarial training conditions.

Significance and Claims

The paper claims that ASGARD addresses a critical gap in UAV security: the inability of existing robust-RL methods to handle action-space attacks that occur after policy inference. By shifting the defense strategy from "hardening the policy" to "actively correcting the output," ASGARD ensures mission continuity even when control commands are compromised.

The authors emphasize that their approach allows UAVs to complete missions despite runtime corruption of action commands, a capability previous defenses lacked. They note that while the current evaluation is restricted to a single quadrotor model in simulation, the architecture is designed to be lightweight enough for onboard deployment. The paper concludes that extending this framework to other robotic systems and achieving sim-to-real transfer remain open directions for future work.