🤖 machine learning

ARMOR: Robust Reinforcement Learning-based Control for UAVs under Physical Attacks

The paper introduces ARMOR, a model-free reinforcement learning controller that ensures robust UAV operation under physical sensor attacks by employing a two-stage training framework to learn attack-resilient latent state representations without requiring privileged attack information during deployment.

Original authors: Pritam Dash, Ethan Chan, Nathan P. Lawrence, Karthik Pattabiraman

Published 2026-08-04

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Pritam Dash, Ethan Chan, Nathan P. Lawrence, Karthik Pattabiraman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to fly a drone. In the real world, robots don't have eyes like humans; they rely on a suite of sensors—GPS for location, gyroscopes for balance, and cameras for motion—to figure out where they are and how to move. This is the world of Reinforcement Learning (RL), a type of artificial intelligence where a computer learns to make decisions by trying things out and getting rewards for good behavior, much like a dog learning tricks with treats. The goal is to create a "brain" that can pilot a drone safely through complex tasks.

However, there is a sneaky problem: these sensors can be tricked. Just as a magician can fool your eyes, a hacker can send fake signals to a drone's sensors. This is called a physical attack. For example, they might send a fake GPS signal that makes the drone think it's in a different spot, or use sound waves to mess up its balance sensors. If the drone's brain believes these lies, it might crash or fly into a building. Scientists have tried to build "safe" brains that avoid danger, but most of these safety nets fail when the drone is being actively tricked by a hacker. They assume the world is just a bit messy, not that someone is actively lying to the robot.

This is where a new idea called ARMOR comes in. The researchers behind this project wanted to build a drone controller that can spot a lie even when the sensors are screaming it's the truth. They didn't just make the drone "tougher"; they taught it a new way of seeing the world. Instead of looking at the raw, easily-tricked sensor data, ARMOR learns to create a "secret summary" of what is really happening. The paper suggests that by using a special two-step training method, they can create a drone that stays on course even when its sensors are being sabotaged, without needing to be retrained for every single new trick a hacker might invent.

The Story of ARMOR: A Drone That Knows When It's Being Lied To

Meet ARMOR (Adaptive Robust Manipulation-Optimized State Representations). Think of ARMOR not just as a pilot, but as a detective that has learned to ignore fake alibis.

In the world of drone control, the usual method is to feed the computer raw data from the sensors: "I am at coordinate X," "I am spinning at speed Y." But if a hacker injects a fake signal, the computer believes the lie. Previous methods tried to teach the drone to be "paranoid" by training it against specific tricks (like a student studying only for one specific type of test question). This is slow, expensive, and if the hacker changes the trick slightly, the drone fails.

ARMOR takes a different approach. It uses a two-stage training framework, which is like having a master teacher and a student.

Stage 1: The Teacher with Super-Vision
First, the researchers train a "Teacher Encoder." This is a smart AI component that has a superpower: it has access to privileged information. In the simulation, the Teacher knows exactly when an attack is happening, which sensor is being lied to, and how big the lie is. It's like a teacher who can see the answer key while the student is taking the test.

Using this reference, the Teacher learns to translate the messy, lying sensor data into a robust latent representation. Imagine this as a "secret code" or a "truthful summary." Even if the GPS says the drone is in the ocean, the Teacher's secret code knows, "No, the GPS is lying; the drone is actually in the sky." The drone's control policy (its brain) learns to make decisions based on this secret code, not the raw sensor data. This allows the drone to learn how to fly perfectly even while under attack, because it's learning from the truth.

Stage 2: The Student Who Learns to See the Truth
Here is the tricky part: in the real world, the drone doesn't have a Teacher with a cheat sheet. It can't know when it's being attacked. So, the researchers train a Student Encoder.

The Student is like a detective who has to figure out the truth without the answer key. It only has access to the drone's history—the stream of sensor data from the last few seconds. The Student is trained to look at this history and guess the "secret code" that the Teacher generated. It learns to spot patterns, like "Hey, the GPS numbers are drifting slowly, which usually means a fake signal," or "The gyroscope is shaking in a weird rhythm."

Once the Student is good at guessing the secret code, the Teacher is retired. The drone is deployed with just the Student and the control policy. Now, when the drone flies in the real world and gets attacked, the Student analyzes the history of sensor data, generates the "truthful summary," and the control policy uses that to keep the drone flying straight.

What the Experiments Showed

The researchers tested ARMOR in a simulated 3D world, pitting it against five different types of physical attacks: GPS spoofing, gyroscope tampering, accelerometer tricks, magnetometer corruption, and optical flow spoofing. They compared ARMOR against other top methods like RARL (which uses adversarial training) and HRP (a hybrid recovery policy).

The results were quite clear in the simulations:

Survival Rate: When faced with attacks, ARMOR kept the drone flying successfully about 88% of the time. In contrast, the older methods struggled, with success rates ranging from 30% to 83%.
Crashes: Perhaps most impressively, ARMOR had 0% crash rate across all the attack types it was tested on. The other methods crashed frequently; for example, under GPS attacks, a competing method crashed 50% of the time.
Staying on Course: When the drone did get pushed off course, ARMOR kept the "state drift" (how far it wandered from the path) incredibly low, around 0.1 meters for GPS attacks. The other methods let the drone drift by several meters or degrees, which is enough to cause a crash or mission failure.

The "Zero-Shot" Superpower

The most exciting finding is what the authors call zero-shot generalization. Usually, if you train a robot to handle GPS lies, it might fail if you suddenly hit it with a gyroscope lie. It's like studying only for math and failing the history test.

ARMOR, however, showed it could handle unseen attacks without any extra training.

When they trained the drone only on GPS attacks and then tested it on gyroscope attacks (which it had never seen before), ARMOR still succeeded 60% of the time. The other method (RARL) failed completely, with a 0% success rate.
When they trained on gyroscope attacks and tested on GPS attacks, ARMOR succeeded 70% of the time, while the other method only managed 5%.

This suggests that by learning the "secret code" of what a real physical state looks like, the drone can recognize when any sensor is lying, even if it's a type of lie it hasn't practiced against.

Why This Matters (and What It Isn't)

The paper suggests that this two-stage method is a major step forward because it solves two big problems at once. First, it's faster and cheaper to train. Traditional methods require thousands of hours of "adversarial training" where a computer fights against a fake attacker over and over. ARMOR skips this by using the Teacher to learn the truth quickly, then teaching the Student to mimic that truth.

Second, it makes drones safer against a wider variety of threats. The authors note that while ARMOR is very good at handling single-sensor attacks and even some multi-sensor attacks, it does have limits. If an attacker messes with too many sensors at once in a complex way, the system might still struggle. Also, these results are currently based on simulations (computer models), not real-world flights with actual hacked hardware. The authors are careful to say that while the results are promising, the next step is to see if it works on real drones in the real world.

In short, ARMOR teaches a drone to stop listening to the noisy, lying sensors and start listening to the "secret summary" of reality. It's a clever trick that turns a vulnerable robot into a resilient one, capable of flying straight even when the world tries to trick it.

Technical Summary: ARMOR – Attack-Resilient Reinforcement Learning Control for UAVs

1. Problem Statement

Unmanned Aerial Vehicles (UAVs) rely heavily on onboard sensors (e.g., GPS, gyroscopes, accelerometers) for perception and control. These sensors are vulnerable to physical attacks, such as GPS spoofing, acoustic gyroscope tampering, and laser-based optical sensor spoofing. Such attacks inject malicious signals or noise, corrupting state estimates and leading to unsafe control actions, trajectory deviations, or crashes.

While Reinforcement Learning (RL) offers adaptive control, existing safe RL methods (e.g., shielding, Control Barrier Functions) are ineffective against physical attacks because they rely on predefined unsafe boundaries. Physical attacks can trick the controller into believing it is operating within safe limits while it actually deviates dangerously. Furthermore, standard adversarial training approaches suffer from three key limitations:

High Training Cost: Iterative generation of adversarial scenarios is computationally expensive.
Limited Generalizability: Policies often fail against attack patterns not seen during training.
Lack of Zero-Shot Effectiveness: They struggle to handle previously unseen attack modalities without retraining.

2. Methodology: ARMOR Framework

The authors propose ARMOR (Adaptive Robust Manipulation-Optimized State Representations), a model-free RL controller designed to operate robustly under adversarial sensor manipulation. Instead of relying on raw sensor observations, ARMOR learns a robust latent representation of the UAV's physical state via a two-stage offline training framework.

Stage I: Teacher Encoder Training (Privileged Information)

In this stage, the system operates in a fully observable simulation environment where "privileged information" is available. This information includes:

The specific sensor under attack.
The magnitude and pattern of the injected bias.
The duration of the attack.
The corrupted physical states.

A Teacher Encoder, implemented as a multi-head Variational Autoencoder (VAE), takes both the observable physical states ( $O_t$ ) and the privileged information ( $X_t$ ) as input. It generates a robust latent state representation ( $\bar{l}_t$ ) consisting of a mean ( $\mu_t$ ) and variance ( $\sigma_t$ ).

Architecture: The VAE includes an attack-type classifier and a reconstruction decoder. The variance $\sigma_t$ serves as an uncertainty estimate, allowing the policy to adjust behavior when uncertainty increases (e.g., during stealthy attacks).
Training: The control policy is trained using Proximal Policy Optimization (PPO) with $\bar{l}_t$ as input. The teacher encoder is trained to minimize a combined loss of reconstruction error, Kullback-Leibler (KL) divergence, and attack classification error.
Goal: To learn a policy that is highly resilient to state manipulations by leveraging attack-aware features.

Stage II: Student Encoder Training (Transfer Learning)

Since privileged information is unavailable in real-world deployment, a Student Encoder is trained to approximate the Teacher's latent representation using only historical onboard sensor data.

Architecture: The Student Encoder is a Temporal Variational Autoencoder (TVAE) built on a Long Short-Term Memory (LSTM) network. It processes a sliding window of historical physical states ( $H$ ) to capture temporal dependencies and the cumulative impact of adversarial disturbances.
Training Strategy: This stage uses supervised learning. The Student Encoder is trained to minimize the distance between its latent output ( $l_t$ ) and the Teacher's latent output ( $\bar{l}_t$ ). It also aligns the control actions produced by the policy when fed $l_t$ versus $\bar{l}_t$ .
Deployment: Once trained, the Student Encoder is used for online inference. The RL policy, originally trained with the Teacher's latent states, is reused unchanged, now accepting the Student's latent states as input.

3. Key Contributions

The paper outlines three primary contributions:

Two-Stage Training Framework: A novel approach that first trains an attack-resilient policy using privileged attack information, then adapts it for real-world deployment using only onboard sensor data via a student encoder.
Robust State Representation: A method to transform high-dimensional physical state information into a resilient latent vector. This representation is specifically optimized to withstand physical attacks, separating task-relevant features from attack-induced perturbations.
Transfer Learning Strategy: A mechanism enabling the RL controller to infer robust latent states without privileged information, relying solely on historical sensor data to achieve zero-shot generalization against unseen attacks.

4. Experimental Results

The authors evaluated ARMOR using a quadcopter simulation in a 3D environment (gym-pybullet) against five types of physical attacks: GPS spoofing, gyroscope tampering, accelerometer tampering, magnetometer corruption, and optical flow spoofing. They compared ARMOR against a Baseline-RL (no encoder), Hybrid Recovery Policy (HRP), and Robust Adversarial Reinforcement Learning (RARL).

Performance Metrics

Mission Success Rate: ARMOR achieved an average success rate of 88% across all attack types, compared to 83% for RARL and significantly lower rates for HRP and Baseline-RL.
Crash Rate: ARMOR incurred 0 crashes in standard attack scenarios, whereas Baseline-RL and HRP suffered high crash rates (up to 100% for Baseline-RL under GPS spoofing).
State Drift: ARMOR maintained minimal state drift (e.g., ~0.1m for GPS attacks) compared to the significant deviations seen in other methods.

Key Findings

Training Efficiency: ARMOR converged significantly faster than RARL. While RARL required ~1.2M timesteps to reach a specific reward level, ARMOR (both Teacher and Student policies) converged in under 600k timesteps, eliminating the need for iterative adversarial scenario generation.
Zero-Shot Generalization: ARMOR demonstrated strong zero-shot capabilities. When trained on a single attack type (e.g., GPS) and tested on unseen attacks (e.g., Gyroscope), ARMOR maintained a 60–70% success rate with low crash rates. In contrast, RARL achieved 0% success and high crash rates in similar zero-shot scenarios.
Stealthy Attacks: ARMOR effectively handled stealthy attacks (gradual bias accumulation), achieving an 80.7% success rate with 0 crashes, outperforming both HRP and RARL.

5. Significance and Claims

The authors claim that ARMOR represents a significant advancement in UAV safety by addressing the specific threat model of physical sensor attacks, which existing safe RL methods fail to mitigate. The significance of the work lies in:

Decoupling Learning from Deployment: By separating the learning of robust representations (using privileged info) from the inference phase (using only sensors), ARMOR achieves high robustness without the computational overhead of continuous adversarial training.
Generalization: The framework provides a path to zero-shot resilience, allowing UAVs to operate safely against attack modalities they have never encountered during training.
Practicality: The method eliminates the need for complex, iterative adversarial training loops, making the training process more efficient and scalable.

The paper acknowledges limitations, noting that generalization to simultaneous multi-sensor attacks is currently limited due to compounding perturbation effects, and future work will focus on extending the framework to other robotic platforms and integrating theoretical safety guarantees.