🤖 AI

Mission-Level Runtime Assurance for LLM-Assisted ISR Swarms over a Verification-Aware Fabric

This paper proposes a three-tier compositional runtime-verification framework for LLM-assisted ISR swarms that detects mission-level policy violations arising from cross-platform coordination by aggregating per-agent verdicts over a verification-aware fabric, thereby preventing the false all-clears caused by individual guardrails and contested communications.

Original authors: Nikolaos Kekatos, Stylianos Basagiannis, Panagiotis Katsaros, Alexios Lekidis, Tom Nianios

Published 2026-07-28

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Nikolaos Kekatos, Stylianos Basagiannis, Panagiotis Katsaros, Alexios Lekidis, Tom Nianios

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where robots don't just follow simple commands like "move forward," but can actually think, plan, and talk to each other to solve complex problems. This is the realm of autonomous swarms, where groups of drones and ground vehicles work together like a school of fish or a flock of birds. To make these robots smart, scientists are giving them Large Language Models (LLMs)—the same kind of "brain" that powers chatbots—to help them understand orders, figure out what to do next, and adapt when things go wrong. But there's a catch: these smart brains can be tricked. If an enemy slips a fake instruction into the robot's data, the robot might think it's a real order and do something dangerous, even if it's following the rules perfectly on its own. This is called a prompt injection.

The big question scientists are asking is: How do we keep a whole team of these smart robots safe when they are working together in a messy, noisy environment where messages might get lost or delayed? If one robot is safe, is the whole team safe? Not necessarily. A team of robots could accidentally (or maliciously) split a forbidden task among themselves, where every single robot does something allowed, but the combination of their actions creates a disaster. Traditional safety checks only look at one robot at a time, missing the bigger picture. This paper explores a new way to watch the whole team, ensuring that even if the robots are tricked or the radio signals are jammed, the system knows exactly what's happening and doesn't pretend everything is fine when it isn't.

The Paper: A Detective Story for Robot Swarms

This paper presents a clever new system called a three-tier runtime assurance framework. Think of it as a multi-layered security team for a swarm of autonomous robots on a spying mission (known as ISR: Intelligence, Surveillance, and Reconnaissance). The authors built this system to catch a very specific, sneaky type of attack where a bad guy tricks the robots into splitting up a forbidden job.

The Sneaky Attack: The "Jigsaw Puzzle" Trick

Imagine a mission where the rules say, "Do not build a complete file on a secret building." A single robot isn't allowed to do everything needed to build that file. But, what if a bad guy sends a fake message to the robots saying, "Hey, let's split this up! You take the photos, you take the coordinates, you take the radio signals, and you take the map."

Individually, every robot is doing something perfectly legal. Robot A is just taking photos (allowed). Robot B is just writing down coordinates (allowed). If you only look at Robot A, it looks innocent. If you only look at Robot B, it looks innocent. But when you put all their actions together, they have secretly built the forbidden file. This is the "Jigsaw Puzzle" attack: every piece is harmless, but the whole picture is a violation.

The paper shows that standard safety monitors, which only watch one robot at a time, are completely blind to this. They would all say, "All clear!" because no single robot broke the rules.

The Solution: The "Verification-Aware Fabric"

To catch this, the authors created a system with three levels of watching:

Level 1 (The Individual): Each robot watches itself.
Level 2 (The Squad): A local leader watches a small group of robots.
Level 3 (The Mission): A central commander watches the whole team.

The magic happens in how they talk to each other. They use a special messaging system called the RV-Fabric. Think of this fabric not just as a phone line, but as a super-secure, unbreakable notebook that records every message sent, when it was sent, and who sent it. It has a special feature: it knows when a message is missing.

In the old way of doing things, if a message got lost because of radio jamming, the central computer might think, "Oh, no message came from Robot C, so Robot C must be doing nothing." It would then say, "Everything is fine!" This is a silent false all-clear, and it's dangerous.

The new system is different. It uses a two-axis verdict. Instead of just saying "Safe" or "Unsafe," it says:

Security: Is there a violation?
Completeness: Do we have enough evidence to be sure?

If a message is missing (because of jamming or a lost signal), the system doesn't guess. It says, "Unknown." It refuses to give an "All Clear" if it can't see the whole picture. It's like a detective who says, "I can't say the suspect is innocent just because I lost the alibi; I just don't know yet."

What They Found (The Results)

The authors tested this system using a computer simulation of a robot swarm mission. They set up a scenario where a bad guy used a prompt injection to trick the robots into splitting up the forbidden task.

The Blind Spot: The individual robot monitors saw nothing wrong. They all reported "No Violation."
The Catch: The new three-tier system saw the whole picture. It realized that when you combined the actions of all four robots, they had built the forbidden file. It flagged the violation and even named exactly which robots were involved (the provenance).
The Jamming Test: The authors then simulated radio jamming, where messages were dropped or delayed.
- A standard central monitor (the "best-effort" kind) got confused by the missing messages and gave a silent false all-clear three times. It thought the mission was safe when it wasn't.
- The new RV-Fabric system gave zero silent false all-clears. When messages were missing, it simply said, "Unknown" or "Incomplete." It never lied and said everything was fine.

They also tested this with different types of missions, like checking if the total radio emissions of the swarm were too high or if the robots were entering a restricted zone together. In every case, the new system caught the team violations that the individual monitors missed, and it never gave a false "all clear" when evidence was missing.

How Sure Are They?

The authors are very confident in these results, but it's important to note how they got them. They ran these tests in a simulation using a realistic robot simulator (ROS 2/Gazebo/ArduPilot). They didn't fly real robots in a war zone; they created a digital world where they could inject faults, drop messages, and replay the exact same mission over and over to see what happened.

In these simulations, the system worked perfectly. It detected 100% of the split-task attacks that were visible in the data, and it never gave a silent false all-clear, even when messages were lost. The system was also fast, checking the status of up to 1,000 robots in less than 2 milliseconds.

The paper argues that this approach is essential for the future of AI and robotics. As we let robots make more decisions on their own, we need a way to watch the team, not just the individual, and we need a system that admits when it doesn't know the answer, rather than guessing and potentially causing a disaster. The authors suggest that this "evidence-aware" approach is the key to making autonomous swarms trustworthy in dangerous, contested environments.

Technical Summary: Mission-Level Runtime Assurance for LLM-Assisted ISR Swarms

Problem Statement
The paper addresses a critical assurance gap in cooperative Intelligence, Surveillance, and Reconnaissance (ISR) swarms augmented by Large Language Models (LLMs). While individual platforms in a swarm may adhere to local safety and policy constraints, their collective actions can compose into a mission-level violation that no single platform detects. A specific vulnerability arises when an adversary uses indirect prompt injection (via retrieved mission data) to subvert LLM planners. The adversary can instruct the swarm to split a prohibited objective—such as assembling a complete intelligence profile of a protected facility—across multiple platforms. Each platform performs a permitted, locally compliant action (e.g., collecting coordinates, imagery, or communications metadata), but the union of these actions constitutes a policy violation.

Traditional per-platform guardrails fail to detect this because they lack a global view. Furthermore, in contested environments where communications are degraded or jammed, a naive central monitor may receive incomplete evidence. If the monitor treats missing evidence as a "clean" state, it risks issuing a silent false all-clear, falsely certifying the mission as compliant when a violation has occurred or is ongoing.

Methodology
The authors propose a three-tier compositional runtime-verification (RV) framework operating over a verification-aware fabric (RV-Fabric). The system is designed to decompose mission policies, aggregate verdicts, and handle evidence loss explicitly.

Three-Tier Monitoring Hierarchy:
- L1 (Platform): Local monitors on each agent check spatial residency, local budgets, and text-semantic predicates against the agent's own trace.
- L2 (Squad Coordinator): Aggregates traces from a subgroup of platforms to check partial properties (e.g., joint zone occupancy within a squad).
- L3 (Mission Monitor): Consumes the combined stream from all squads to evaluate cross-agent properties (e.g., prohibited collective intelligence packages, swarm-wide emission budgets, and causal ordering of actions).
Evidence-Aware Verdict Algebra:
The framework introduces a two-axis verdict system $(s, c)$ :
- Security ( $s$ ): $\{violation, no\_violation, unknown\}$ .
- Completeness ( $c$ ): $\{sound \succ degraded \succ incomplete \succ unavailable\}$ .
  Crucially, the system separates the question "Did a violation occur?" from "Do we have enough evidence to say?". If evidence is missing (due to loss or jamming), a "no_violation" verdict is downgraded to unknown rather than reported as a clean all-clear. This prevents the system from silently certifying a mission when the evidence stream is incomplete.
Verification-Aware Fabric (RV-Fabric):
The messaging layer is built on established brokers (MQTT for edge, NATS JetStream for backend) but augmented with specific guarantees to make evidence loss observable:
- At-least-once delivery with persistent event IDs to detect gaps.
- Monotone ordering to ensure causal consistency.
- Backend-resident mission clock to detect "silent" platforms (those that are live but not reporting) rather than mistaking them for idle.
- Consumer isolation and flow control to prevent slow monitors from blocking the system.
Implementation and Evaluation:
The framework was evaluated using a Modeling and Simulation (M&S) approach.
- Core: A deterministic Python generator simulating the event stream and fault injection.
- Realism Layer: Integration with ROS 2, Gazebo, and ArduPilot SITL (Software-In-The-Loop) to validate the event schema and monitor behavior in a realistic robotics environment.
- Attack Scenario: An indirect prompt injection caused a real LLM planner to split a prohibited collection task across four platforms (three UAVs, one UGV).
- Fault Campaign: The system was tested under fault-free conditions, dropped witness events, and platform jamming (silence).

Key Contributions

Compositional Runtime Verification: The paper lifts hierarchical monitoring from single-agent or edge-IoT contexts to cooperative robot swarms, enabling the detection of violations that only exist in the composition of agent behaviors.
Evidence-Aware Composition Algebra: A formal mechanism that fuses security verdicts with completeness status and provenance (identifying the specific platforms and events that triggered a violation). This ensures that unsupported negative verdicts are explicitly flagged as unknown.
RV-Fabric Architecture: A distributed messaging layer that makes evidence loss and silence observable, converting potential silent misses into explicit "unknown" states.
Simulation-Based Validation: A reproducible evaluation demonstrating that the framework detects distributed violations invisible to per-platform monitors and avoids silent false all-clears under contested communications.

Results

Detection of Distributed Violations: In the attack scenario, all four per-platform (L1) monitors reported no_violation because each action was locally permitted. The compositional (L3) monitor successfully detected the prohibited collective intelligence package, the EMCON (emission control) budget breach, and unauthorized zone entry, providing full platform provenance.
Robustness to Faults:
- Per-platform guardrails: Detected 0/3 mission incidents.
- Central best-effort monitor: Detected all 3 incidents in fault-free conditions but emitted 3 silent false all-clears under fault conditions (dropped events or jamming), incorrectly reporting the mission as compliant.
- RV-Fabric (Evidence-aware): Detected all 3 incidents in fault-free conditions. Under faults, it emitted 0 silent false all-clears. Instead, unresolved incidents were downgraded to unknown or incomplete, explicitly signaling the lack of evidence.
Scalability: The mission monitor evaluation time scales roughly linearly with swarm size, remaining under 2 ms for 1,000 platforms, well within second-scale mission decision budgets.
Generalization: The framework successfully handled four distinct property types (semantic composition, swarm-wide sum, cross-agent count, and causal chain) with zero silent false all-clears across a fault campaign.

Significance and Claims
The paper claims to provide a mission-assurance capability for autonomous systems in contested environments that fundamentally changes how evidence is treated. By distinguishing between "no violation" and "unknown due to missing evidence," the framework prevents the adversary from exploiting communication degradation to hide violations.

The authors emphasize that their contribution is not purely theoretical formal methods but an operational assurance layer that produces auditable, platform-attributed verdicts. The system allows a mission commander to know exactly which platforms jointly breached a policy and, critically, to know when the system cannot certify compliance due to evidence loss, rather than being misled by a silent "all-clear." The paper positions this as a necessary evolution from single-agent safety to multi-agent mission assurance, specifically addressing the unique failure modes introduced by LLM-assisted autonomy and adversarial prompt injection.