⚡ electrical engineering

UAVGENT: A Language-Guided Distributed Control Framework

The paper proposes UAVGENT, a three-layer framework that integrates natural language instructions interpreted by an LLM supervisor with a distributed inner-loop controller to enable multi-drone systems to execute complex, evolving missions while maintaining formal robustness and stability guarantees.

Original authors: Ziyi Zhang, Xiyu Deng, Guannan Qu, Yorie Nakahira

Published 2026-02-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ziyi Zhang, Xiyu Deng, Guannan Qu, Yorie Nakahira

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the director of a movie, but instead of actors, you are commanding a swarm of 24 drones. Your goal is to have them chase three getaway cars, form a perfect grid around them, and then switch to a circle formation if the cars split up.

In the past, giving these instructions was like trying to conduct an orchestra where every musician is deaf to the conductor and only listens to their neighbor. If the cars changed direction, the director (you) would have to frantically shout new coordinates to every single drone, or the whole formation would collapse into chaos.

UAVGENT is a new system that solves this by introducing a "Smart Assistant" (an AI) between you and the drones. Here is how it works, broken down into three layers using simple analogies:

1. The Director (You) – The Outer Layer

You don't need to know the math or the coordinates. You just speak naturally.

What you say: "Chase those three cars in a grid formation."
What happens: You give a high-level command. You don't micromanage the drones. You are the "visionary."

2. The Smart Assistant (The LLM Supervisor) – The Middle Layer

This is the magic part. In the middle of the action, an AI (Large Language Model) acts as your Chief of Staff.

The Problem: The cars suddenly speed up and split in three different directions. The original "grid" plan is now impossible because the cars are too far apart.
The Old Way: The drones would keep trying to hold the grid, crash into each other, or lose the cars.
The UAVGENT Way: The Smart Assistant notices the chaos. It thinks, "Wait, the cars split. The user wants to track them, but a single grid won't work anymore."
The Fix: The Assistant automatically rewrites the plan. It says, "Okay, split the drones into three smaller groups. Group A follows Car 1, Group B follows Car 2, Group C follows Car 3. Keep them in small grids."
The Result: You don't have to say a word. The system adapts to the changing world instantly, ensuring the drones stay on task without you needing to be a math genius.

3. The Drones (The Inner Layer) – The Distributed Control

The drones themselves are like a school of fish or a flock of birds. They don't talk to a central computer; they only talk to the drones and cars right next to them.

How they move: Each drone looks at its neighbors and says, "I need to stay 2 meters away from you and 2 meters away from that car."
Why it's safe: Even if the "Smart Assistant" makes a slight mistake or the wind blows them off course, the drones have a built-in "safety net" (mathematical guarantees) that keeps them from crashing or losing the formation. They self-correct locally, like a flock of birds that instantly adjusts if one bird gets pushed.

The "Police Chase" Scenario

The paper tested this with a simulation of a police chase:

The Setup: 24 drones are told to surround three cars in a square grid.
The Chaos: The cars speed up and drive in three different directions.
The Glitch (Old Systems): The drones would get confused, the grid would stretch and break, and they might lose the cars.
The UAVGENT Solution:
- The Inner Layer (drones) quickly split into three groups to follow the cars.
- The Middle Layer (AI) noticed the groups were messy and uneven. It stepped in and said, "Rebalance! Give 8 drones to Car 1, 8 to Car 2, 8 to Car 3. Fix the grid shapes."
- The drones instantly reorganized into perfect, smaller grids around each car.
- Finally, the director said, "Make one group a circle, one a square, and one a cross." The AI translated that instantly, and the drones morphed into those shapes while still chasing the cars.

Why This Matters

Think of it like a GPS vs. a Human Navigator.

Old Systems: You have to manually input every turn into the GPS. If traffic changes, you have to recalculate the whole route.
UAVGENT: You tell the GPS, "Get me to the airport." If there is a traffic jam, the GPS automatically reroutes you without you asking. It understands the intent (get to the airport) and handles the details (avoiding traffic) on its own.

In short: UAVGENT lets humans give simple, high-level goals in plain English, while an AI "Smart Assistant" handles the complex, split-second adjustments, and the drones use simple local rules to stay safe and stable. It combines the best of human creativity with the reliability of math.

1. Problem Statement

The paper addresses the challenge of controlling multi-drone systems (swarms) in dynamic environments using natural language. While Large Language Models (LLMs) allow non-experts to issue high-level commands (e.g., "form a circle and track the red car"), existing approaches suffer from two critical limitations:

Brittleness in Dynamic Environments: Most "language-to-robot" pipelines are "compile-once," meaning a prompt is translated into a plan and executed until failure. They lack the ability to continuously adapt to changing targets or environmental disturbances.
Lack of Formal Guarantees: Integrating LLMs often sacrifices the mathematical stability and robustness guarantees provided by traditional control theory. Small errors in language interpretation can cascade into large collective failures in multi-agent coordination.

The core problem is how to combine centralized, flexible language-based reasoning with distributed, robust feedback control to achieve complex behaviors with provable stability.

2. Methodology: The UAVGENT Framework

The authors propose UAVGENT, a three-layer hierarchical architecture that operates on separated time scales:

A. Outer Layer: Human Instruction (Sporadic)

Input: A human operator issues natural language instructions (e.g., formation type, target selection, search regions).
Translation: An LLM translates these instructions into:
- A nominal drone reference configuration ( $p_{cmd}$ ).
- An intent summary (e.g., stationary, track, or search).
Reference Types:
- Stationary: Fixed world-frame coordinates.
- Track: Relative to moving targets.
- Search: Within a bounded region until a target is detected.

B. Middle Layer: LLM Supervisor (Periodic)

Function: Acts as an "assistant" to the human, performing periodic verification and correction at intervals $\{t_k\}$ .
Process:
1. Receives the current state and the latest intent.
2. Auto-Verification: Checks if the current execution matches the human's intent.
3. Correction: If inconsistencies are detected (e.g., targets split, formation degrades), the LLM updates the enforced reference ( $p^r_a$ ) to re-align the swarm with the intent.
4. Re-grounding: For moving targets, it re-associates drones to targets if the original assignment becomes invalid due to target divergence.
Key Feature: It handles major environmental changes without requiring constant human intervention, only intervening when the system drifts from the intent.

C. Inner Layer: Distributed Controller (Continuous)

Mechanism: A distributed edge-error feedback controller runs continuously between LLM updates.
Interaction Graph: The neighbor graph is radius-induced. Two agents (drone-to-drone or drone-to-target) are neighbors if they are within a sensing radius $r$ . This graph is dynamic and topology-free (no centralized connectivity enforcement).
Control Law: Each drone $i$ computes control input $u_i$ based on local relative errors:
$u_i(t) = \sum_{j \in N_i(t)} \left( (p_j - p_i) - (p^r_j - p^r_i) \right)$
Where $p^r$ is the enforced reference. Targets act as moving anchors ( $p^r_{target} = p_{target}$ ).
Robustness: This layer ensures stability and tracks the reference generated by the LLM using only local information.

3. Key Contributions

Hierarchical Architecture: A novel framework coupling centralized LLM reasoning with distributed control. The LLM handles high-level intent and adaptation, while the distributed controller ensures low-level stability.
Theoretical Guarantees:
- The authors derive a theoretical guarantee characterizing tracking performance under bounded disturbances and piecewise-smooth references.
- They prove Exponential Input-to-State Stability (ISS) for the edge-formation error.
- They provide an upper bound on the controller error between LLM verification steps, showing that the system converges even with discrete jumps in the reference caused by LLM updates.
Dynamic Adaptation: The system handles complex scenarios like target splitting, merging, and formation switching (e.g., from grid to circle) automatically via the LLM supervisor, without human micromanagement.
Radius-Induced Topology: The control relies purely on geometric proximity, making it robust to communication dropouts and scalable to large swarms without a fixed communication graph.

4. Results and Simulations

The framework was evaluated in two primary scenarios:

Police-Chasing Scenario:
- Setup: 24 drones tracking 3 suspect vehicles.
- Events: Suspects split into different directions; the swarm had to split into sub-swarms. Later, the user requested different formations (circle, square, cross) for different groups.
- Outcome: The inner layer distributed control initially distributed drones to targets but allowed formation distortion. The LLM supervisor detected the distortion, re-balanced the drone-to-target assignments, and regenerated clean formation references. The system successfully transitioned between complex formations while maintaining tracking.
Search-and-Rescue Scenario:
- Setup: Drones searching a forest for a missing person within a bounded region.
- Events: Random waypoint assignment, event-driven re-tasking upon negative observations, and encirclement upon detection.
- Outcome: The LLM efficiently managed the search logic (assigning waypoints, expanding search regions if needed). Upon detection, the swarm seamlessly transitioned to a 3D encirclement formation to mark the location for ground responders.

Performance Metrics: The simulations demonstrated low time-to-detection, efficient waypoint utilization, and high formation quality (low radius error) during the encirclement phase.

5. Significance and Conclusion

UAVGENT represents a significant step toward deploying autonomous multi-robot systems in real-world, unstructured environments.

Bridging the Gap: It successfully bridges the gap between the flexibility of LLMs and the rigor of control theory. It moves beyond "compile-once" LLM pipelines to a "language-in-the-loop" system that continuously verifies and corrects execution.
Human-in-the-Loop Efficiency: It drastically reduces the cognitive load on human operators. Humans only need to issue high-level commands when major changes occur; the LLM supervisor handles the adaptation and minor corrections.
Provable Safety: By retaining a distributed inner-loop controller with formal stability guarantees, the system ensures that even if the LLM makes a suboptimal correction, the physical system remains stable and bounded.

The authors conclude that this approach enables complex, adaptive multi-agent behaviors with provable robustness, making it a viable candidate for critical applications like emergency response, surveillance, and environmental monitoring. Future work involves integrating multimodal (visual-language) models to further enhance real-world deployment capabilities.