⚡ electrical engineering

DARA: Distributed Agent Reinforcement-based Adaptive routing in Flying Ad-hoc Network Environments

This paper proposes DARA, a fully decentralized routing framework for Flying Ad-hoc Networks that utilizes distributed reinforcement learning to enable UAVs to autonomously select optimal next-hop relays based on local information, thereby significantly improving packet delivery, reducing latency, and balancing energy consumption compared to conventional routing protocols.

Original authors: Badia Bouhdid, Adel Ben Mnaouer, Adnan Ahmed Abi Sen, Bassem Ouni

Published 2026-06-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Badia Bouhdid, Adel Ben Mnaouer, Adnan Ahmed Abi Sen, Bassem Ouni

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a swarm of drones flying over a disaster zone, trying to pass critical messages (like "fire here!" or "injured person there!") to a command center on the ground. This is a Flying Ad-hoc Network (FANET).

The problem is that these drones are moving fast in 3D space, the wind changes, and the airwaves get noisy. It's like trying to play a game of "telephone" while running through a crowded, shifting maze. If one drone drops out of range, the message gets lost.

Traditional ways of organizing these drones (like having a "team captain" or using old-school road-map rules) often fail because the team changes too fast, or the captain gets overwhelmed.

Enter DARA: A new, smarter way for these drones to talk to each other.

The Core Idea: The "Smart Individual" vs. The "Strict Manager"

Think of the old methods like a military unit with a strict commander.

The Commander (Centralized/Clustering): One drone (or a ground station) tells everyone else where to go. If the commander gets cut off or the team moves too fast, the whole unit freezes or gets lost.
The Old Rules (MANET): Drones shout out, "Who can hear me? Who can pass the message?" This creates a lot of noise (traffic) and wastes battery power.

DARA is different. It's like a swarm of independent, super-smart bees.

No Boss: There is no central commander. Every single drone is its own boss.
Learning by Doing: Each drone has a tiny "brain" (an AI agent) that learns from experience, just like a dog learning to fetch. It tries different paths, sees what works, and remembers the best ones.

How DARA Works: The "Three-Step Dance"

The paper describes DARA as a three-step process that happens constantly:

1. The "Hello" Wave (Beacon Emission)
Every few seconds, each drone shouts out a quick "Hello!" to its neighbors. This message says:

"I am here."
"I am moving this fast."
"I have this much battery left."
"I am this far from the ground station."
Analogy: It's like a group of hikers in a fog constantly tapping their sticks and saying, "I'm here, I'm tired, I'm heading north."

2. The "Brain" Decision (Reinforcement Learning)
When a drone needs to pass a message, it doesn't ask a boss. It looks at its "neighbors" (the drones it just heard from) and asks its internal AI:

"Who has the best signal?"
"Who is moving toward the goal?"
"Who isn't too busy or out of battery?"
The Magic: The AI uses a system called Q-Learning. Imagine a video game where the drone gets a "gold star" (reward) if the message gets through quickly and safely, and a "red X" (penalty) if it gets stuck or lost. Over time, the drone learns exactly which neighbor to pick to get the most gold stars.

3. The Handoff (Relay)
The drone picks the best neighbor and passes the message. That neighbor then becomes the new "sender" and repeats the process.

Analogy: It's like a relay race where every runner decides on the fly who to pass the baton to, based on who is running the fastest and is closest to the finish line, without waiting for a coach's instruction.

Why is this better? (The Results)

The authors tested DARA in a computer simulation (using a tool called ns-3) against three other popular methods:

AODV: The old "shout and wait" method.
EMASS: A method that tries to group drones into teams with a captain.
ESOFCluster: A fuzzy-logic method that also tries to group drones.

They tested these in three scenarios:

Scenario 1: Calm weather, moderate speed.
Scenario 2: Chaotic weather, fast speeds, and sudden bursts of messages.
Scenario 3: A crowded swarm (50 drones) with high interference.

The Findings:

More Messages Delivered: DARA got more messages to the ground station than the others (up to 25% better in some cases).
Faster Delivery: The messages arrived much quicker (up to 45% faster) because DARA didn't waste time shouting for routes or waiting for a captain to decide.
Less Battery Wasted: Because DARA didn't need to send constant "status reports" to a central boss or re-organize teams constantly, the drones saved a lot of energy (up to 45% less energy used for routing).
Less Noise: The network was quieter. DARA didn't flood the air with unnecessary control messages.

The Bottom Line

The paper claims that DARA solves the problem of "too much movement" by letting every drone make its own smart decisions based on what it sees right now.

Instead of relying on a fragile chain of command or a noisy shouting match, DARA turns the swarm into a team of self-learning individuals. By letting each drone learn from its own successes and failures, the whole group becomes more reliable, faster, and longer-lasting, even when the environment is chaotic and changing rapidly.

What the paper does NOT claim:

It does not claim this is currently flying on real drones in a real disaster (it was a computer simulation).
It does not claim to solve security issues (like hackers) or adversarial attacks.
It does not claim to work for medical uses or specific clinical applications.

It simply proves that this specific "decentralized learning" approach works better than the current standard methods in a simulated, high-speed drone environment.

Technical Summary: DARA – Distributed Agent Reinforcement-based Adaptive Routing in Flying Ad-hoc Network Environments

Problem Statement
Flying Ad-hoc Networks (FANETs), composed of Unmanned Aerial Vehicles (UAVs), face unique routing challenges distinct from traditional Mobile Ad-hoc Networks (MANETs). These include highly dynamic three-dimensional topologies, rapid link disruptions due to high mobility, severe energy constraints, and the lack of persistent infrastructure. Conventional MANET protocols (e.g., AODV, OLSR) degrade significantly in these environments due to frequent route breakages and high overhead. Similarly, existing FANET-specific solutions often rely on hierarchical clustering or centralized Software-Defined Networking (SDN) architectures. The paper argues that clustering approaches suffer from high re-clustering costs and control overhead during rapid topology changes, while centralized SDN approaches introduce single points of failure and latency due to reliance on global state updates. Furthermore, many existing Reinforcement Learning (RL) approaches for FANETs remain semi-centralized or embedded within clustering frameworks, failing to achieve true decentralization.

Methodology
To address these limitations, the authors propose DARA (Distributed Agent Reinforcement-based Adaptive Routing), a fully decentralized routing framework. In DARA, every UAV operates as an independent, autonomous learning agent using Q-learning. The system does not rely on cluster heads or a central controller.

Network Model: The architecture consists of Low Altitude Platform (LAP) UAVs responsible for data acquisition and High Altitude Platform (HAP) UAVs acting as aerial relays to a Disaster Management Station (DMS).
State Representation: Each agent constructs a local state vector comprising residual energy, link quality (a weighted combination of RSSI, Packet Reception Ratio, and Link Expiration Time), distance to the nearest HAP, relative velocity of neighbors, and neighbor buffer occupancy.
Action Space: The action is the selection of a next-hop neighbor from a candidate set filtered by link quality thresholds and forward progress toward the HAP.
Reward Function: The agent receives a scalar reward designed to balance four objectives: packet delivery reliability (PDR), link quality, energy consumption, and congestion avoidance (buffer occupancy). The reward function penalizes energy-intensive transmissions and forwarding to congested nodes.
Operational Workflow: The protocol operates in three stages:
1. Beacon Emission: Periodic broadcasting of local status (position, energy, velocity) to maintain neighbor tables.
2. Control Epoch: An RL-driven decision cycle where the agent selects a next hop based on its Q-table (using an $\epsilon$ -greedy policy), transmits the packet, observes the outcome (ACK/timeout), and updates the Q-value.
3. Relay Behavior: Receiving nodes check for duplicates, attempt direct HAP transmission if in range, or forward the packet to the next hop selected by their own local RL agent.

Key Contributions
The paper outlines four primary contributions:

Decentralized Framework: The introduction of a fully distributed routing framework where each UAV acts as an independent RL agent, eliminating the bottlenecks associated with clustering structures and centralized coordinators.
Multi-Objective Reward Design: The development of a compact state representation and a multi-goal reward function that jointly optimizes link reliability, routing stability, congestion avoidance, and energy usage.
Lightweight Algorithms: The design of practical algorithms for beacon exchange, RL-driven next-hop selection, and relay behavior, forming a three-phase workflow executable efficiently on resource-constrained UAVs.
Comprehensive Evaluation: Extensive simulations using ns-3 coupled with ns3-gym to validate DARA against established baselines (AODV, EMASS, and ESOFCluster) across varying mobility models and traffic patterns.

Results
Simulations conducted over a 2km $\times$ 2km area with 30 to 50 LAPs and varying mobility speeds (10–25 m/s) demonstrate that DARA consistently outperforms the baseline protocols:

Packet Delivery Ratio (PDR): DARA achieves a 10–25% improvement over baselines, with the most significant gains observed in high-mobility and high-density scenarios where traditional protocols suffer from link instability.
End-to-End Delay: DARA reduces delay by 20–45% compared to AODV and clustering-based schemes, attributed to the avoidance of route discovery flooding and cluster maintenance overhead.
Throughput: The protocol sustains 18–40% higher throughput by efficiently utilizing stable links and avoiding congestion hotspots.
Routing Overhead: DARA reduces control overhead by 45–65% compared to baselines, as it eliminates route discovery floods and cluster re-election signaling.
Energy Efficiency: Routing-related energy consumption is reduced by 25–45%, primarily due to the elimination of excessive control signaling and the selection of energy-efficient paths.
Link Stability: DARA maintains significantly longer link holding times, indicating superior adaptability to 3D mobility.

Significance and Claims
The paper claims that DARA validates the potential of decentralized, reinforcement learning-driven routing as a scalable and robust solution for next-generation autonomous UAV networks. By pushing intelligence to the network edge and relying solely on local interactions, DARA addresses the critical limitations of centralized and hierarchical approaches, specifically control-plane bottlenecks and single points of failure. The authors assert that the "free-clustering" design is particularly advantageous in highly dynamic environments where minimizing control complexity directly improves network efficiency.

The paper concludes modestly, acknowledging limitations such as the scalability of tabular Q-learning in extremely large state spaces and the current assumption of fully cooperative behavior. Future work is proposed to integrate deep reinforcement learning for better generalization and to incorporate security mechanisms for adversarial environments.

The Core Idea: The "Smart Individual" vs. The "Strict Manager"

How DARA Works: The "Three-Step Dance"

Why is this better? (The Results)

The Bottom Line

More like this