📊 statistics

Energy-Efficient Routing Protocol in Vehicular Opportunistic Networks: A Dynamic Cluster-based Routing Using Deep Reinforcement Learning

This paper proposes CR-DRL, a deep reinforcement learning-based dynamic cluster routing protocol for Vehicular Opportunistic Networks that significantly enhances energy efficiency, extends node lifetimes, and improves delivery reliability by optimizing relay selection and cluster overlap in highly dynamic environments.

Original authors: Meisam Sharifi Sani, Saeid Iranmanesh, Raad Raad, Faisel Tubbal

Published 2026-02-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Meisam Sharifi Sani, Saeid Iranmanesh, Raad Raad, Faisel Tubbal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a city where thousands of cars are driving around, but there are no traffic lights, no cell towers, and no internet cables connecting them. Instead, these cars talk to each other directly, like friends whispering secrets as they pass by on the street. This is what researchers call an Opportunistic Network.

The problem? Cars move fast. One second, Car A is next to Car B; the next second, they are miles apart. If Car A has a message for Car C, but Car B is in the middle, Car A has to "store" the message, "carry" it while driving, and "forward" it to Car C only when they happen to meet again. This is called Store-Carry-Forward.

But here's the catch: If every car tries to shout the message to everyone they pass, the network gets clogged (like a crowded party where everyone is talking at once). Also, cars have limited battery and memory. If a car gets too tired (low battery) or too full of messages (full memory), it can't help anymore.

The Solution: The "Smart Traffic Captain" (CR-DRL)

The authors of this paper propose a new system called CR-DRL. Think of it as a super-smart, self-learning traffic captain that organizes these chaotic cars into teams.

Here is how it works, broken down into simple concepts:

1. Forming Teams (Clustering)

Instead of every car talking to every other car, the system groups nearby cars into clusters (like small neighborhoods).

The Cluster Head: In every neighborhood, one car is chosen to be the "Team Captain" (Cluster Head). This captain is responsible for collecting messages from the other cars in the group and passing them along.
Who becomes the Captain? In the old days, they might just pick the first car they saw. But in this new system, the AI looks at three things before picking a captain:
1. Energy: Is the car's battery full? (We don't want to pick a car that's about to die).
2. Memory: Does the car have space in its trunk (buffer) to hold messages?
3. Social Skills: Has this car met many other cars recently? (A car that meets lots of people is better at passing messages along).

2. The "Brain" (Deep Reinforcement Learning)

How does the system know which car is the best captain? It uses Deep Reinforcement Learning (DRL).

The Analogy: Imagine a video game character learning to play. At first, it makes mistakes and loses points. But every time it makes a good move (like picking a captain with a full battery), it gets a "reward." Over time, the AI learns the perfect strategy to keep the network running smoothly without draining anyone's battery.
The Actor-Critic: The paper uses a specific type of AI called "Actor-Critic."
- The Actor is the decision-maker (it picks the captain).
- The Critic is the judge (it says, "Good job!" or "That was a bad choice, try again").
- They work together to get better and better at making decisions in real-time.

3. The "Flexible Fence" (Dynamic Overlap)

In the old methods, the size of these "neighborhoods" was fixed. If the city was empty, the neighborhoods were too small. If the city was jammed, the neighborhoods were too big and chaotic.

The Innovation: CR-DRL uses a "Smart Fence" (called a heuristic function).
- If the city is crowded, the fence shrinks so the teams stay small and efficient.
- If the city is empty, the fence expands so the teams can reach further and stay connected.
- This ensures that even if cars are far apart, the network stays connected, like a rubber band that stretches and shrinks perfectly.

4. The "Bridge Builders" (Common Members)

Sometimes a message needs to go from Neighborhood A to Neighborhood C, but they don't touch.

The system finds a car that belongs to both neighborhoods (a "Common Member"). This car acts as a bridge, carrying the message from one team to the other.
The AI makes sure these bridges are chosen carefully to avoid loops (cars running in circles) and to keep the path short.

Why Does This Matter? (The Results)

The researchers tested this system in a simulation of a busy city. Here is what they found compared to older methods:

Longer Life: The cars (nodes) lasted 21% longer because the AI didn't pick tired cars to do the heavy lifting.
Less Waste: The whole network used 17% less energy.
Faster Delivery: Messages arrived 28.5% faster because the AI found the best paths.
Fewer Stops: Messages needed 30% fewer "hops" (stops) to get to their destination.
Stability: The network stayed connected much better, even when traffic was crazy.

The Big Picture

Think of CR-DRL as a smart conductor for an orchestra of moving cars. Instead of everyone playing randomly and making noise, the conductor organizes them into sections, picks the best soloists (captains) based on who has the energy to play, and adjusts the music (network size) depending on how many musicians are on stage.

This allows cars to talk to each other efficiently, save their batteries, and get messages to their destination quickly, even in a chaotic, moving city. It's a huge step toward making our future smart cities run smoothly without needing a million cell towers.

1. Problem Statement

Vehicular Opportunistic Networks (OppNets) operate under the Store-Carry-Forward (SCF) paradigm, where data is stored on nodes and forwarded when connectivity is re-established. However, these networks face significant challenges due to:

Highly Dynamic Topology: Rapid vehicle mobility leads to intermittent connectivity and frequent network partitions.
Resource Constraints: Limited energy (battery) and buffer capacity on nodes.
Routing Inefficiencies: Traditional routing protocols struggle with unpredictable contact patterns, leading to high latency, low delivery ratios, and excessive energy consumption.
Limitations of Existing Methods:
- Clustering-based methods often rely on static thresholds or fixed infrastructure (e.g., Roadside Units), failing to adapt to rapid density changes.
- Reinforcement Learning (RL) methods often suffer from high computational overhead, poor convergence in dynamic environments, or workload imbalance (selecting the same nodes as cluster heads repeatedly).
TCP/IP Compatibility: OppNets often lack the stable connections required for standard TCP/IP protocols.

2. Methodology: CR-DRL

The authors propose CR-DRL (Cluster-based Routing using Deep Reinforcement Learning), a novel protocol that integrates an Actor-Critic (AC) deep learning framework with a heuristic function for dynamic cluster management.

A. System Model & Architecture

Network: A city-scale vehicular network where nodes (vehicles) move according to mixed mobility models (Random Waypoint, Gauss-Markov, Working Day Movement).
Clustering: Nodes are grouped into clusters. Each cluster has a Cluster Head (CH) acting as a gateway and Common Members acting as relays between clusters.
Virtual Segments: The protocol creates "virtual network segments" to mimic stable links, enabling TCP/IP compatibility without physical infrastructure.

B. Deep Reinforcement Learning (Actor-Critic Framework)

The routing and cluster head selection are modeled as a Markov Decision Process (MDP):

State Space ( $s$ ): Aggregated node states including:
- Maximum Encounter Rate ( $\phi_{v_{imax}}$ ): Historical frequency of contact with neighbors.
- Residual Energy ( $\Psi_{vi}$ ): Remaining battery relative to initial energy.
- Buffer Size ( $\eta_{vi}$ ): Current buffer occupancy relative to capacity.
Action Space ( $a$ ): Selection of the optimal Cluster Head.
Reward Function ( $R$ ): A weighted sum designed to maximize delivery ratio and throughput while minimizing End-to-End (E2E) delay:
$R = M_1(\hat{\mu}) + M_2(\hat{\eth}) - M_3(\hat{D})$
Where $\hat{\mu}$ is delivery ratio, $\hat{\eth}$ is throughput, and $\hat{D}$ is delay.
Stability Mechanisms: To prevent convergence issues common in DRL, the authors employ:
- Entropy Regularization: Balances exploration and exploitation.
- Experience Replay: Learns from past routing decisions to avoid overfitting to transient states.
- Adaptive Learning Rates & Gradient Clipping: Ensures stable training.

C. Adaptive Distance Threshold (ADT) Heuristic

To address the issue of varying cluster sizes in different densities, a heuristic function dynamically adjusts the maximum allowable distance between nodes to form clusters:

Formula: $\varrho_d = \Delta \cdot \frac{1}{\sqrt{H}}$ , where $H$ is node density and $\Delta$ is a scaling factor (optimized at 1.5).
Function:
- In low-density areas, the threshold increases, allowing clusters to span larger distances to maintain connectivity.
- In high-density areas, the threshold decreases, preventing clusters from becoming too large and reducing intra-cluster communication overhead.

D. Routing Process

Cluster Formation: Nodes calculate Manhattan distances and select CHs based on the AC algorithm's policy.
Data Transmission: Source sends data to its CH.
Inter-Cluster Forwarding: CHs forward data to Common Members (nodes shared between clusters) to reach the destination cluster.
Loop Prevention: Uses a Time-To-Live (TTL) of 300s and a hop count limit of 10. Messages track visited clusters to avoid loops.

3. Key Contributions

Novel AC-Based Routing: First protocol in OppNets to utilize a Deep Reinforcement Learning (Actor-Critic) model specifically for dynamic cluster head selection based on real-time energy, encounter rate, and buffer metrics.
Dynamic Overlap Adjustment: Introduction of the ADT heuristic to dynamically adjust cluster overlap based on node density, solving the scalability issues of fixed-distance clustering.
TCP/IP Compatibility: A middleware architecture that transforms dynamic clusters into virtual network segments, allowing standard TCP/IP protocols to function in intermittent OppNets without Roadside Units (RSUs).
Balanced Resource Utilization: The AC algorithm prevents workload imbalance by dynamically evaluating node capabilities, ensuring no single node is overused as a cluster head.

4. Simulation Results

The protocol was evaluated using the ONE (Opportunistic Network Environment) simulator with a Helsinki map scenario (50–250 nodes). It was compared against baselines like EEHCHR, SCF-Architecture, Mobility Adaptive, RLProph, and Latency-Aware routing.

Key Performance Improvements:

Energy Efficiency:
- Extends node lifetime by up to 21%.
- Reduces overall energy consumption by 17%.
- Keeps nodes active 15% longer than benchmarks.
Communication Performance:
- Delivery Ratio: Up to 10% higher than baselines.
- Latency: 28.5% lower E2E delay.
- Throughput: 7% higher data throughput.
- Hop Count: 30% fewer transmission steps (hops) required.
Stability:
- Reduces network partitioning by 85% compared to SCF-Architecture.
- Achieves near-complete connectivity (1 partition) within 35 seconds, whereas SCF remains fragmented.
- Significantly lowers the Cluster Head Change Rate, indicating more stable clusters.

5. Significance

The CR-DRL protocol represents a significant advancement in Vehicular Ad-hoc Networks (VANETs) and Smart City infrastructure. By successfully integrating Deep Reinforcement Learning with heuristic clustering, it solves the critical trade-off between adaptability (handling dynamic topology) and efficiency (energy and latency).

Its ability to create virtual stable links makes it a viable candidate for deploying standard internet protocols (TCP/IP) in highly mobile, infrastructure-less environments. This paves the way for more reliable Intelligent Transport Systems (ITS), real-time traffic management, and robust data exchange in future 6G-enabled edge computing environments.