Deep Reinforcement Learning-Based Dynamic Mode Selection and Power allocation for Clustered Vehicular Networks

Imagine a world where your car doesn't just drive itself but also talks to every other car, traffic light, and street sign around it. This is the future of "Vehicular Networks," a high-speed digital conversation happening on our roads to prevent crashes and manage traffic. But here's the tricky part: these cars are moving fast, the roads are crowded, and the "airwaves" they use to talk are like a busy highway with only a few lanes. If everyone shouts at once, no one hears anything. To solve this, engineers use "clustering," which is like grouping cars into small teams led by a "team captain" (the Cluster Head) to organize the conversation. The big question researchers are trying to answer is: How does this captain decide the best way to talk? Should the team shout directly to each other (a short-range "sidelink" or D2D mode), or should they call a central tower (a cellular or C-V2X mode) to relay the message? And how loud should they shout? Shout too loud, and you drown out neighbors; shout too soft, and your message gets lost.

This paper dives into that exact problem, asking how these car captains can make smart, split-second decisions to keep the network reliable without wasting energy. The authors propose a solution using "Deep Reinforcement Learning," which is essentially a computer program that learns by trial and error, much like a video game character getting better at a level by playing it thousands of times. Instead of following a rigid rulebook, this AI learns to adapt to the chaos of traffic, interference, and changing distances. The study suggests that by letting these AI captains learn from their environment, they can make better choices than traditional, rule-based methods, ensuring safety messages get through quickly and reliably while saving battery power.

The Smart Team Captains of the Road

In the study, the researchers set up a virtual highway simulation to test their idea. They imagined a stretch of road 2 kilometers long with cars zooming at speeds up to 31 meters per second (about 70 mph). In this digital world, the cars aren't just driving; they are constantly forming and breaking up into groups, or "clusters," based on who is close to whom. Each cluster has a leader, the "Cluster Head," whose job is to decide how the group communicates.

The researchers taught these leaders a new way to think using a specific type of AI called Proximal Policy Optimization (PPO). Think of PPO as a very disciplined student who learns by trying different strategies and getting feedback. In this case, the "student" is the Cluster Head. Every time it makes a decision, it gets a score (a "reward"). If it picks the right communication mode and power level to get a message through quickly and reliably, it gets a high score. If it causes a delay or wastes power, it gets a low score. Over time, the AI learns the perfect balance, adapting to the fact that cars are always moving and the "noise" from other vehicles is constantly changing.

The Great Communication Showdown

To see if this AI learning method actually works, the researchers pitted their PPO "student" against five other strategies, or "baselines," that represent how we might solve the problem without a learning brain:

1. The Distance Guessers: These strategies only look at how far apart the cars are. If the average distance is under 400 meters, they shout directly to each other; otherwise, they call the tower. They don't care about the actual quality of the connection or interference.
2. The Power Schedulers: These are similar to the distance guessers but try to be a bit smarter about how loudly they shout based on distance.
3. The "Always Call the Tower" Team: This strategy ignores direct communication entirely and always routes messages through the cellular network.
4. The Random Chatter: This strategy picks a communication mode and power level completely at random, just to see what happens.

The results of this digital race were quite revealing. The PPO AI didn't just win; it showed a completely different way of playing the game.

Reliability: The Safety Net
The most important job of these networks is safety. The paper measured "reliability," which is the percentage of messages that successfully arrive. The PPO AI was a champion here. In simulations with 25 cars, it achieved 100.0% reliability. Even with 100 cars on the road, it stayed incredibly high at 99.750%.
In contrast, the "Distance Guessers" (the non-learning methods) struggled. With 50 cars, their reliability dropped to 87.297%, and with 100 cars, it was around 94.971%. The "Always Call the Tower" strategy did even worse, dropping to 67.469% with 100 cars. The random strategy was the worst of all, failing to deliver messages consistently. The paper suggests that the AI learned to avoid the "dead zones" where direct shouting fails and the "congestion" where calling the tower is too slow, finding a sweet spot that the simple rules missed.

Power: The Battery Saver
Here is where the story gets really interesting. Usually, to get better reliability, you have to shout louder (use more power). The "Distance Guessers" and "Always Call the Tower" strategies shouted at a fixed, high volume of 18 dBm (a specific unit of power) to try and force their messages through.
The PPO AI, however, learned to whisper. It achieved its near-perfect reliability while using significantly less power. For 25 cars, it used an average of 10.013 dBm. For 100 cars, it dropped to 9.612 dBm. That is a massive saving—up to 8.39 dB less power than the fixed-power baselines. The paper shows that the AI didn't just get lucky; it learned that shouting louder isn't always the answer. By choosing the right mode at the right time, it saved energy without sacrificing safety.

Speed and Efficiency: The Trade-off
The paper also looked at "latency" (how long it takes for a message to arrive) and "spectral efficiency" (how much data can be squeezed into the airwaves). Here, the simple rules had a slight edge. The distance-based strategies were sometimes a tiny bit faster (around 7.45 ms vs. the AI's 8.702 ms at 100 cars) and squeezed a bit more data through.
However, the authors point out that this speed came at a terrible cost: much lower reliability and much higher power usage. The simple rules were like a sprinter who runs fast but trips over their own feet, while the AI was a marathon runner who kept a steady, reliable pace. The paper concludes that for safety-critical applications, the AI's ability to guarantee that the message arrives (reliability) while saving power is far more valuable than shaving off a fraction of a millisecond on speed.

What This Means for the Road Ahead

The study suggests that the future of smart traffic isn't about following rigid rules like "if you are close, shout; if you are far, call." Instead, it's about giving the network a brain that can feel the environment. The PPO AI learned to handle the messy, real-world chaos of moving cars and interference in a way that simple math formulas couldn't.

While the results are based on computer simulations and not real-world driving tests yet, the findings are strong. The paper demonstrates that learning-based adaptive policies can outperform traditional, distance-only methods. It proves that by teaching the network to balance reliability, speed, and power, we can build a safer, more efficient future for our roads. The authors note that while their method is a big step forward, future work will need to test these ideas in even more complex scenarios, like multi-city networks and with imperfect information, to ensure they are ready for the real world. But for now, the "smart team captain" approach looks like a winning strategy for keeping our connected cars talking safely and efficiently.