💻 computer science

PALCAS: A Priority-Aware Intelligent Lane Change Advisory System for Autonomous Vehicles using Federated Reinforcement Learning

This paper introduces PALCAS, a priority-aware intelligent lane change advisory system for autonomous vehicles that utilizes multi-agent federated reinforcement learning with a novel reward function to optimize safety, efficiency, and destination urgency in both mandatory and discretionary lane-changing scenarios.

Original authors: Yassine Ibork, Nhat Ha Nguyen, Myounggyu Won, Lokesh Das

Published 2026-05-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yassine Ibork, Nhat Ha Nguyen, Myounggyu Won, Lokesh Das

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a busy highway as a giant, chaotic dance floor. Everyone is trying to move forward, but some dancers need to exit the room soon, while others are just cruising. If everyone tries to cut in line at the last second, or if no one knows what the person next to them is planning, the result is a traffic jam or, worse, a collision.

This paper introduces PALCAS, a smart system designed to teach autonomous cars (self-driving vehicles) how to dance together safely and efficiently. Here is how it works, broken down into simple concepts:

1. The Problem: Too Many Solo Dancers

Currently, most self-driving car systems act like solo dancers. They only look at the cars immediately around them.

The Issue: If a car needs to exit the highway in 500 meters, a "solo" system might wait too long to move to the right lane, causing a sudden, dangerous swerve. Conversely, a car going straight might move to the right lane too early, blocking the exit for others.
The Risk: This lack of coordination leads to accidents, traffic jams, and missed exits.

2. The Solution: A "Federated" Dance Team

PALCAS solves this by using a method called Federated Reinforcement Learning. Think of this like a team of local dance instructors (called RSUs or Roadside Units) who manage small sections of the highway.

Local Training: Each instructor teaches the cars in their specific zone how to dance based on what they see locally.
Sharing Secrets (Without Sharing Privacy): Instead of sending all the video footage of the cars to a central boss (which would be slow and a privacy risk), the instructors only send the lessons learned (the math behind the decisions) to a central server.
The Global Lesson: The central server mixes these lessons together to create one "Master Dance Manual" and sends it back to all the instructors.
The Result: Every instructor gets smarter over time. They learn from traffic patterns in other parts of the highway without ever seeing the private data of other drivers.

3. The "Priority" Rule: Who Goes First?

The most unique part of PALCAS is its Priority-Aware system. It doesn't treat every car the same; it looks at the destination.

The "Urgent Exit" Car: If a car is close to its exit, the system gives it a high priority. It gently nudges the car to move to the rightmost lane early, so it can exit smoothly without cutting anyone off.
The "Long Haul" Car: If a car is going far down the highway, the system tells it to stay in the fast lanes (the left lanes) and not bother moving to the right yet.
The Analogy: Imagine a concert where people are leaving early. PALCAS is the usher who tells the people leaving early to move to the aisle now, while telling the people staying for the encore to stay in their seats. This prevents a stampede at the door.

4. The "Reward" System: How They Learn

The cars learn through a system of rewards and penalties, similar to training a pet or playing a video game:

Efficiency Reward: You get points for keeping traffic moving fast.
Safety Reward: You get points for staying a safe distance from other cars (using a strict "safety bubble" rule).
Comfort Reward: You get points for smooth driving. No jerky braking or sudden acceleration.
Priority Reward: This is the special sauce. You get a huge reward for moving to the exit lane at the perfect time. If you wait too long, you get a penalty. If you move too early and block the exit lane, you also get a penalty.
Deadlock Penalty: If a car gets stuck on an on-ramp (the road merging onto the highway), it gets a penalty to encourage it to merge quickly.

5. The Results: A Smoother Ride

The researchers tested this system in a computer simulation of a busy highway with many on-ramps and off-ramps. They compared PALCAS against two other methods:

Centralized: One giant brain controlling all cars (slow and risky if the connection breaks).
Isolated: Each section of the highway learning alone (no teamwork).

PALCAS won on almost every metric:

Faster Traffic: Cars moved about 7% faster on average compared to the isolated method.
Safer: It reduced collisions significantly (by nearly 76% compared to the isolated method).
More Comfortable: The cars accelerated and braked much more smoothly, like a human driving carefully rather than a robot jerking the wheel.
Better Exits: Cars were much more likely to successfully reach their exit without getting stuck or missing it.

Summary

PALCAS is like a smart traffic conductor that uses a team of local coaches to teach self-driving cars how to coordinate. By sharing "lessons learned" instead of private data, and by prioritizing cars based on where they need to go, it creates a highway where cars flow like water rather than crashing like bumper cars. It makes the road safer, faster, and more comfortable for everyone.

1. Problem Statement

Lane-changing on urban highways, particularly those with frequent on- and off-ramps, is a complex decision-making task involving the coordination of lateral and longitudinal vehicle controls. Current approaches face several challenges:

Safety and Efficiency: Improper lane changes contribute to over 10% of highway accidents and 10% of traffic congestion. Human drivers often fail to make critical decisions due to perception errors.
Limitations of Existing AI:
- Single-Agent Systems: Often focus on the ego vehicle's efficiency, neglecting global traffic coordination.
- Centralized Multi-Agent Systems (MARL): While they improve coordination, they suffer from data privacy risks, high computational overhead, and a lack of scalability in large, heterogeneous networks.
- Federated Learning (FL) Gaps: Existing FL applications in autonomous driving primarily focus on low-level motion control (e.g., collision avoidance) rather than integrated, high-level multi-agent lane-change decision-making that balances mandatory and discretionary maneuvers.

2. Methodology: PALCAS Framework

The authors propose PALCAS, a Priority-Aware Lane-Change Advisory System built on a Federated Multi-Agent Reinforcement Learning (Fed-MARL) framework.

System Architecture

Infrastructure: The highway is segmented into clusters, each managed by a Roadside Unit (RSU). RSUs act as agents coordinating Connected and Automated Vehicles (CAVs) within their coverage.
Communication:
- V2X (Vehicle-to-Everything): CAVs communicate with their local RSU.
- I2I (Infrastructure-to-Infrastructure): RSUs communicate with each other to share global traffic statistics.
- Federated Learning: RSUs train local models using local data and periodically upload parameters to a central server for aggregation (FedAvg), then download the updated global model. This preserves data privacy while enabling knowledge sharing.

Mathematical Formulation

Problem Type: The system is formulated as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP).
Algorithm: The system utilizes Parameterized Deep Q-Networks (PDQN) to handle a hybrid action space:
- Discrete Actions: Lane change left, lane change right, or maintain lane.
- Continuous Actions: Acceleration or deceleration commands.
State Space: A hierarchical representation fusing:
- Microscopic: Ego-vehicle state (position, velocity, acceleration) and surrounding vehicle states.
- Macroscopic: Cluster-level traffic density and average speed.
- Global: Aggregate traffic statistics shared via I2I communication.

Reward Function Design

The core innovation is a priority-guided reward function ( $R_k$ ) designed to balance safety, efficiency, comfort, and destination urgency:
$R(t)_k = \sum (\iota r_e + \zeta r_s + \xi r_c + \lambda r_{lc} + \psi r_d)$

Efficiency Reward ( $r_e$ ): Encourages maintaining speeds close to the maximum limit and optimizing cluster-level flow.
Safety Reward ( $r_s$ ): Based on the Responsibility-Sensitive Safety (RSS) model, calculating minimum safe longitudinal and lateral distances to prevent collisions.
Comfort Reward ( $r_c$ ): Penalizes acceleration/deceleration exceeding comfortable thresholds ( $1.47 m/s^2$ ).
Priority-Guided Lane-Change Reward ( $r_{lc}$ ): The novel component. It dynamically weights lane-change decisions based on:
- Urgency ( $u_t$ ): Vehicles nearing their exit receive negative rewards if they are not in the correct lane, calculated using time-to-exit vs. time-needed-to-change-lanes.
- Staging Penalty ( $p_{stage}$ ): Discourages vehicles from occupying the exit lane too early, keeping faster lanes open for through-traffic.
Deadlock Penalty ( $r_d$ ): Prevents ramp vehicles from getting stuck at the end of acceleration lanes.

3. Key Contributions

First-of-its-Kind Framework: PALCAS is the first work to jointly address mandatory and discretionary lane changes using a federated learning paradigm for large-scale cooperative decision-making.
Hierarchical State Representation: Integrates microscopic vehicle data with macroscopic cluster-level and global traffic information, enhancing situational awareness.
Priority-Guided Reward Mechanism: Unifies maneuver feasibility and destination urgency, allowing the system to dynamically prioritize vehicles based on their proximity to exits and current traffic conditions.
Scalable Privacy-Preserving Architecture: Demonstrates a federated approach that enables agents to generalize from unseen scenarios without sharing raw data, suitable for heterogeneous traffic environments.

4. Simulation Results

The system was evaluated using SUMO (traffic simulation) and Eclipse MOSAIC (V2X communication) with a 2.4km, 5-lane highway scenario. It was compared against:

Baseline-1: Centralized MARL (single agent controlling all).
Baseline-2: Decentralized MARL without federated learning (no knowledge sharing).

Key Performance Metrics (at 60% CAV penetration):

Traffic Efficiency: PALCAS achieved the highest average speed (30.03 m/s), outperforming Baseline-1 by 3.12% and Baseline-2 by 6.86%.
Safety (Collision Rate): PALCAS reduced the collision rate by 19.67% compared to Baseline-1 and 75.5% compared to Baseline-2.
Driving Comfort: Acceleration trajectories were significantly smoother, staying within the $[-1, 1] m/s^2$ range, whereas baselines fluctuated between $[-4.5, 2.6] m/s^2$ .
Destination Success Rate (DSR): PALCAS improved DSR by 132.94% over Baseline-1 and 113.9% over Baseline-2 at 60% penetration, effectively preventing missed exits.
Merging Success Rate (MSR): PALCAS achieved >93% success, comparable to baselines, though Baseline-2 slightly outperformed in MSR due to its myopic (short-term) local optimization.

Ablation Study:
Removing the priority-guided reward component resulted in a drastic performance drop, confirming its critical role in coordinating complex multi-vehicle maneuvers and preventing congestion.

5. Significance

Real-World Applicability: The system addresses the specific challenges of urban highways with frequent ramps, a scenario where current centralized or single-agent systems struggle.
Privacy and Scalability: By using Federated Learning, PALCAS offers a viable path for deploying cooperative autonomous driving systems without compromising data privacy or requiring massive centralized computing power.
Safety and Efficiency Balance: The priority-aware reward function successfully resolves the trade-off between keeping traffic flowing (staying in fast lanes) and ensuring vehicles reach their destinations (moving to exit lanes), significantly reducing congestion and accidents.
Real-Time Feasibility: Inference times were measured at under 100ms for 90% of decisions, indicating the system is suitable for real-time deployment.

In conclusion, PALCAS demonstrates that combining Federated Learning with Priority-Aware Reinforcement Learning creates a robust, scalable, and safe framework for next-generation autonomous lane-change management.