Dynamic Lookahead Distance via Reinforcement Learning-Based Pure Pursuit for Autonomous Racing

1. Problem Statement

Pure Pursuit (PP) is a standard geometric path-tracking algorithm in autonomous racing due to its simplicity and low computational cost. However, its performance is critically dependent on the lookahead distance ($L_d$), creating a fundamental trade-off:

• Short $L_d$: Improves responsiveness in curves but causes steering oscillations and instability on straight segments.
• Long $L_d$: Ensures stability on straights but leads to "cutting corners" and poor tracking in sharp turns.

Existing solutions rely on fixed lookahead (requiring manual retuning for every track) or heuristic adaptive rules (e.g., $L_d$ based on speed or curvature). These heuristics are often hand-tuned, lack generalization across different tracks or vehicle dynamics, and fail to capture complex context-dependent behaviors required for high-speed racing.

2. Methodology

The authors propose a hybrid control framework that augments the classical Pure Pursuit controller with a Reinforcement Learning (RL) agent. Instead of replacing the geometric controller, the RL agent dynamically adjusts the single interpretable parameter ($L_d$) in real-time.

A. System Architecture

• Localization: Monte Carlo Localization (MCL) using a particle filter and LiDAR data.
• Global Planning: A minimum-curvature racing line is precomputed using the TUM global_racetrajectory_optimization framework to generate a reference trajectory with waypoints, curvature, and a target speed profile.
• Control Loop:
  1. PPO Agent: Observes vehicle state and outputs a dynamic $L_d$.
  2. Pure Pursuit Controller: Uses the dynamic $L_d$ to calculate the target waypoint and steering command.
  3. Actuation: Steering and throttle commands are executed on the vehicle.

B. Reinforcement Learning Setup

• Algorithm: Proximal Policy Optimization (PPO) implemented via Stable-Baselines3.
• Observation Space ($s_t$): A 5-dimensional vector:
  • Vehicle speed ($v_t$).
  • Path curvature at three horizons: closest ($\kappa_0$), medium ($\kappa_1$), and far ($\kappa_2$).
  • Curvature change ahead ($\Delta\kappa = \kappa_1 - \kappa_0$).
• Action Space ($a_t$): Continuous output representing the lookahead distance $L_{t+1}$, constrained to $[0.35, 4.0]$ meters. An exponential smoothing filter is applied to reduce jitter.
• Reward Function ($R_t$): A weighted sum designed to promote:
  • Progress: Waypoints advanced ($\Delta p_t$).
  • Speed: Vehicle velocity ($v_t$).
  • Smoothness: Penalizing large changes in $L_d$ and high curvature.
  • Safety: Collision penalties (LiDAR range < 0.2m) and stall penalties.
  • Shaping: A heuristic term guiding $L_d$ toward an ideal value based on speed and curvature to stabilize training.
• Training Environment: F1TENTH Gym simulator. Training utilized Optuna for hyperparameter tuning, KL-divergence penalties, and learning-rate decay for stability.

3. Key Contributions

1. Hybrid RL-Geometric Control: A novel framework that integrates PPO to adaptively tune the Pure Pursuit lookahead distance, preserving the interpretability and robustness of classical control while adding data-driven adaptability.
2. Context-Aware Policy: The agent learns to map multi-horizon curvature and speed features to an optimal $L_d$, effectively learning to increase lookahead on straights and decrease it in curves without manual rule-crafting.
3. Zero-Shot Generalization: The policy is trained on a single track (Austin) and evaluated on unseen tracks (Montreal, Yas Marina) without fine-tuning, demonstrating robustness to track geometry changes.
4. Sim-to-Real Transfer: Successful deployment on a real 1:10-scale RoboRacer vehicle, validating that the learned policy transfers effectively from simulation to hardware.

4. Experimental Results

The proposed RL-based Dynamic-L controller was compared against:

• Fixed-L PP: Pure Pursuit with a static lookahead.
• Adaptive PP: Pure Pursuit with a hand-crafted speed-dependent lookahead rule.

Simulation Results (Unseen Tracks):

• Montreal Track:
  • RL PP: Completed 10/10 laps at +13% speed profile. Mean lap time: 33.16s.
  • Adaptive PP: Completed 10/10 laps at +10% speed profile. Mean lap time: 34.15s.
  • Fixed PP: Required -10% speed reduction to complete laps. Mean lap time: 41.36s.
• Yas Marina Track:
  • RL PP: Completed 10/10 laps at +15% speed profile. Mean lap time: 46.05s.
  • Adaptive PP: Completed 10/10 laps at +14% speed profile. Mean lap time: 46.79s.
  • Fixed PP: Failed to complete repeated laps at accelerated speeds; recorded 52.81s at original speed.

Real-World Validation:

• On a physical 1:10-scale car with a speed cap of 5 m/s, the RL PP completed 10/10 laps with a mean time of 11.13s.
• The Fixed-L PP controller failed to complete repeated laps reliably under the same conditions.

5. Significance

This work demonstrates that Reinforcement Learning can effectively augment classical control without sacrificing interpretability. By focusing on adapting a single, high-level parameter ($L_d$) rather than learning end-to-end control, the system achieves:

• Superior Robustness: The ability to handle aggressive speed profiles and unseen track geometries without retuning.
• Performance: Faster lap times compared to both fixed and heuristic adaptive baselines.
• Practicality: A lightweight, modular solution compatible with standard ROS2 autonomy stacks, bridging the gap between theoretical RL adaptability and the safety requirements of real-world autonomous racing.

The paper concludes that learning-based adaptation of geometric controller parameters is a viable path toward more intelligent, context-aware autonomous systems.