⚡ electrical engineering

Learning to Tune Pure Pursuit in Autonomous Racing: Joint Lookahead and Steering-Gain Control with PPO

This paper proposes a Proximal Policy Optimization-based reinforcement learning approach that jointly tunes the lookahead distance and steering gain of the Pure Pursuit controller in real-time, demonstrating superior lap time, tracking accuracy, and steering smoothness compared to traditional fixed-parameter, velocity-scheduled, and model-predictive control methods in autonomous racing.

Original authors: Mohamed Elgouhary, Amr S. El-Wakeel

Published 2026-02-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mohamed Elgouhary, Amr S. El-Wakeel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot car to race around a track. The car needs to stay on the road, go as fast as possible, and not crash into the walls.

This paper is about a new way to teach that robot car how to steer. Instead of building a super-complex brain from scratch, the authors took a simple, old-school steering method called Pure Pursuit and gave it a "smart assistant" powered by Artificial Intelligence (AI).

Here is the breakdown using simple analogies:

1. The Old Way: The "Fixed Gaze" Driver

Imagine you are driving a car. To stay in your lane, you pick a point on the road ahead of you and steer toward it. This is Pure Pursuit.

The Problem: If you pick a point too close to your car (like looking at the bumper), you will jerk the steering wheel wildly, making the ride bumpy and unstable.
The Problem: If you pick a point too far away (like looking at the horizon), you will turn too late. You'll cut corners too wide or hit the wall on sharp turns.

Traditionally, engineers tried to solve this with a simple rule: "If you go fast, look further ahead. If you go slow, look closer."

The Flaw: This is like a rigid rulebook. It works okay on a straight road, but it fails when the track gets weird. It doesn't know that a sharp turn at 30 mph needs a different strategy than a sharp turn at 60 mph. It's like wearing the same size shoes for running a marathon and climbing a mountain.

2. The New Way: The "Smart Co-Pilot"

The authors decided to keep the simple steering method but add a Reinforcement Learning (RL) agent. Think of this agent as a smart co-pilot sitting next to the driver.

This co-pilot doesn't drive the car; it just whispers two numbers to the driver every split second:

How far ahead to look? (The "Lookahead Distance")
How aggressively to turn? (The "Steering Gain")

3. How the Co-Pilot Learns (The Video Game Analogy)

The AI didn't learn by reading a manual. It learned by playing a video game (a simulation) thousands of times, just like a human learning to play Mario Kart.

The Training: The AI played on a virtual track (Hockenheim). It tried millions of different combinations of "how far to look" and "how hard to turn."
The Reward: If it stayed on the track and went fast, it got a "high score" (reward). If it crashed or went slow, it got a "low score" (penalty).
The Result: Eventually, the AI figured out the perfect dance. It learned that on a long straight, it should look far ahead and turn gently. But the moment it sees a sharp curve coming up, it instantly says, "Look closer! Turn harder!"

4. Why This is Special

Most AI racing cars try to replace the driver entirely, learning to steer from scratch. This is like trying to teach a baby to drive a Formula 1 car. It's hard to understand why the AI made a move, and it's hard to make it work on a new track.

This paper's approach is different:

It keeps the simple driver: The core steering logic is still the old, reliable "Pure Pursuit."
It tunes the knobs: The AI just adjusts the two dials (Lookahead and Aggressiveness) in real-time.
It transfers easily: Because the AI learned how to adjust rather than memorizing a specific track, it could jump from the training track to two completely new tracks (Montreal and Yas Marina) and perform perfectly without any re-training. This is called "Zero-Shot Transfer."

5. The Real-World Test

The authors didn't just stop at the computer. They put their AI on a real, small-scale race car (about the size of a microwave).

The Result: The AI-tuned car was faster, smoother, and more accurate than:
- The car with the fixed rules (the "rulebook" driver).
- The car with the "speed-based" rules.
- Even a very complex, math-heavy computer model (called MPC) that is usually considered the gold standard.

The Big Takeaway

Think of this like a smart thermostat for your car's steering.
Old cars had a manual dial you had to set yourself. This new system has a smart sensor that feels the road, the speed, and the curves, and automatically turns the dial to the perfect setting instantly.

It proves that you don't need a massive, complicated AI brain to win a race. Sometimes, you just need a simple steering method with a smart assistant that knows exactly how to tweak the settings for the moment.

1. Problem Statement

Pure Pursuit (PP) is a popular geometric path-tracking controller in autonomous racing due to its computational efficiency and interpretability. However, its performance is highly sensitive to two key parameters:

Lookahead Distance ( $L_d$ ): Determines the target point on the reference path. Small values yield agile cornering but cause oscillation on straights; large values smooth motion but cause understeer in tight bends.
Steering Gain ( $g$ ): Scales the steering command. Aggressive gains induce noise, while conservative gains slow convergence.

Limitations of Existing Solutions:

Rule-based Schedules: Traditional methods adjust $L_d$ and $g$ using fixed linear functions of velocity or curvature. These often fail to transfer across different tracks, speed profiles, or vehicle platforms.
Model Predictive Control (MPC): While powerful, MPC requires significant modeling effort, online optimization, and tuning, making it computationally expensive and less interpretable.
End-to-End RL: Directly learning steering commands often lacks interpretability and struggles with sim-to-real transfer.

Goal: The authors aim to retain the simplicity and interpretability of Pure Pursuit while using Reinforcement Learning (RL) to jointly and online tune the lookahead distance ( $L_d$ ) and steering gain ( $g$ ) to adapt to varying track geometries and speeds without per-map retuning.

2. Methodology

The proposed framework integrates a Proximal Policy Optimization (PPO) agent with a classical Pure Pursuit controller within a ROS 2 stack.

A. System Architecture

Localization: Uses LiDAR-based Monte Carlo Localization (MCL) for pose estimation.
Reference Path: Generates a global minimum-curvature raceline with a friction-limited speed profile using TUM's global optimization tools.
Control Loop:
1. The PPO policy observes the state and outputs $(L_d, g)$ .
2. These parameters are passed to the Pure Pursuit controller.
3. PP computes the steering angle $\gamma = g \cdot \frac{2y'}{L_d^2}$ (where $y'$ is the lateral error to the lookahead point).
4. A safety "teacher" (linear rule) acts as a fallback if RL commands become stale.

B. Reinforcement Learning Formulation

Algorithm: Proximal Policy Optimization (PPO) using Stable-Baselines3.
Observation Space ( $s_t$ ): A compact 5-dimensional vector:
- Current velocity ( $v_t$ ).
- Curvature taps at three horizons (near, mid, far) along the raceline ( $\kappa_0, \kappa_1, \kappa_2$ ).
- Curvature difference ( $\Delta \kappa = \kappa_1 - \kappa_0$ ).
Action Space ( $a_t$ ): A continuous 2D vector:
- Lookahead distance $L_d \in [0.35, 4.0]$ meters.
- Steering gain $g \in [0.45, 1.15]$ .
- Note: Actions are smoothed via a first-order exponential filter before application to prevent sudden jumps.
Reward Function: Designed to balance:
- Speed progress ( $v_t$ ).
- Agreement with a "teacher" baseline (penalizing deviation from heuristic targets).
- Smoothness (penalizing rapid changes in $L_d$ and $g$ ).
- Safety (penalizing collisions and low speed).
- Pre-bend preparation (rewarding early reduction of $L_d$ before sharp turns).

C. Training Strategy

Environment: F1TENTH Gym simulator.
Curriculum: Training is performed on the Hockenheim track with the speed profile uniformly scaled by +30% to force the policy to learn robust curvature-speed relationships rather than overfitting to a specific speed.
Stability: Uses KL-constrained updates, gradient clipping, and observation/return normalization (VecNormalize).

3. Key Contributions

Joint Parameter Tuning Framework: Unlike prior work that tunes only $L_d$ or uses end-to-end control, this paper proposes a hybrid approach where RL jointly optimizes both lookahead distance and steering gain, preserving the geometric steering law of PP.
Practical Training Recipe: The authors provide a robust training setup including specific reward shaping, hyperparameter tuning (Optuna), and stability techniques (KL constraints, entropy bonuses) that enable learning a curvature-and-speed-aware policy.
Sim-to-Real Deployment: The system is deployed on a physical F1TENTH vehicle using a ROS 2 stack, demonstrating zero-shot transfer to unseen tracks (Montreal and Yas Marina) without retraining.
Comprehensive Evaluation: The study includes ablation studies, comparisons against fixed-PP, velocity-scheduled adaptive PP, and a Kinematic MPC tracker, validating performance in both simulation and real-world hardware.

4. Experimental Results

The system was evaluated on three tracks: Hockenheim (training), Montreal, and Yas Marina (zero-shot evaluation).

A. Simulation Results (Montreal & Yas Marina)

Lap Time: The Joint RL-PP controller achieved the fastest lap times, outperforming:
- Fixed PP: Significantly slower (e.g., 41.35s vs 32.85s on Montreal).
- Velocity-Scheduled Adaptive PP: Slower than RL-PP (e.g., 34.25s vs 32.85s).
- Kinematic MPC: Surprisingly, the MPC tracker was slower (48.06s) under the specific constraints and horizon used, likely due to the complexity of online optimization vs. the agility of the tuned geometric controller.
Zero-Shot Generalization: The policy trained on Hockenheim successfully adapted to Montreal and Yas Marina without fine-tuning, maintaining low lap-to-lap variance.
Ablation Study: The Joint (Ld, g) variant consistently outperformed the Ld-only variant, proving that learning the steering gain provides measurable benefits in handling tight corners and straights.

B. Real-World Results (F1TENTH Car)

Performance: On the physical car (capped at 6 m/s for safety), the Joint RL-PP achieved the best mean lap time (9.46s) compared to Adaptive PP (9.72s) and Fixed PP (9.85s).
MPC Performance: The MPC tracker was significantly slower (15.42s) on the real car, highlighting the computational overhead and tuning difficulty of MPC in real-time hardware constraints compared to the lightweight RL-tuned PP.
Stability: The "teacher" fallback was never triggered during evaluation runs, indicating the RL policy remained stable and responsive throughout the laps.
Tracking Quality: Qualitative analysis showed the RL-PP followed the reference path more closely with smoother steering inputs and less lateral error than baselines.

5. Significance and Conclusion

This paper demonstrates that deep reinforcement learning can effectively complement classical geometric control to enhance adaptability and robustness in autonomous racing.

Interpretability: By keeping the Pure Pursuit steering law fixed and only tuning its parameters, the system remains interpretable and safe, unlike black-box end-to-end controllers.
Efficiency: The approach avoids the heavy computational cost of MPC while outperforming it in lap time and tracking accuracy in the tested scenarios.
Generalization: The ability to transfer from a high-speed simulation environment to a real car on unseen tracks with zero-shot adaptation validates the robustness of the learned policy.

The authors conclude that policy-guided parameter tuning is a reliable method to improve classical controllers, reducing the need for per-track manual retuning and enabling high-performance autonomous racing on resource-constrained platforms.