💻 computer science

Before Parc Fermé: RL-Time Pruning for Efficient Embodied LLMs in Autonomous Driving

The paper proposes "Before Parc Fermé" (BPF), a novel pruning strategy that performs iterative model compression during Reinforcement Learning to optimize embodied LLM controllers for autonomous driving, achieving superior performance-memory trade-offs and up to 27% higher decode throughput compared to post-training pruning or selecting smaller dense models.

Original authors: Luca Benfenati, Ali Azimi, Matteo Risso, Fabio Carapellese, Daniele Jahier Pagliari, Alessio Burrello

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Luca Benfenati, Ali Azimi, Matteo Risso, Fabio Carapellese, Daniele Jahier Pagliari, Alessio Burrello

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, super-smart robot driver named RobotxR1. This driver is powered by a "brain" called a Large Language Model (LLM). Think of this brain like a world-class Formula 1 engineer who has read every book on driving, knows the physics of every car, and can understand any instruction you give it in plain English.

However, there's a problem: this engineer is too big. They carry a massive backpack full of books (memory) and take a long time to think before giving instructions (latency). If you try to put this heavy backpack on a small, fast race car (the actual robot), the car becomes too slow to race in real-time. It's like trying to run a marathon while carrying a piano.

The Problem: When to Shrink the Brain?

To make the robot faster, engineers usually try to "prune" the brain. Pruning is like editing a manuscript: you cut out sentences, paragraphs, or whole chapters that aren't necessary, making the book shorter and lighter.

But here is the tricky part: When do you do the editing?

After the race is over? (Post-training): You train the full brain, let it learn everything, and then try to cut it down. The problem is, the brain has already memorized the "wrong" heavy parts, and cutting them might break its ability to drive.
Before the training starts? (Pre-training): You cut the brain first, then train it. The problem is, you don't know yet which parts are actually useless until the brain has tried to drive.

The Solution: "Before Parc Fermé" (BPF)

The authors of this paper propose a new strategy called Before Parc Fermé (BPF).

To understand the name, imagine Formula 1 racing. Before a race, cars go into a "Parc Fermé" (a locked garage) where they are sealed up. No one is allowed to touch or modify the cars anymore. The car that enters the garage is the car that races.

In the world of AI training, the "Parc Fermé" is the moment when training is finished and the model is locked in place. The authors argue that you shouldn't wait until the model is locked in the garage to start editing it. Instead, you should start editing while the car is still being tuned on the track.

They call this RL-Time Pruning (Reinforcement Learning Time Pruning).

How It Works: The Two Variants

The paper tests two ways to do this "on-the-track" editing:

BPF-RL (The Iterative Edit): Imagine the robot driver is learning to drive a race track. Every few laps, the engineers step in, look at the driver's notes, and say, "You don't need this specific paragraph about tire pressure; let's cut it out." Then, the driver continues the next few laps with the lighter notes, learning to adapt to the missing information immediately. They repeat this process until the notes are just the right size.
BPF-SFT/RL (The Two-Stage Edit): First, they do a light edit while the driver is learning the basic rules (Supervised Fine-Tuning). Then, once the driver starts racing in real-time simulations (Reinforcement Learning), they do the heavy editing, cutting out more fluff while the driver is actively reacting to the track.

The Results: Lighter, Faster, and Smarter

The researchers tested this on a real autonomous driving setup with two parts:

DecisionxR1: The "Brain" that decides what to do.
MPCxR1: The "Hands" that actually steer the car.

Here is what they found:

Better Trade-offs: If you just picked a naturally smaller, weaker brain (a 1.5B model) instead of a big one, the car drove worse. But if they took the big brain and used their BPF-SFT/RL method to shrink it, the resulting "mini-brain" was 1.69 times better at balancing size and performance than just picking the small brain to begin with. It kept the "smarts" of the big model but the "weight" of the small one.
Real-World Speed: When they put these models on a real robot car equipped with a Jetson AGX Orin computer, the pruned models were 27% faster at generating instructions.
The "Verbose" Trap: They discovered a surprising twist. Sometimes, making a model smaller didn't make the whole system faster. Why? Because the smaller model sometimes started writing longer sentences to explain its decisions. It's like a lighter runner who starts talking to themselves the whole time, slowing them down. This taught them that you can't just look at the model size; you have to watch the whole pipeline.

The Bottom Line

The paper claims that for robots that need to think and act in real-time (like self-driving cars), you shouldn't wait until training is over to make the AI smaller. Instead, you should shrink it while it is learning, allowing the robot to adapt to its own "surgery" in real-time.

This "Before Parc Fermé" approach creates a robot driver that is light enough to be fast, but smart enough to handle complex driving tasks, outperforming both the heavy original models and the naturally small models.

Technical Summary: Before Parc Fermé (BPF) for Efficient Embodied LLMs

Problem Statement

Embodied Large Language Models (LLMs) are increasingly deployed in robotic control pipelines, such as autonomous driving, to interpret natural language commands and adapt controller parameters. However, these models face significant deployment challenges due to memory overheads and generation latency, which hinder real-time execution on constrained hardware (e.g., Jetson AGX Orin).

While pruning offers a solution to reduce model size and computational cost, the optimal timing for pruning within the specific training pipeline of embodied LLMs remains unclear. These systems typically undergo Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) with closed-loop feedback.

Post-training pruning (after SFT and RL) compresses the final model but offers the remaining weights no opportunity to adapt to the removed structures.
Pre-RL pruning (after SFT, before RL) removes structures before the model has learned the specific closed-loop behaviors required for the task, potentially discarding critical components.
Direct selection of smaller dense models (e.g., 1.5B vs. 3B parameters) often results in a suboptimal trade-off between model size and control adaptability, as larger models provide significantly better reasoning capabilities that smaller models cannot match.

The core challenge is determining when to prune to maximize the retention of reasoning and control abilities while minimizing resource usage.

Methodology: Before Parc Fermé (BPF)

The authors propose Before Parc Fermé (BPF), a family of pruning strategies that integrates compression directly into the training process, specifically coupling pruning with the RL phase. The name "Parc Fermé" refers to the Formula 1 period where cars are locked from modification; here, it metaphorically denotes the post-training stage where the architecture is fixed. BPF compresses the model before this lock-in, while it is still being optimized for closed-loop behavior.

The framework utilizes LLM-Pruner with a Taylor-based importance score to perform structured pruning. The paper evaluates two primary BPF variants against several baselines:

BPF-RL (Iterative Pruning during RL):
- The model undergoes full SFT without pruning.
- During the RL phase, pruning is performed iteratively at predefined intervals ( $\Delta t_{RL}$ ).
- At each interval, the lowest-scoring parameter groups are removed to incrementally reach the target pruning ratio.
- This allows the remaining weights to adapt to the removed structures while still receiving closed-loop task feedback.
BPF-SFT/RL (Split Pruning):
- SFT Stage: A partial compression is applied after an initial warm-up phase, removing a portion of the model ( $\rho_{SFT}$ ).
- RL Stage: The remaining compression ( $\rho - \rho_{SFT}$ ) is achieved iteratively during RL, similar to BPF-RL.
- This approach leverages SFT to remove clearly redundant structures based on task prompts, while relying on RL feedback to refine the compression based on closed-loop performance.

Baselines for Comparison:

Post-Training Pruning (PTP): Pruning applied after full SFT and RL.
PTP + Recovery: PTP followed by additional RL steps to recover performance.
SFT-stage Pruning: Single compression step during SFT before RL.
Smaller Dense Models: Directly using smaller architectures (e.g., Qwen2.5-1.5B) from the same model family.

Key Contributions

Introduction of BPF: A novel pruning recipe that couples RL with structured pruning for embodied LLM controllers, demonstrating that compression timing is a critical design choice.
Performance vs. Efficiency Trade-off: The authors show that BPF strategies, particularly BPF-SFT/RL, achieve superior trade-offs between model size and task performance compared to post-training pruning or simply selecting smaller dense models.
End-to-End Evaluation: The work emphasizes that pruning embodied LLMs must be evaluated holistically. Changes in generated token counts due to pruning can alter downstream latency and behavior, necessitating full-pipeline evaluation rather than isolated metric analysis.

Experimental Results

The methods were evaluated on RobotxR1, an autonomous driving pipeline comprising two modules:

DecisionxR1: Evaluates if current behavior matches user instructions.
MPCxR1: Adapts Model Predictive Control (MPC) parameters.

1. Task Performance and Adaptability

BPF-SFT/RL Superiority: On the DecisionxR1 module (3B parameters), BPF-SFT/RL achieved the best trade-off. At a 45% pruning ratio, it improved control adaptability by 11.4% compared to PTP+R and 17% compared to PTP.
Comparison with Smaller Models: When compressing the 3B model, BPF-SFT/RL achieved a 1.69× better size–end-to-end performance trade-off than directly selecting the 1.5B dense model. Specifically, BPF-SFT/RL reduced parameters by 0.91B with a 7.61% loss in adaptability, whereas switching to the 1.5B model reduced parameters by 1.55B but incurred a 21.7% loss.
Joint Pruning: Pruning both modules jointly revealed that MPCxR1 is more sensitive to pruning than DecisionxR1. Moderate pruning of MPCxR1 (e.g., 10%) combined with higher pruning of DecisionxR1 (e.g., 30-45%) yielded the best Pareto-optimal results.

2. Embedded Deployment (Jetson AGX Orin)

Throughput and Memory: Pruned models showed significant improvements in decode throughput and memory footprint. For instance, pruning DecisionxR1 by 45% reduced memory from 2.07GB to 1.51GB and increased throughput from 45 to 58 tokens/sec.
Latency Nuance: The paper notes that reducing parameters does not automatically reduce end-to-end latency. Pruning can increase the verbosity of generated tokens (e.g., MPCxR1 generating more text), which can offset computational gains. The optimal configuration for latency (20% pruning on DecisionxR1, 0% on MPCxR1) reduced latency by 19.5% with only a 0.8% drop in task performance.

Significance and Claims

The paper claims that BPF-SFT/RL represents the most effective strategy for deploying embodied LLMs in autonomous driving. Its primary significance lies in demonstrating that:

Timing Matters: Pruning during the RL phase, while the model is still adapting to closed-loop feedback, preserves more reasoning and control capability than pruning after training is complete.
Efficiency vs. Capability: It is possible to compress larger, more capable models significantly (e.g., 45%) while retaining performance levels closer to the original large model than what is achievable by simply switching to a smaller dense model.
Holistic Evaluation: Effective compression for robotic systems requires evaluating the entire control pipeline, as pruning-induced changes in token generation length can critically impact downstream latency and system behavior.

The authors acknowledge limitations, including the lack of joint training for pruned modules (MPCxR1 was trained on outputs from unpruned DecisionxR1), the use of fixed pruning intervals, and the absence of quantization-aware pruning, which currently requires padding to align dimensions for inference runtimes.