Technical Summary: CLOVER

Problem Statement

End-to-end autonomous driving planners are typically trained via imitation learning on a single logged human trajectory. However, they are evaluated using rule-based planning metrics (e.g., NAVSIM PDMS/EPDMS) that assess safety, feasibility, progress, and comfort. This creates a fundamental training–evaluation mismatch:

1. Trajectories close to the logged path may still violate safety rules or induce unsafe interactions.
2. Alternative trajectories that deviate from the demonstration may be valid and high-scoring but are never seen during training.

This mismatch is particularly limiting for proposal-selection planners, which generate multiple candidate trajectories and select the best one. Their performance is constrained by two coupled factors: the generator's ability to cover high-quality alternatives (coverage) and the scorer's ability to correctly rank them (ranking quality). Single-trajectory supervision often collapses the candidate set around one realized behavior, failing to capture the multi-modal nature of driving (e.g., maintaining speed vs. braking vs. changing lanes).

Methodology: CLOVER Framework

CLOVER (Closed-LOop Value Estimation and Ranking) is a framework designed to close the loop between proposal generation and trajectory ranking. It employs a lightweight generator–scorer architecture and operates in two distinct training stages.

Architecture

• Generator: Produces $K$ candidate future ego trajectories ($\tau_i$) based on multi-view camera images and ego state.
• Scorer: A trajectory-level module that predicts planning-metric sub-scores (collision, drivable area, progress, comfort, etc.) for each candidate.
• Inference: Candidates are ranked by the composed predicted score (using the official PDMS rule $\Gamma_{PDMS}$), and the top-ranked trajectory is selected. The rule-based evaluator is not required at deployment.

Stage 1: Evaluator-Filtered Proposal Coverage

To address the lack of diversity in single-trajectory imitation, CLOVER constructs a pseudo-expert set $\mathcal{E}(o)$ for each scene during training.

• Generation: Instead of perturbing the logged trajectory, candidates are generated from interpretable action families (e.g., varying lateral offsets, acceleration profiles, stop-go behaviors, boundary cases) using privileged metric-cache information (route centerlines, future obstacle occupancy) available only in training.
• Filtering: Candidates are pre-filtered for drivable area and occupancy, scored by the true rule-based evaluator, and selected via a coverage-aware strategy to ensure diverse sub-score patterns and score ranges.
• Training Objective: The generator is trained to cover these pseudo-experts while retaining the logged trajectory as a prior. The scorer is pre-trained to predict the true evaluator sub-scores.
  • Loss: $L_{stage1} = \lambda_{gt} L_{gt} + \lambda_{pe} L_{pe} + \lambda_{score} L_{score}$, where $L_{pe}$ encourages coverage of the pseudo-expert set.

Stage 2: Conservative Closed-Loop Self-Distillation

This stage refines the expanded proposal distribution using a conservative, alternating procedure to avoid the instability of directly maximizing an imperfect learned reward.

1. Scorer Phase: The scorer is fitted to the true evaluator sub-scores on the current set of generated proposals.
2. Generator Phase: A frozen "teacher" (the current generator) produces proposals. The scorer evaluates them to select two target sets:
   • Top-k Set: Selected by the composed predicted score.
   • Vector-Pareto Set: Consisting of non-dominated proposals in the sub-score space (preserving safety-progress-comfort trade-offs).
   • The student generator is trained to cover these targets using a stability regularization term that discourages drift from the teacher's distribution.
   • Loss: $L_{gen} = \lambda_{traj} L_{gt} + \lambda_{topk} L_{set} + \lambda_{pareto} L_{set} + \lambda_{stab} L_{stab}$.

Theoretical Justification

The paper analyzes when an imperfect scorer can reliably improve the generator. It posits a selected-set enrichment condition: if the targets selected by the scorer (Top-k or Pareto) are statistically enriched with true high-quality trajectories compared to the current proposal distribution, then conservative set-level distillation increases the probability mass on high-score trajectories. The authors show that CLOVER satisfies this condition empirically, as scorer-selected targets have significantly higher true scores than the average proposal pool.

Key Contributions

1. CLOVER Framework: A closed-loop value estimation and ranking framework that couples proposal generation and trajectory scoring to address the training–evaluation mismatch.
2. Evaluator-Filtered Pseudo-Expert Supervision: A method to extend single-trajectory imitation to set-level multi-modal training, constructing diverse, high-quality candidate sets using privileged training-time information.
3. Conservative Closed-Loop Self-Distillation: An alternating training strategy that fits the scorer to true sub-scores and refines the generator using teacher-selected Top-k and Vector-Pareto targets with stability regularization, avoiding the pitfalls of unrestricted reward maximization.
4. Theoretical and Empirical Analysis: A demonstration that imperfect scorers can effectively guide refinement if the selected targets are statistically enriched under the true evaluator, validated by empirical data showing significant enrichment gaps in scorer-selected sets.

Experimental Results

CLOVER was evaluated on NAVSIM v1, NAVSIM v2 (EPDMS), NavHard, and nuScenes open-loop benchmarks.

• NAVSIM v1: Achieved 94.5 PDMS, outperforming the strong baseline DrivoR (93.7) and ranking first or second across all sub-scores.
• NAVSIM v2 (navtest): Achieved 90.4 EPDMS (updated official implementation) and 87.2 EPDMS (original code), matching or surpassing the strongest reported results.
• NavHard: Achieved 48.3 EPDMS on the challenging navhard-two-stage split, matching the strongest reported result.
• nuScenes Open-Loop: Achieved the lowest L2 displacement error (0.31m under ST-P3 protocol) and best/tied-best collision rate (0.10%) among compared methods.
• Ablation & Analysis:
  • Stage 1 significantly expanded the proposal space (Oracle@64 increased from 0.9933 to 0.9976) but introduced a low-score tail.
  • Stage 2 refined this distribution, improving the selected PDMS to 0.9448 while preserving high diversity (Pairwise ADE of 5.20 vs. baseline 1.80).
  • The gains were attributed to a stronger candidate distribution rather than just selecting a better top-1 trajectory.

Significance and Claims

The paper claims that CLOVER establishes a new state-of-the-art performance on major autonomous driving planning benchmarks. Its significance lies in:

• Resolving the Mismatch: Explicitly using evaluator-provided feedback to train a trajectory-level value estimator and refine the candidate distribution, rather than relying solely on single-trajectory imitation.
• Reliable Refinement: Demonstrating that an imperfect learned scorer can serve as a useful mediator for proposal refinement, provided the selected targets are statistically enriched under the true evaluator and updates remain conservative.
• Proposal-Centric Improvement: Showing that improvements in final planning metrics stem from enhancing both the quality and diversity of the entire candidate set, not just the final selection.

The authors note that while CLOVER resembles actor-critic reinforcement learning, it is distinct because it operates entirely offline on logged scenes with a rule-based evaluator, without online environment interaction or trial-and-error exploration. Future work is suggested to integrate history-aware or sequence-level scorers to better model temporal consistency for metrics like Extended Comfort (EC).