Technical Summary: Bridging Local Observation and Global Simulation in Closed-Loop Traffic Modeling (CRAFT)

1. Problem Formulation

The paper addresses a fundamental local-to-global context mismatch in autoregressive traffic simulators. Most state-of-the-art simulators are trained on large-scale, ego-centric driving logs (e.g., Waymo, nuPlan) under an imitation learning paradigm. In these logs, the ego vehicle possesses rich local observations, but surrounding agents are only partially observed due to sensor limits, occlusions, and perception boundaries.

Consequently, simulators learn incomplete context–action mappings. A valid maneuver in a log (e.g., a vehicle braking) may be recorded without the downstream context justifying it (e.g., an occluded obstacle). When such models are deployed in globally observable closed-loop environments, they reproduce these actions even when the true causal context is absent. This leads to unrealistic behaviors during long-horizon rollouts, including abnormal stops, unsafe interactions, and traffic rule violations, which degrade the behavioral rationality of the simulation.

Existing solutions often rely on retraining the base simulator via supervised fine-tuning or reinforcement learning, which can suffer from sparse feedback, credit-assignment difficulties, and high computational costs. The authors propose a test-time alignment strategy that corrects these biases without retraining the base generator.

2. Methodology: CRAFT

The authors propose CRAFT (Contextual pReference Alignment Framework for Traffic Simulation), a framework that decouples the generation of behaviors from their evaluation. It consists of three main stages:

A. Grouped Complete-Context What-If Simulation

Instead of relying solely on static logs, CRAFT utilizes the frozen base simulator ($\pi_\theta$) as a "globally observable sandbox."

• Process: Starting from logged initial states, the base simulator generates $N$ diverse stochastic rollouts (what-if scenarios) using different random seeds.
• Outcome: These rollouts evolve into diverse futures where all agents interact under the full map context. This process exposes simulator-induced failures that are hidden in the original logs but emerge during closed-loop generation.
• Annotation: An automated rule-based evaluator assigns binary labels ($0$ for abnormal, $1$ for reasonable) to every agent-time token based on human-aligned driving priors (safety, rule compliance, kinematic feasibility). This creates a grouped preference dataset ($D_{pref}$) containing alternative futures for the same initial scene.

B. Training the Contextual Preference Evaluator (CPE)

A lightweight Contextual Preference Evaluator ($\pi_\phi$) is trained on $D_{pref}$ to predict the rationality of motion tokens under complete scene context.

• Architecture: The CPE reuses the frozen scene encoder of the base simulator and employs a temporal convolutional decoder to output a dense preference score matrix ($R \in [0, 1]^{N_a \times T_f}$).
• Loss Functions: The training objective combines three components:
  1. Absolute Token-Level Supervision ($L_{bce}$): Weighted Binary Cross-Entropy against the auto-labeled binary ground truth.
  2. Intra-Trajectory Preference ($L_{intra}$): A margin ranking loss that encourages the model to distinguish between normal behavior segments and subsequent failure segments within a single rollout.
  3. Inter-Rollout Preference ($L_{inter}$): A Bradley-Terry objective that compares alternative futures generated from the same initial scene, teaching the evaluator to prefer globally coherent trajectories over locally plausible but inconsistent ones.

C. Test-Time Preference Alignment

During inference, the CPE acts as a plug-in guidance module that reweights the decoding distribution of the frozen base simulator.

• Lookahead Search: At each step $t$, the base simulator proposes $K$ candidate action tokens.
• Independent Marginal Approximation: To avoid the intractable $K^{N_a}$ complexity of evaluating all joint action combinations, CRAFT samples $K$ parallel joint action scenarios. It evaluates the preference score of each candidate action against a sampled plausible background traffic (approximating the joint expectation).
• Reweighting: The original decoding probabilities are calibrated using the preference scores ($R$) via a softmax with a guidance scale $\beta$:
  $$P_{post} \propto P_{prior} \cdot \exp(\beta \cdot R)$$
  This steers the autoregressive decoding toward behaviors consistent with the full traffic scene before execution.

3. Key Contributions

1. Identification of Mismatch: The paper explicitly identifies and analyzes the degradation of closed-loop behavior caused by training on ego-centric logs with incomplete context, showing that imitation learning leads to spurious context-action associations.
2. Decoupled Evaluator (CPE): A novel evaluator trained on self-generated "what-if" rollouts grounded by human-aligned priors. It learns to distinguish globally coherent behaviors from locally plausible but contextually inconsistent ones without modifying the base simulator's weights.
3. Plug-in Test-Time Alignment: A strategy to deploy CPE as a lightweight, inference-time module that guides autoregressive decoding. This approach improves behavioral rationality across different base simulators (demonstrated on CAT-K and SMART) without the instability or cost of retraining.

4. Experimental Results

The framework was evaluated on the Waymo Open Motion Dataset (WOMD) using the WOSAC benchmark and additional behavioral rationality metrics.

• Safety and Compliance: CRAFT significantly reduced abnormal behaviors compared to baselines (including Top-K sampling, Auto-labelers, and RL-finetuned models).
  • Collisions: Reduced by 31.2%.
  • Traffic Violations: Reduced by 33.2%.
  • Offroad: Reduced by 16.6% (scene-level).
• Behavioral Rationality: CRAFT achieved the lowest Jensen-Shannon Distance (JSD) for speed, angular speed, and nearest-agent distance distributions, indicating better alignment with real-world driving distributions.
• Realism Trade-off: The paper notes a slight degradation in the WOSAC "Realism" metric (which measures one-to-one similarity to logged trajectories). The authors argue this is a necessary trade-off: the base model's high realism score often reflects a close match to incomplete local logs, whereas CRAFT prioritizes global behavioral rationality, correcting behaviors that are statistically likely in logs but irrational in full context.
• Efficiency: CRAFT adds moderate runtime overhead (76ms per step) compared to Top-K sampling (2ms) but is significantly faster than rule-based auto-labelers (~491ms) while providing superior performance.

5. Significance and Claims

The paper claims that CRAFT offers a lightweight and complementary path to improving closed-loop traffic simulation.

• Shift from Passive to Active: It shifts the paradigm from passive imitation of logs to active alignment with global scene context.
• Failure Discovery: By using the simulator itself to generate failure cases (what-if rollouts), the method creates a self-supervised signal to correct the very biases the simulator introduces.
• Generalizability: The plug-in nature of CPE allows it to improve the behavioral rationality of different base simulators without retraining them, making it applicable to existing world models and simulators where retraining is costly or unstable.
• Limitations: The authors modestly acknowledge that the method is bottlenecked by the quality of the candidate set proposed by the base simulator. If the optimal action is not included in the Top-K candidates (due to the base model's prior being too weak), CPE cannot recover it, only re-rank the available suboptimal choices.

In summary, CRAFT bridges the gap between local observation and global simulation requirements, demonstrating that test-time preference alignment can significantly enhance the safety and rationality of autonomous driving simulations without compromising the underlying generative model's structure.