💻 computer science

Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features

Adaptive-WAM is a quality-guided early-exit planning framework that leverages intermediate features from video diffusion models to dynamically allocate computational depth, achieving state-of-the-art autonomous driving performance with significantly reduced latency by bypassing iterative video generation.

Original authors: Sining Ang, Yuguang Yang, Yan Wang

Published 2026-08-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sining Ang, Yuguang Yang, Yan Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. To do this safely, the robot needs to understand not just what is happening right now, but how the world will change in the next few seconds. Will that pedestrian step off the curb? Will the car in front brake suddenly? For a long time, scientists have tried to solve this by building "world models"—massive AI brains that try to imagine the entire future, frame by frame, like a movie director filming a scene before it happens. These models are incredibly powerful, but they are also heavy and slow. It's like asking a chef to bake a full three-course meal just to decide if they should add a pinch of salt to the soup. The robot has to wait for the whole "movie" of the future to be generated before it can make a single driving decision, which takes too much time and computer power for a real car moving at highway speeds.

This brings us to a big question: Do we really need to watch the whole movie to know what to do? Maybe the robot can figure out the right move just by glancing at the first few seconds of the "future film," or by peeking at the middle of the script. This is the heart of the new research called Adaptive-WAM. The researchers wanted to see if they could make these giant, slow AI models faster and smarter by letting them "exit early." Instead of forcing the computer to run every single step of the calculation, they built a system that checks the quality of the answer as it goes. If the answer looks good enough, the computer stops and drives. If not, it keeps working. It's like a student taking a test who realizes they already know the answer to question five, so they don't need to waste time reading the rest of the chapter before writing it down.

The paper, titled "Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features," tackles this by looking at how these video-generating AI models work. These models usually break down the future into tiny steps, adding more detail as they go, much like an artist sketching a rough outline and then slowly filling in the colors. The researchers discovered something surprising: the AI doesn't need to finish the whole painting to know where the car should go. They found that the "middle layers" of the AI's brain—where it has started to make sense of the scene but hasn't finished rendering the final pixels—already contain enough information to make a safe driving decision.

To test this, the team built a new planner using a massive video model called Wan2.2. Imagine this model as a deep tunnel with many rooms. Usually, a car has to walk through every single room from start to finish to get the answer. The researchers, however, installed "exit doors" at several points along the tunnel (specifically at layers 5, 9, 15, 18, 22, and 30). At each exit, a small, fast "judge" looks at the trajectory (the path the car plans to take) that the AI has generated so far. This judge asks, "Is this path good enough?" If the answer is yes, the car exits immediately and drives. If the answer is no, the car keeps walking deeper into the tunnel to get a better answer.

The results were exciting. The researchers found that the quality of the driving decision didn't depend much on how "noisy" or blurry the future video was, but it did depend heavily on how deep the AI looked. They discovered that the best single path often came from the middle of the tunnel, not the very end. By using their smart "exit strategy," the new planner could make decisions in about 170 milliseconds on a powerful computer chip (an A100). This is roughly 10% faster than a standard planner that always stops at a fixed middle point, and nearly 47% faster than a planner that insists on walking through the entire tunnel. Even better, the system didn't just get faster; it got slightly more accurate, scoring 90.79 on a standard driving test called NAVSIM, beating the fixed-depth versions.

The paper also showed that this "smart exit" trick works even when the robot drives in a completely different city without any extra training. When tested on the nuScenes dataset (a different collection of driving scenes), the system made very few mistakes, with an average error of just 0.88 meters and a collision rate of only 0.08%. This suggests that the AI learned a general understanding of driving that isn't tied to one specific dataset.

Crucially, the researchers ruled out the idea that the computer needs to generate the full, high-definition future video to make a decision. They proved that the extra time spent rendering the final "movie" frames is wasted effort for the purpose of planning. They also showed that simply freezing the AI's brain and only training the exit doors wasn't enough; the whole system needed to be tuned together to work best.

In short, Adaptive-WAM is a clever way to make super-smart AI drivers much more efficient. It teaches the robot to trust its gut feeling when the answer is clear, rather than wasting time double-checking everything. By letting the AI stop early when the plan is good, the car can react faster to the real world, bringing us one step closer to self-driving cars that are both safe and fast. The code for this system will be released, allowing others to build on this idea of "quality-guided early exits" to make even smarter machines.

Technical Summary: Adaptive-WAM

Problem Statement

Large video diffusion models offer rich spatiotemporal priors for autonomous driving planning, yet existing World-Action Models (WAMs) often incur the full computational cost of iterative future-video generation and VAE decoding, even when deployment only requires a low-dimensional ego trajectory. Current approaches typically couple prediction and action through a predetermined backbone path or an iterative denoising loop, creating unnecessary rigidity. The core question addressed is: How much of a video diffusion model must be executed to make a reliable driving decision?

The authors identify two often-conflated axes:

Video Diffusion Timestep: Controls the latent noise level.
DiT Depth: Controls the number of transformer blocks evaluated.

Preliminary analysis on NAVSIM reveals that planning performance is largely insensitive to video-noise levels (timesteps) but exhibits significant depth-dependent variation. Furthermore, strong trajectories can be decoded from intermediate layers, and different depths solve partially overlapping sets of scenes, suggesting that a fixed-depth execution is suboptimal.

Methodology: Adaptive-WAM

The paper proposes Adaptive-WAM, a quality-aware, multi-exit planner built upon a Wan2.2-TI2V-5B backbone. The architecture decouples the video generation process from the planning decision process during inference.

Architecture and Inference Flow

Backbone: The model utilizes the Wan2.2 video DiT. During inference, it processes the current front-camera image and a text description (derived from map metadata, ego speed, and maneuver history) at a fixed video-noise index ( $s^* = 17$ ). It does not consume ground-truth future videos or perform full iterative denoising loops during deployment.
Multi-Exit Trajectory Heads: Six intermediate DiT blocks (layers 5, 9, 15, 18, 22, and 30) serve as exit points. At each exit $\ell$ , a projection layer converts the hidden representation $h_\ell$ into trajectory-conditioning tokens, which are fed into an independent ReCogDrive-style 5-step trajectory diffusion head. Each head generates a single candidate trajectory.
Quality-Guided Adaptive Routing:
- A lightweight trajectory-quality scorer (based on a fine-tuned DINOv2-Small encoder and an MLP) evaluates each decoded trajectory.
- The scorer predicts components of the planning metric (NC, DAC, DDC, TTC, EP, Comfort) rather than a strict total ordering, acknowledging that planning rewards are often saturated and tied.
- Decision Logic: The system maintains a pool of decoded trajectories. If the highest predicted quality score ( $\hat{q}$ ) among the attempted exits exceeds a threshold $\eta$ , inference terminates, and the best accumulated trajectory is returned. Otherwise, the system continues execution from the cached hidden state to the next deeper exit.
- If no exit meets the threshold, the system defaults to the highest-scoring trajectory from the final exit (layer 30).

Training Strategy

Video-Domain Adaptation: The Wan backbone is adapted using LoRA (Low-Rank Adaptation), while trajectory projections and heads are fully optimized. The objective combines the native video diffusion loss with trajectory diffusion losses for all exits.
Scorer Training: The scorer is fine-tuned on generated trajectories using soft-label binary cross-entropy against evaluator-provided component targets. It operates with gradient isolation from the actor to avoid altering the trajectory distribution.
Planner-Only RL: After imitation learning, the backbone and scorer are frozen. The trajectory heads are refined using DiffGRPO (a planner-only reinforcement learning method), treating the full 5-step denoising chain as a single action.

Key Contributions

Systematic Diagnosis: The paper demonstrates that driving planning performance is robust to video-noise levels but highly sensitive to DiT depth. It reveals that intermediate features provide a strong planning substrate and that different depths solve distinct, non-redundant sets of scenes.
Quality-Aware Multi-Exit Architecture: Adaptive-WAM introduces a mechanism to dynamically allocate backbone depth based on the quality of the currently decoded plan, avoiding the need for full video generation or fixed-depth execution.
Efficiency-Performance Trade-off: The method achieves state-of-the-art planning performance among compared world-model planners while significantly reducing inference latency by skipping unnecessary computation for "easy" scenes.

Experimental Results

NAVSIM v1 and v2

NAVSIM v1: The adaptive single-trajectory planner achieves 90.8 PDMS. A fixed-exit variant with 64 proposals (using block 22) reaches 92.6 PDMS, outperforming the DriveVA baseline.
NAVSIM v2: Adaptive-WAM achieves 89.9 EPDMS, yielding the best reported results among compared front-view video world-model planners.
Latency: On an A100 GPU, the adaptive planner averages 170 ms end-to-end latency. This is approximately 10% faster than a fixed block-15 planner (190 ms) and 47% faster than a fixed full-depth planner (320 ms).

Zero-Shot Transfer

Without target-domain fine-tuning, the model transfers to nuScenes, achieving an average L2 error of 0.88 m and a collision rate of 0.08%. This performance is competitive with DriveVA (0.84 m, 0.06%) but with substantially lower inference costs, as Adaptive-WAM avoids full video generation.

Ablation Studies

Backbone: Joint LoRA training of the Wan backbone is critical; freezing the backbone or using separate caching recovers only a fraction of the performance.
Scorer: DINOv2-Small is sufficient for the quality scorer; using the Wan backbone features for scoring offers negligible gains (0.03 points) at a much higher computational cost.
Depth: While block 15 is generally the strongest fixed exit, the adaptive policy outperforms it by dynamically selecting deeper layers for difficult scenes.

Significance and Claims

The paper claims that intermediate video-DiT features are robust planning representations that do not require full image synthesis to be effective. By coupling multi-depth trajectory heads with a tie-aware quality controller, Adaptive-WAM demonstrates a practical path from large generative world models to efficient, real-time driving policies.

The authors emphasize that their approach:

Avoids the iterative classifier-free denoising loop and VAE decoding required for future-video synthesis.
Dynamically allocates computation based on trajectory quality rather than committing to a single readout depth.
Achieves state-of-the-art results on NAVSIM while maintaining lower average latency than the strongest fixed-depth baselines.

The work suggests that for autonomous driving planning, the "world model" does not need to fully render the future; it only needs to compute enough latent features to decode a reliable trajectory.