🤖 AI

Driving on Registers

The paper introduces DrivoR, a pure-transformer architecture for end-to-end autonomous driving that utilizes camera-aware register tokens to compress multi-camera features into a compact representation, enabling efficient trajectory generation and scoring with interpretable sub-scores while outperforming or matching state-of-the-art baselines across multiple benchmarks.

Original authors: Ellington Kirby, Alexandre Boulch, Yihong Xu, Yuan Yin, Gilles Puy, Éloi Zablocki, Andrei Bursuc, Spyros Gidaris, Renaud Marlet, Florent Bartoccioni, Anh-Quan Cao, Nermin Samet, Tuan-Hung VU, Matthieu

Published 2026-02-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ellington Kirby, Alexandre Boulch, Yihong Xu, Yuan Yin, Gilles Puy, Éloi Zablocki, Andrei Bursuc, Spyros Gidaris, Renaud Marlet, Florent Bartoccioni, Anh-Quan Cao, Nermin Samet, Tuan-Hung VU, Matthieu Cord

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. Traditionally, you might give the robot a massive instruction manual with thousands of pages, telling it exactly what to do in every possible situation. This paper introduces a new way to teach the robot, called DrivoR, which is much smarter, faster, and simpler.

Here is the breakdown of how it works, using everyday analogies:

1. The Problem: Too Much Information

Imagine you are driving and looking through six different windows (cameras). A standard computer vision system tries to look at every single pixel in every window, creating a mountain of data. It's like trying to read a library of books while driving; the computer gets overwhelmed trying to process all that information before it can decide whether to turn or stop. This makes the system slow and expensive to run.

2. The Solution: The "Register" Note-Takers

The authors of DrivoR realized the robot doesn't need to read the whole library. It just needs a few key notes.

They introduced something called "Register Tokens." Think of these as a small team of specialized note-takers sitting in the passenger seat for each camera.

Instead of the computer trying to process the whole image, these note-takers scan the view and write down only the most important details (like "car ahead," "red light," or "empty road").
They compress a huge, messy image into a tiny, neat summary.
The Result: The computer now has to read just a few sentences instead of a whole book. This makes the system incredibly fast and efficient without losing the ability to see danger.

3. The Two-Step Decision Process

Once the note-takers have summarized the scene, the robot has to decide what to do. DrivoR splits this job into two separate teams, like a Coach and a Referee.

The Coach (Trajectory Generator): This team looks at the summary and says, "Here are 100 different ways we could drive for the next few seconds." They generate many possible paths (trajectories) quickly.
The Referee (Scoring System): This team looks at those 100 paths and grades them. But here is the cool part: the Referee doesn't just give one score (like "Good" or "Bad"). It gives a report card with specific grades:
- Safety: "How likely are we to crash?"
- Comfort: "Will the passengers feel sick?"
- Efficiency: "How fast are we getting there?"

4. Driving with a "Personality"

Because the Referee gives separate grades for safety, comfort, and speed, you can change the robot's personality without retraining it.

The "Safe" Mode: You tell the computer, "I care 100% about safety and comfort." The robot picks the path with the highest safety score, even if it's a bit slower.
The "Sporty" Mode: You tell the computer, "I want to get there fast." The robot picks the path that maximizes progress, even if it's a bit bumpy.

It's like having a car that can instantly switch from a "Grandma driver" to a "Racing driver" just by turning a dial.

5. The Results

The paper tested this system on some very tough driving simulations (like a video game that is harder than real life).

Performance: DrivoR performed as well as, or better than, the most complex systems currently available.
Speed: Because it uses those "note-takers" to compress the data, it runs 3 times faster than similar systems.
Simplicity: It doesn't need a massive dictionary of pre-written driving moves or complex 3D maps. It learns directly from the camera images and the "note-takers."

Summary

DrivoR is a new way to build self-driving cars that is like hiring a team of smart note-takers to summarize the road, a Coach to suggest many driving options, and a Referee to grade those options based on what you value (safety vs. speed). It proves you don't need a super-computer to drive well; you just need a smart, efficient way to process information.

Technical Summary: DrivoR – Driving on Registers

Problem Statement

End-to-end (E2E) autonomous driving aims to map raw sensor data directly to driving decisions, reducing labeling costs and avoiding intermediate modular annotations. While trajectory-proposal methods have shown strong performance by generating multiple candidate paths and selecting the best one, they face a significant computational bottleneck. Current state-of-the-art approaches often rely on heavy backbones (e.g., large Vision Transformers or CNNs) that output thousands of tokens per frame. Processing these tokens for hundreds of trajectory candidates creates a computational burden that scales poorly with resolution and sensor count. Furthermore, existing solutions often rely on uniform feature pooling, which treats all camera inputs as equally informative and enforces rigid resolution requirements, potentially discarding critical planning context.

Methodology: DrivoR Architecture

The authors propose DrivoR, a simple, efficient, pure-transformer architecture designed for E2E planning. The system avoids complex intermediate representations (like Bird's Eye View projections) and large pre-computed trajectory dictionaries. Instead, it utilizes a three-stage transformer pipeline:

1. Perception Encoder with Camera-Aware Registers

The core innovation lies in the perception encoder, which replaces uniform pooling with camera-aware register tokens.

Mechanism: For each of the $N$ input cameras, the model introduces a fixed set of $R$ learnable register tokens. These are concatenated with standard patch tokens and fed into a pretrained Vision Transformer (ViT) backbone (specifically DINOv2 ViT-S).
Compression: The ViT compresses the high-dimensional visual features into these $R$ registers per camera. The output is a compact set of $N \times R$ "scene tokens."
Advantage: This approach drastically reduces the sequence length for downstream processing (from thousands of tokens to a few dozen) while preserving planning-relevant context. The registers are camera-specific, allowing the model to differentiate between front, side, and rear views.
Training: The backbone is fine-tuned using Low-Rank Adaptation (LoRA) to learn the vision-to-register compression efficiently, keeping the parameter count low.

2. Trajectory Decoder

The trajectory decoder generates candidate paths using a standard transformer decoder architecture.

Input: It takes a set of learnable trajectory queries, augmented with ego-state information (pose, velocity, acceleration, commands).
Attention: Queries attend to the compressed "scene tokens" via cross-attention.
Output: The decoder produces a set of candidate trajectories, each represented as a sequence of $(x, y, \theta)$ poses in the ego-agent's local frame.
Loss: Training utilizes a Winner-Takes-All (WTA) or minimum-over- $N$ (MoN) loss, supervising only the closest predicted trajectory to the ground truth, encouraging diversity in the proposal set.

3. Scoring Decoder (Disentangled)

A distinct scoring module evaluates the quality of the generated trajectories.

Disentanglement: Crucially, the decoded trajectories are re-embedded into a new feature space and detached from the gradient graph before entering the scoring decoder. This prevents the trajectory generation process from being influenced by the immediate quality of the scorer during training, enforcing a separation between generation and evaluation.
Sub-scores: Instead of a single scalar score, the model predicts six interpretable sub-scores corresponding to the Predictive Driver Model Score (PDMS) components (e.g., safety, comfort, efficiency, progress).
Behavior Conditioning: At inference, these sub-scores can be weighted differently (via a set of $\lambda$ weights) to modulate driving behavior (e.g., prioritizing safety over progress) without retraining the model.

Key Contributions

Simple Transformer Architecture: A pure-transformer E2E planner that eliminates the need for intermediate BEV representations or large trajectory vocabularies.
Register-Based Token Compression: The first work to repurpose ViT register tokens specifically for compressing visual features in E2E planning. This achieves a compact scene representation that retains critical context while reducing computational load by orders of magnitude compared to full feature maps.
Disentangled Scoring: A modular design that separates trajectory generation from scoring, enabling the learning of interpretable sub-scores and allowing for behavior-conditioned driving policies at inference time.

Experimental Results

DrivoR was evaluated on three major benchmarks: NAVSIM-v1, NAVSIM-v2, and the photorealistic closed-loop HUGSIM.

NAVSIM-v1: DrivoR outperforms all existing camera-only methods, achieving a PDMS of 93.7 (using trainval data) and 94.0 with additional synthetic data, nearing human-level performance (94.8). It significantly outperforms strong baselines like RAP and UniAD.
NAVSIM-v2: On the challenging navhard-two-stage split, DrivoR achieves a new state-of-the-art EPDMS of 54.6, surpassing models like GTRS and ZTRS. Notably, it achieves this with a much smaller backbone (ViT-S) and fewer parameters (~40M) compared to competitors.
HUGSIM: In a zero-shot evaluation on the closed-loop HUGSIM benchmark, DrivoR (trained only on NAVSIM-v1) achieved the highest Road Completion (49.8%) and HD-Score (35.7) among reported baselines, demonstrating strong generalization to photorealistic 3D environments.
Efficiency: The register-based compression yields a 3x improvement in throughput (110ms vs. 400ms per forward pass on an A100 GPU) and a 3x reduction in GFLOPS and peak memory usage compared to a ViT-L baseline, making real-time deployment more feasible.

Significance and Claims

The paper posits that a pure-transformer architecture, when combined with targeted token compression via registers, is sufficient for accurate, efficient, and adaptive end-to-end driving. The authors highlight that:

Uniform pooling is suboptimal; camera-aware registers allow the model to learn which views are most informative (e.g., front cameras for planning, rear cameras for specific safety checks).
Disentangling scoring from generation improves performance and enables controllable driving behaviors.
High performance does not require massive trajectory dictionaries or complex intermediate 3D representations; a simple, well-structured transformer pipeline can match or exceed the performance of more complex systems while being significantly more efficient.

The work suggests a shift towards lightweight, register-based compression strategies for real-time autonomous driving systems, demonstrating that "less is more" when the compression is learned and context-aware.