🤖 machine learning

Test-Time Coverage: Test-Conditioned Data Curation for Deployment-Aware Learning

This paper introduces TTCov, a test-conditioned data curation method that constructs a Knowledge Atlas from deployment samples to select training data with optimized coverage and distribution matching, thereby enhancing autonomous driving performance without requiring model updates at inference time.

Original authors: Nadine Chang, Maying Shen, Shizhe Diao, Jialiang Wang, Jingde Chen, Thomas Breuel, Pavlo Molchanov, Rafid Mahmood, Jose M. Alvarez

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nadine Chang, Maying Shen, Shizhe Diao, Jialiang Wang, Jingde Chen, Thomas Breuel, Pavlo Molchanov, Rafid Mahmood, Jose M. Alvarez

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. You have a massive library of video footage from millions of miles of driving—rainy nights in Boston, sunny highways in Las Vegas, crowded streets in Singapore. If you tried to feed every single second of that footage into the robot's brain, it would take forever and cost a fortune. So, you have to be picky. You need to curate a special "training diet" of videos that will make the robot smart enough to handle the real world.

For a long time, scientists picked these videos based on how "different" or "interesting" they looked. It was like a chef picking ingredients just because they were rare or colorful, without checking if they were actually what the customer wanted to eat. The problem is, the real world is messy and changes constantly. A robot trained on generic "interesting" videos might get confused when it hits a specific, tricky situation it never saw before. The big question in this corner of artificial intelligence is: How do we choose the right training data so the robot is ready for the specific roads it will actually drive on, without needing to relearn everything every time it moves to a new city?

This is where a new method called TTCov (Test-Time Coverage) comes in, and it's like giving the robot a cheat sheet before the test even starts. Instead of trying to fix the robot's brain while it's driving (which is slow and risky), TTCov fixes the training menu beforehand.

Here's how it works, using a playful analogy: Imagine you are preparing for a driving test in a new city. Instead of just memorizing random traffic rules, you first look at the city's map and the specific test route you'll be taking. You realize, "Okay, this test has a lot of school zones, wet roads, and pedestrians crossing at night."

TTCov does exactly this, but with the help of a super-smart AI librarian (a Large Language Model).

The Atlas (The Cheat Sheet): First, the system builds a "Task Atlas." Think of this as a giant, organized list of every single thing that could happen on the road, written in simple, atomic sentences like "Pedestrian crosses street" or "Car stops at red light." It starts with general knowledge but then looks at the actual test videos (the "deployment" data) to add any new, weird scenarios it finds, like "School bus stops in front of a puddle."
The K-Atlas (The Recipe): Next, it counts how often those specific things happen in the test videos. If the test route has 50% school buses and 10% rainy roads, the "Knowledge Atlas" (K-Atlas) becomes a recipe that says, "Your training diet must have 50% school bus videos and 10% rainy road videos."
The Selection (The Shopping Trip): Finally, TTCov goes through the massive pool of available training videos and picks the perfect subset that matches that recipe. It uses a greedy algorithm (a step-by-step picker) to grab videos that fill in the gaps, ensuring the training set looks exactly like the test set in terms of what's happening, not just what it looks like.

The paper tests this on autonomous driving, specifically using a dataset called Navhard, which is full of tricky, difficult driving scenarios. They compared TTCov against other methods that just pick "diverse" or "random" videos. The results suggest that TTCov is a game-changer. When they trained a driving model using the TTCov-selected data, the robot drove significantly better on the difficult test tracks. In fact, with a specific budget of data, TTCov performed almost as well as a dataset that was manually curated by humans with perfect knowledge (an "oracle"), and it even beat that human-curated dataset when they had more data to work with.

One of the coolest parts is how it handles moving to new cities. If you take the robot from Pittsburgh to Las Vegas, you don't have to start over. TTCov just looks at the new city's test videos, updates its "recipe" (the K-Atlas) to include Las Vegas-specific things (like desert heat or different traffic patterns), and picks the best new videos to add to the mix. The robot learns the new city without forgetting the old one.

The authors show that by matching the content of the training data to the content of the real-world test, you get a much safer and more capable driver. They measured this using a score called EPDMS, where TTCov consistently scored higher than other methods across different data sizes. While the method relies on having some test data available beforehand (you can't predict the future perfectly), the paper suggests that this approach of "curating for the test" is a smarter, more efficient way to build AI than just throwing random data at it. It's a reminder that in AI, sometimes the secret to a better brain isn't more data, but the right data.

Technical Summary: Test-Time Coverage (TTCov)

Problem Statement

Deployed AI systems, particularly in safety-critical physical domains like autonomous driving (AD), are typically trained on broad candidate data pools. Standard data curation methods select training subsets based on training-side criteria (e.g., diversity, uncertainty, redundancy) rather than directly optimizing for the deployment test distribution. This creates a potential mismatch where the curated training data does not reflect the specific conditions the model will encounter in the real world.

While transductive learning and test-time adaptation (TTA) techniques attempt to bridge this gap by using unlabeled test data to update model weights during inference, these approaches introduce significant inference latency and complicate model validation in real-time, safety-critical scenarios. Furthermore, they scale poorly with deployment volume as they require per-sample model tuning. The paper posits a need for a method that adapts the training distribution before training begins, leveraging test-side information without modifying model weights at inference.

Methodology: TTCov

The authors introduce TTCov (Test-Time Coverage), a data-level, test-conditioned curation framework. Instead of updating model weights, TTCov curates a training dataset that aligns with the deployment distribution by decomposing the problem into two components: Coverage (what deployment-relevant content to include) and Distribution (how frequently that content should appear).

1. The Atlas (Coverage Representation)

To represent coverage, TTCov constructs an interpretable Task Atlas, a collection of Atomic Propositions (APs) describing deployment-relevant concepts (scenarios, agents, conditions, behaviors).

Seeding: The Atlas is initially seeded with open-world task knowledge generated by Large Language Models (LLMs) to form a structured ontology of driving conditions.
Expansion: The Atlas is expanded using unlabeled deployment-side data (e.g., driving logs). A Multimodal LLM (MLLM) extracts dense captions from test videos, which are then converted into APs.
Unification: A three-stage process refines these APs:
1. Distillation: Complex descriptions are converted into single-clause declarative sentences.
2. Proposition Conversion: Sentences are transformed into subject-verb-object (SVO) or subject-attribute atomic units.
3. Nuisance Filtering: Non-essential modifiers (colors, specific sizes, brand names, specific street names) are removed to ensure the APs capture generalizable concepts (e.g., "pedestrian crosses crosswalk" rather than "pedestrian in a red shirt crosses Broadway").
4. Deduplication: Semantically similar phrases are clustered and unified using LLM verification to ensure a consistent vocabulary.

2. The K-Atlas (Distribution Representation)

To represent distribution, TTCov instantiates the matched deployment APs with their frequencies, creating a Knowledge Atlas (K-Atlas).

The K-Atlas is a target distribution vector $p^\star$ , where each element represents the importance frequency of a specific AP in the deployment test set.
This operationalizes the deployment distribution as a concrete curation target.

3. Greedy Selection Optimization

The goal is to select a budgeted training subset $S$ (size $B$ ) from a candidate pool $C$ such that the subset's AP distribution $\hat{p}_S$ minimizes the Kullback-Leibler (KL) divergence to the target K-Atlas $p^\star$ :
$S^\star := \arg \min_{S \subset C, |S|=B} D_{KL}(p^\star \parallel \hat{p}_S)$
Since this is a combinatorial optimization problem, TTCov employs a greedy iterative algorithm (Algorithm 1). In each step, it selects the sample from the remaining candidate pool that maximally reduces the current KL gap between the accumulated subset and the target distribution.

Key Contributions

TTCov Framework: A data-level method for test-conditioned curation that adapts the training dataset to the unlabeled deployment distribution before model training, avoiding the latency and complexity of online model adaptation.
Atlas and K-Atlas: The introduction of an interpretable coverage representation (Atlas) and a frequency-weighted target distribution (K-Atlas) specifically designed for deployment-conditioned curation.
Optimization Formulation: Formulating data curation as a budgeted K-Atlas matching problem and solving it via a greedy selection procedure over candidate samples.
Monotonic Atlas Evolution: A mechanism for adapting to changing deployment conditions (e.g., city-to-city expansion) by updating the K-Atlas for new domains while retaining previously selected data, avoiding catastrophic forgetting.
Empirical Validation: Comprehensive evaluation using coverage metrics, distribution matching, and downstream end-to-end (E2E) autonomous driving performance against state-of-the-art baselines.

Experimental Results

The authors evaluated TTCov on the Navhard benchmark (a challenging subset of the NAVSIM framework) using the Latent Transfuser (LTF) planner.

Performance: TTCov consistently outperformed baselines, including Coreset (diversity-based) and SSE (semantic selection), across various data budgets (0.5x to 1.5x the baseline).
- At a 1x budget, TTCov achieved performance parity with the manually curated Navtrain oracle (24.42 vs 24.70 EPDMS) without requiring human annotation.
- At budgets exceeding the oracle size, TTCov surpassed both the oracle and all other baselines (26.40 EPDMS at 1.5x).
Distribution Matching: TTCov achieved the lowest KL divergence to the K-Atlas target compared to all baselines, demonstrating superior alignment between the selected training data and the test distribution.
City-to-City Adaptation: In a multi-round domain expansion experiment (adding cities sequentially), TTCov significantly outperformed baselines in both the new target city and the original city, proving its ability to adapt to new domains without re-curating data for previous cities or suffering from catastrophic forgetting.
Coverage: TTCov selected a significantly higher volume of training points with embeddings close to the test distribution (measured by Nearest Neighbor distance and MMD) compared to Coreset and SSE.

Significance and Claims

The paper claims that TTCov represents a shift from reactive model patching to proactive, test-driven AI development. By leveraging an interpretable Atlas of atomic propositions, TTCov enables:

Safety-Aware Curation: The ability to explicitly target safety-critical and rare scenarios (e.g., pedestrians, school buses, specific traffic light interactions) that are often missed by generic diversity metrics.
Scalability: The framework scales to evolving deployment conditions (e.g., new cities) without the computational overhead of per-sample model adaptation.
Interpretability: The use of atomic propositions provides a transparent view of what concepts the training data covers, offering insights into the model's potential failure modes.

The authors note that while TTCov is tied to the existence of a test set and the knowledge ceiling of the underlying LLMs, it offers a robust, scalable path toward aligning training data with real-world deployment distributions in physical AI systems.