⚡ electrical engineering

From Big Data to Fast Data: Towards High-Quality Datasets for Machine Learning Applications from Closed-Loop Data Collection

This paper introduces the "Fast Data" concept for automotive systems engineering, which shifts data selection and recording to the vehicle for real-time, context-aware closed-loop collection, thereby generating higher-quality, more relevant datasets for machine learning while reducing costs and irrelevant data compared to traditional Big Data and Smart Data approaches.

Original authors: Philipp Reis, Jacqueline Henle, Stefan Otten, Eric Sax

Published 2026-04-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Philipp Reis, Jacqueline Henle, Stefan Otten, Eric Sax

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to drive a car. To do this, you need to show it millions of examples of driving situations.

For a long time, the automotive industry used a method called "Big Data." Think of this like a security camera in a grocery store that records everything 24/7, every single second, whether someone is buying milk, walking down the aisle, or just standing there staring at the ceiling. Later, a team of humans has to watch hours of footage to find the one second where a kid dropped an ice cream cone (a rare, important event). This is expensive, slow, and full of useless junk.

Then came "Smart Data." This was like giving the security camera a brain that says, "Hey, if a kid drops ice cream, start recording!" But there's a catch: the camera still has to record everything first to figure out if a kid dropped ice cream, and then it sends that footage to a central office to be analyzed. It's better, but it's still slow and wasteful.

This paper introduces a new concept called "Fast Data."

The "Fast Data" Analogy: The Smart Sous-Chef

Imagine a busy restaurant kitchen.

Big Data is like a chef who chops every single vegetable in the warehouse, cooks every single dish possible, and then hopes a customer orders something.
Smart Data is like a chef who only cooks when they hear a customer order, but they still have to run to the back of the store to check the inventory before deciding what to cook.
Fast Data is like a super-intelligent Sous-Chef standing right next to the stove.

This Sous-Chef (the car) knows exactly what the Head Chef (the AI developer) needs.

If the Head Chef needs more practice with "burnt toast" (a rare, dangerous driving scenario), the Sous-Chef immediately grabs the bread and burns it right then and there.
If the Sous-Chef sees they already have 1,000 pictures of "perfect toast," they ignore the next 100 perfect toasts that walk by.
They don't wait for instructions from the back office. They make the decision in the moment, based on what they have already cooked and what they still need to cook.

The Core Problem: The "Long Tail" of Driving

Real-world driving is weird. Most of the time, cars drive on empty highways (boring, common stuff). But the dangerous stuff—like a child running into the street or a tire blowing out in the rain—happens very rarely.

If you just record everything (Big Data), 99% of your hard drive is full of boring highway driving. You waste money storing it and time sending it over the internet. You might miss the rare, dangerous moments because you were too busy recording the boring ones.

The Solution: A Closed-Loop System

The authors propose a system where the car acts as a smart filter with a memory.

The Goal: The car knows what kind of "recipe" (dataset) the AI needs. Maybe it needs more "rainy night" scenarios or more "sudden braking" examples.
The Check: As the car drives, it constantly asks: "Do I already have enough examples of this? Or is this a new, rare, or important moment that I should save?"
The Decision:
- If it's just another sunny day on the highway? Discard it. (Save space and money).
- If it's a weird, scary, or rare situation? Record it immediately!
The Loop: The car updates its own "shopping list" in real-time. If it just recorded a rainy night, it knows it doesn't need another one right now. If it hasn't seen a child on a bike in a while, it becomes hyper-aware of bikes.

Why This Matters

This approach changes the game in three ways:

Quality over Quantity: Instead of a mountain of useless data, you get a diamond of high-quality, relevant data.
Speed: The car learns and improves much faster because it's not waiting for humans to sort through terabytes of junk later.
Cost: You save a fortune on storage and internet bandwidth because you aren't sending boring data to the cloud.

The Bottom Line

The paper argues that to build safe, smart self-driving cars, we can't just be passive recorders. We need to be active curators.

Think of it like building a library.

Big Data is buying every book ever printed and hoping someone reads the right one.
Fast Data is a librarian who knows exactly what stories the community needs, goes out, finds only those specific stories, and puts them on the shelf, ignoring the rest.

By moving the decision-making from the "back office" to the "car itself," we can build better AI faster, cheaper, and safer.

1. Problem Statement

The development of advanced Machine Learning (ML) models for automotive systems (e.g., vision-language and multimodal models) relies heavily on the quality, relevance, and representativeness of training datasets. Current data collection paradigms face significant limitations:

Big Data Paradigm: Relies on exhaustive, offline recording of massive data volumes followed by centralized post-processing. This leads to high redundancy (overrepresentation of common scenarios), high storage/transmission costs, and delayed value extraction.
Smart Data Paradigm: Attempts to filter data based on estimated utility but still relies on centralized, post-hoc evaluation. This maintains long feedback loops between data generation and model improvement.
The Core Challenge: Real-world driving data is uneven; safety-critical and rare events (the "long tail") occur sparsely. Exhaustive recording is infeasible due to resource constraints, yet current "Smart Data" approaches cannot dynamically adapt collection strategies to the current state of the dataset being built. There is a need to shift from passive logging to active, closed-loop data collection that occurs at the data source (the vehicle) in real-time.

2. Methodology

The authors propose a conceptual and architectural framework for Fast Data, defined as a paradigm where data selection and recording decisions are made at the edge (the vehicle) in real-time, based on a closed feedback loop.

A. Dataset Quality Model

The paper establishes a formal definition of dataset quality for data collection, moving beyond general data quality to specific characteristics relevant to ML training:

Quality Dimensions: Based on ISO 5259, the authors distinguish between inherent (intrinsic to the data) and system-dependent qualities. They focus on inherent characteristics as they are directly influenced by the collection process.
Key Quality Characteristics (QC): Four specific QCs are identified as critical for collection strategies:
1. Relevance: Suitability for the target problem.
2. Balance: Distribution of entities/attributes (avoiding over/under-representation).
3. Diversity: Degree of difference between samples at the feature level (preventing overfitting).
4. Similarity: Degree of resemblance between data points (high similarity can lead to redundancy).
Metrics: These characteristics are quantified via quality metrics (e.g., Vendi Score for diversity, k-nearest neighbors for similarity).

B. Analysis of Generation Methods

The paper evaluates four data sources:

Real-World Driving: High accuracy and ground truth but expensive and uncontrolled.
Simulation: High controllability and scalability but limited realism and fidelity.
Generative AI: Good for augmentation but struggles with mode collapse and realism.
Open Datasets: Good for benchmarking but suffer from data aging and bias.
Conclusion: No single source is sufficient. Real-world data remains indispensable for ground truth, but it must be collected selectively to complement synthetic data.

C. Taxonomy of Collection Strategies

The authors categorize existing strategies and identify their limitations:

Data-Intrinsic Triggers: Decisions based solely on the current data point (e.g., rule-based thresholds, semantic detection, error-based triggers). These are fast but dataset-agnostic; they cannot assess redundancy or global diversity.
Dataset-Specific Triggers: Decisions based on the relationship between a new sample and an existing dataset (e.g., anomaly detection).
- Limitation: Most current implementations are static (trained offline on historical data). They do not adapt to the evolving state of the dataset during the collection process.

D. The Closed-Loop Fast Data Framework

The core methodology proposes a shift to a Closed-Loop system where collection is a feedback-driven process:

State Estimation: The system maintains a compact, online estimate ( $\hat{\theta}$ ) of the current dataset's state (e.g., current diversity, balance).
Target State: A target state ( $\theta^*$ ) defines the desired dataset properties (e.g., "increase diversity in corner cases").
Value Evaluation: For every incoming data point $x$ , the system calculates a value $\psi$ representing how much that sample reduces the gap between the current state $\hat{\theta}$ and the target $\theta^*$ .
Control Action: A policy function $F$ decides whether to retain ( $u=1$ ) or discard ( $u=0$ ) the sample based on $\psi$ .
Recursive Update: The decision updates the dataset state estimate, closing the loop.

3. Key Contributions

Dataset Quality Model for Fast Data: Derivation of a quality model specifically for data collection, identifying Relevance, Balance, Diversity, and Similarity as the primary characteristics to optimize.
Analysis of Generation Pathways: A comparative analysis showing that while synthetic and open data are useful, selective real-world collection remains indispensable for ground truth and validation, necessitating a Fast Data approach.
Taxonomy of Triggers: A distinction between Data-Intrinsic (local properties) and Dataset-Specific (global properties) triggers, highlighting that current methods are often only "dataset-conditioned" (static) rather than truly "dataset-specific" (adaptive).
Formal Closed-Loop Formulation: The mathematical formulation of Fast Data collection as a discrete-time feedback control system. This defines the architectural requirements for a system that is state-aware (knows the current dataset composition) and requirement-aware (knows the target goals).

4. Results and Findings

Paradigm Shift: The paper demonstrates that moving from "Big Data" (volume-driven) to "Fast Data" (purpose-driven) requires a structural change in control logic, not just a temporal one (moving decisions to the edge).
Limitations of Current Approaches: Existing "Smart Data" approaches fail to optimize for dataset-level objectives (like diversity) because they lack a real-time, adaptive view of the dataset state. They often record data that is redundant relative to what has already been collected.
Feasibility of Closed-Loop: The proposed formal model proves that it is theoretically possible to treat dataset generation as a control problem where the "plant" is the dataset formation process, and the "controller" is the selection policy.

5. Significance

Resource Efficiency: By recording only high-value data at the source, this approach drastically reduces storage, transmission bandwidth, and processing costs associated with irrelevant data.
Improved ML Performance: Datasets generated via this method will have higher information density, better coverage of critical/long-tail scenarios, and improved balance, directly leading to more robust and safer ML models for autonomous driving.
Scalability: Provides a structured foundation for scalable ML development in automotive engineering, enabling continuous improvement of models through a tight feedback loop between data collection and model requirements.
Future Direction: The paper sets the stage for implementing these architectures, moving the industry from passive data logging to active, intelligent data curation at the edge.

In summary, the paper argues that the future of automotive ML lies not in collecting more data, but in collecting the right data through a closed-loop, state-aware, and adaptive Fast Data paradigm.