🤖 machine learning

TailBooster: A Dual-Layer Generative Framework for Extreme Value Augmentation with Operational Validity Enforcement

TailBooster is a dual-layer generative framework that combines statistical tail extraction with deep learning-based operational validity enforcement to synthesize realistic extreme air transport events, significantly improving the prediction accuracy of rare delays and abnormal air times compared to conventional data augmentation methods.

Original authors: Karim Aly, Alexei Sharpanskykh, Jacco Hoekstra

Published 2026-08-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Karim Aly, Alexei Sharpanskykh, Jacco Hoekstra

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to predict the weather. You show it thousands of sunny days and a few rainy ones. The robot gets really good at predicting sunshine, but when a massive, rare hurricane hits, it has no idea what to do because it's never seen anything like it before. This is a common problem in the world of Artificial Intelligence, specifically in a field called "generative modeling." These are smart computer programs designed to learn patterns from data and create new, fake data that looks just like the real thing. Scientists use them to fill in gaps when they don't have enough real records, like predicting rare traffic jams or extreme flight delays. But here's the catch: these robots are terrible at the "extremes." They tend to ignore the rare, wild outliers and just copy the boring, average stuff. Even worse, sometimes they get creative in the wrong way, inventing impossible scenarios—like a plane flying from New York to London in five minutes. This paper tackles that exact problem: how do we teach AI to understand the rare, crazy events without letting it make up nonsense?

The researchers behind this study, working at Delft University of Technology, have built a new system called TailBooster. Think of it as a two-step quality control team for a factory that makes fake flight records. Their goal was to fix the "tail" of the data distribution—the part that holds the rare, extreme events like severe delays or unusually long flight times. Standard AI models usually ignore these tails, focusing only on the "middle" of the data where most flights happen. TailBooster changes the game by using a dual-layer approach to force the AI to pay attention to the extremes while simultaneously acting as a strict editor to ensure the new data makes sense in the real world.

Here is how the magic happens. First, the system acts like a detective with a magnifying glass. It scans the historical flight records and uses a statistical rule (called the Interquartile Range, or IQR) to pull out the "weird" flights—the ones with the longest delays or strangest flight times. It sets these rare records aside to create a special training camp. The AI model is then trained twice: once on all the normal flights to learn the basics, and again on just those rare, extreme flights. This ensures the AI learns what a "disaster" looks like, not just what a "normal day" looks like.

But learning about extremes isn't enough; the AI could still invent impossible scenarios, like a plane flying 5,000 miles in 10 minutes. That's where the second layer comes in. This is the "operational validity" check. Imagine a seasoned flight instructor who has flown thousands of routes. This instructor knows that a flight from New York to London must take a certain amount of time based on the distance. If the AI generates a record that breaks this rule, the instructor throws it in the trash. In TailBooster, this instructor is a smart computer program (an autoencoder) that has studied the real historical data to learn the "rules of the road" without needing a human to write down every single rule. It filters out any fake record that violates these learned patterns.

The results of this process are impressive. The team tested TailBooster on over 60,000 U.S. domestic flight records. They compared their new, cleaned-up fake data against the old, messy kind. The findings suggest that TailBooster is a significant upgrade. When they used this new data to train other AI models to predict extreme delays, the errors dropped dramatically. For predicting extreme flight times, the error rate fell by about 47–49%. For predicting extreme arrival delays, the error dropped by 29–57%. This means the models trained on TailBooster's data were much better at guessing what happens during a crisis.

The paper also explicitly rules out some common shortcuts. For instance, they found that simply telling the AI to "make more rare events" by forcing it to focus on a specific category doesn't work if the AI hasn't actually seen enough examples of those events to learn the pattern. They also showed that trying to hard-code physical rules (like writing a math formula for how fast a plane can fly) is too rigid and hard to apply to different airports. Instead, their data-driven approach of letting the computer learn the rules from history works much better.

In short, TailBooster suggests that by combining a statistical spotlight on the rare events with a smart, data-driven filter to remove the impossible ones, we can create synthetic data that is both realistic and useful. This is a big deal for anyone trying to predict the unpredictable, from airlines trying to manage delays to researchers studying rare events in other fields. The authors are careful to note that while their method works well on this specific dataset of New York flights from January 2023, the approach is designed to be flexible enough to work in other areas where extreme events matter and where we don't have all the physical rules written down yet. It's a step toward making AI not just a good student of the average, but a reliable expert on the extraordinary.

Technical Summary: TailBooster

1. Problem Statement

Air transportation systems face significant operational, economic, and safety risks from extreme events, such as severe arrival delays and abnormal air times. These events are inherently rare in historical records, creating a fundamental learning problem for machine learning models: the distributional tails of continuous operational metrics are sparsely populated, providing insufficient training signal for regression tasks. Consequently, models generalize well under nominal conditions but fail precisely where accurate prediction is most critical—at the extremes.

Existing synthetic data augmentation approaches fail to address two simultaneous limitations for mixed-type tabular records:

Tail Under-representation: Conventional generative models (GANs, VAEs, Diffusion) prioritize high-density regions of the data distribution, systematically under-representing distributional tails regardless of how well they approximate the overall distribution.
Operational Invalidity: Standard generative models offer no guarantee of preserving operational correlations between features. They frequently produce operationally infeasible instances (e.g., short air times paired with long flight distances) that cannot be used to train downstream decision-making systems.

While Extreme Value Theory (EVT) offers a statistical foundation for tail modeling, existing EVT-augmented generative models are designed for continuous, homogeneous feature spaces and do not extend to the mixed-type tabular data characteristic of aviation. Furthermore, domain-constrained approaches often rely on hand-crafted symbolic rules that require specific domain knowledge and are not easily transferable.

2. Methodology: TailBooster Framework

The paper proposes TailBooster, a dual-layer generative framework that combines generative modeling with two anomaly detection layers to produce synthetic flight records with improved extreme representativeness and operational validity. The framework is fully data-driven and model-agnostic regarding the generative component.

Architecture and Pipeline

The pipeline processes historical flight records through the following stages:

Input: The framework accepts a full historical dataset ( $D$ ), a user-defined list of target features ( $T$ ) whose tails require augmentation (e.g., "Air Time," "Arrival Delay"), and a list of operationally correlated features ( $X_c$ ) used to learn feasibility constraints.
Statistical Layer (1st Anomaly Detection):
- Extreme Subset Extraction: Using the Interquartile Range (IQR) method, the framework isolates extreme subsets ( $E^{(j)}$ ) for each target feature $f_j \in T$ . Records falling outside the Tukey fences ( $Q_1 - 1.5 \cdot IQR$ or $Q_3 + 1.5 \cdot IQR$ ) are extracted.
- Purpose: This provides a tail-concentrated training signal to dedicated generative models, addressing the scarcity of extreme examples in the full dataset.
Generative Stage:
- Multiple generative models are trained: one on the full historical dataset ( $D_0$ ) to preserve nominal patterns, and one on each extreme subset ( $E^{(j)}$ ) to amplify tail signals.
- Model Choice: The authors utilize a Tabular Variational Autoencoder (TVAE) for its training stability and lower computational cost compared to diffusion models, though the framework is agnostic to the specific generative architecture.
- Sampling: Synthetic records are sampled from all trained models.
Relational Validity Filter:
- A rejection sampling step removes synthetic records where the origin–destination airport pair does not exist in the historical data. This ensures all records correspond to operationally plausible routes.
Deep Learning Layer (2nd Anomaly Detection):
- Operational Cleaning: Pre-trained autoencoders are trained on the operationally correlated features ( $X_c$ ) from the real historical data to learn the empirical operational envelope (e.g., the relationship between distance and air time).
- Filtering: Synthetic records are passed through these autoencoders. Records with reconstruction errors exceeding a threshold (set at the 99th percentile of errors on real data) are discarded as operationally invalid. This step enforces validity without requiring hand-crafted physical rules.

Output Datasets

The framework produces three distinct datasets for evaluation:

Naïve Synthetic ( $S_{naïve}$ ): Validity-filtered output from the model trained on the full dataset, without extreme augmentation or operational cleaning.
Augmented Synthetic ( $S_{aug}$ ): Merged, cleaned synthetic records from all models (full dataset + extreme subsets).
Augmented Real ( $D_{aug}$ ): The original real historical data enriched with the cleaned synthetic extreme records.

3. Key Contributions

The research makes four primary contributions:

Dual-Layer Framework: A novel architecture for mixed-type tabular data that integrates deep generative models with statistical (IQR) and deep learning (autoencoder) anomaly detection to simultaneously address tail under-representation and enforce operational validity in a fully data-driven manner.
Data-Driven Operational Cleaning: A demonstration that an autoencoder-based cleaning mechanism markedly improves the operational validity of synthetic records compared to conventional generation, eliminating the need for domain-specific symbolic rules.
Predictive Utility Evidence: Empirical evidence that targeted extreme value augmentation consistently improves the predictability of extreme events across six regression models spanning three distinct families (tree-based ensembles, kernel-based, and instance-based).
Comparative Evaluation: A comprehensive assessment showing measurable quality gains for both practitioners with access to real data (via augmentation) and those without (via high-quality synthetic data), outperforming conventionally generated synthetic data.

4. Experimental Results

The framework was evaluated on U.S. domestic flight records (New York State, Jan 2023) across five dimensions: diversity, statistical similarity, fidelity, operational validity, and utility.

Preservation Checks:
- Diversity: The framework maintained coverage of all historical clusters in PCA and t-SNE projections.
- Statistical Similarity: The Augmented Synthetic dataset achieved an overall similarity of 86.26% (vs. 79.98% for Naïve Synthetic), with significant gains in bivariate similarity due to operational cleaning.
- Fidelity: The Augmented Synthetic dataset was harder to distinguish from real data than the Naïve Synthetic baseline, particularly within extreme subsets (discriminability dropped from ~0.90 to ~0.55 for extreme air times). Memorization checks (Distance to Closest Record) confirmed no significant overfitting.
Primary Improvement Targets:
- Operational Validity: Visual analysis of pairwise correlations (e.g., Air Time vs. Distance) showed that the Naïve Synthetic dataset contained implausible records (e.g., short air times for long distances). The TailBooster pipeline successfully removed these violations and covered the extreme tails observed in real data.
- Utility (Regression Performance): Training on the Augmented Synthetic dataset reduced Mean Absolute Error (MAE) significantly compared to Naïve Synthetic:
  - Extreme Air Time: 47–49% reduction in MAE across six regression algorithms.
  - Extreme Arrival Delay: 29–57% reduction in MAE.
- Augmented Real: Training on real data enriched with synthetic extremes ( $D_{aug}$ ) consistently outperformed training on real data alone, reducing MAE for extreme air time from 6.34–9.88 min to 2.57–8.59 min.

5. Significance and Claims

The paper claims that TailBooster addresses a critical gap in the literature where no existing approach simultaneously targets tail under-representation in mixed-type tabular records and enforces operational validity without relying on hand-crafted domain rules.

For Practitioners with Data: The framework offers a method to enrich existing historical datasets with operationally valid synthetic extremes, directly improving the predictability of rare, high-impact events like severe delays.
For Practitioners without Data: It provides a mechanism to generate high-quality synthetic data that preserves both nominal patterns and extreme-value structures, enabling model development and benchmarking where access to real operational records is restricted.
Generalizability: Being fully data-driven and model-agnostic, the framework is transferable to any domain where extreme-event prediction is critical and specific physical governing equations or symbolic rules for validity are unavailable.

The authors note limitations, including the evaluation on a single temporal and geographic dataset, the visual rather than quantitative assessment of operational validity, and the modest size of extreme subsets used for training. They suggest future work could explore adaptive thresholds, lightweight resampling for extremely sparse tails, and the integration of physics-informed mechanisms where governing laws are known.