Generative Augmentation of Imbalanced Flight Records for Flight Diversion Prediction: A Multi-objective Optimisation Framework

1. Problem Statement

Flight diversions are rare, high-impact events in aviation (127 diversions out of ~61,000 flights in the dataset) that pose significant safety, economic, and operational challenges. Predicting these events using machine learning is hindered by severe class imbalance, where the minority class (diverted flights) is vastly outnumbered by the majority class (normal flights). This imbalance biases models toward the majority class, resulting in poor detection of actual diversions.

Furthermore, standard predictive models struggle because publicly available flight data often lacks features strongly correlated with diversions (e.g., specific weather conditions, crew status, or maintenance issues). Traditional oversampling techniques like SMOTE often fail because they rely on simple interpolation, generating unrealistic data points that lead to overfitting. The core problem is how to generate high-fidelity synthetic data for the minority class that preserves complex statistical dependencies and operational constraints without introducing noise or unrealistic patterns.

2. Methodology

The authors propose a comprehensive framework involving data preprocessing, generative modeling, multi-objective hyperparameter optimization, and a six-stage evaluation protocol.

A. Data and Preprocessing

• Dataset: Used the "TranStats Database" (BTS) containing ~61,000 US domestic flights.
• Feature Engineering: Selected 14 features for generation (e.g., carrier, airport IDs, times, delays).
• Handling Imbalance: The dataset was split to isolate the 127 diverted flights. Missing values in diverted records (e.g., "Actual Elapsed Time") were imputed using diversion-specific attributes to ensure a consistent feature set for training classifiers.
• Post-Generation: Relational features (e.g., specific airport names) were reconstructed after generation, and rejection sampling was applied to remove routes not present in historical data.

B. Generative Models

Four models were evaluated to generate synthetic diversion records:

1. Gaussian Copula (GC) with KDE: A statistical baseline using Kernel Density Estimation to model marginals and a Gaussian copula for dependencies.
2. Tabular Variational Autoencoder (TVAE): A deep learning model using an encoder-decoder structure with a latent space distribution.
3. Conditional Tabular GAN (CTGAN): A GAN variant using conditional sampling to handle imbalanced categorical data and mode-specific normalization for continuous variables.
4. CopulaGAN: Combines copula transformations with adversarial learning to capture dependencies before training the GAN.

C. Multi-Objective Hyperparameter Optimization

Instead of using default configurations, the study employed Tree-structured Parzen Estimator (TPE) for automated hyperparameter tuning.

• Objective Function: A custom composite score was designed to balance four competing metrics:
  1. Maximize Realism: Percentage of valid origin-destination pairs.
  2. Maximize Statistical Similarity: Average of marginal and bivariate distribution similarities.
  3. Maximize Utility: PR-AUC (Precision-Recall Area Under Curve) of a downstream classifier.
  4. Minimize Fidelity: F1 score of a classifier trying to distinguish real vs. synthetic data (lower is better).
• Search Space: Optimized parameters included epochs, embedding dimensions, layer depths (fixed at 2 to prevent overfitting on small data), and learning rates.

D. Six-Stage Evaluation Framework

The quality of synthetic data was assessed through:

1. Realism: Validity of origin-destination pairs.
2. Diversity: Coverage of data clusters via PCA and t-SNE.
3. Operational Validity: Correlation checks (e.g., Distance vs. Air Time) to ensure physical feasibility.
4. Statistical Similarity: KS-tests for numerical features and Total Variation Distance for categorical features.
5. Fidelity: Difficulty of a Random Forest classifier to distinguish real from synthetic data (measured by F1 and Balanced Accuracy).
6. Utility: Performance of a diversion prediction model trained on augmented data (Train-Augmented-Test-Real) vs. real-only data (Train-Real-Test-Real).

3. Key Contributions

1. Multi-Objective Optimization Framework: Introduced a novel approach to tune deep generative models specifically for rare events, balancing realism, statistical fidelity, and downstream predictive utility simultaneously.
2. Comprehensive Evaluation: Demonstrated that relying on a single metric is insufficient; a multi-faceted evaluation (including operational validity and utility) is required for aviation data.
3. Performance on Extreme Imbalance: Showed that hyperparameter-optimized deep generative models (TVAE, CTGAN, CopulaGAN) can outperform statistical baselines (GC) and non-optimized deep models even with extremely small minority classes (127 samples).
4. Augmentation Size Analysis: Investigated the impact of varying augmentation sizes (300 to 200,000 samples), revealing a trade-off between detection capability (PR-AUC) and false positive rates (MCC).

4. Results

• Realism & Diversity: Optimized models significantly outperformed non-optimized counterparts. Optimized TVAE showed a 7-fold improvement in realism, while CTGAN and CopulaGAN doubled their realism scores. Optimized models also captured the full diversity of real data clusters, whereas non-optimized models failed to cover the data space.
• Statistical Similarity: Optimized models achieved statistical similarity comparable to or slightly better than the GC baseline, with improved bivariate correlations.
• Fidelity: Optimized models produced data that was indistinguishable from real data by a Random Forest classifier (F1 scores dropped significantly, and balanced accuracy hovered around 0.5, indicating maximum confusion).
• Utility (Predictive Performance):
  • Models trained on augmented data (TATR) significantly outperformed those trained on real data only (TRTR) in predicting diversions.
  • PR-AUC increased substantially with optimized augmentation, proving better rare-event detection.
  • Augmentation Size: PR-AUC improved as augmentation size increased up to the point of class balance, after which it plateaued. Conversely, MCC (which penalizes false positives) declined with excessive oversampling, highlighting the need to balance augmentation size based on the specific operational goal (safety vs. false alarm avoidance).
• Operational Validity: While generally good, some optimized models generated physically impossible records (e.g., short flight times for long distances), suggesting a need for domain-specific constraints in future work.

5. Significance and Conclusion

This study provides robust evidence that synthetic data augmentation, when driven by multi-objective hyperparameter optimization, is a viable and effective strategy for predicting rare aviation events.

• Practical Impact: It enables airlines and air traffic management to develop proactive decision-making tools for flight diversions, even when historical data is scarce and lacks strong predictive features.
• Generalizability: The methodology is feature-independent and applicable to any domain with extreme class imbalance (e.g., fraud detection, medical diagnosis).
• Key Insight: The choice of optimization target matters. Prioritizing PR-AUC is best for safety-critical detection (maximizing recall), while prioritizing MCC is better for operational balance (minimizing false alarms).

The research concludes that while deep generative models are challenging to train on small datasets, systematic optimization of their hyperparameters can yield synthetic data that not only mimics real statistics but also actively improves the predictive capabilities of downstream machine learning models.