Forensic-Oriented Intrusion Detection Using Synthetic Network Traffic Data and Explainable Artificial Intelligence

Technical Summary: Forensic-Oriented Intrusion Detection Using Synthetic Network Traffic Data and Explainable Artificial Intelligence

Problem Statement
Digital forensic investigations of network intrusions face two structural barriers that prevent the widespread adoption of machine learning (ML) pipelines: data availability and transparency. First, real network captures often contain sensitive, user-identifiable information that cannot be used for training under data protection regulations (e.g., GDPR) or chain-of-custody requirements prohibiting the processing of original evidence. Second, high-performance ML models (e.g., deep learning, ensemble methods) often operate as "black boxes," lacking instance-level justifications required for court-defensible reporting. Existing research addresses these issues independently, but no prior framework integrates synthetic data generation, intrusion detection, and explainability within a single pipeline governed by established forensic standards (ISO/IEC 27037, 27041, 27042, and NIST SP 800-86).

Methodology
The authors propose a forensic-oriented framework that operationalizes the strict separation between original digital evidence and derived analytical artifacts. The methodology follows a six-step process:

1. Evidence Handling: Original datasets (CICIDS2017, UNSW-NB15, Kitsune) are treated as immutable, read-only artifacts with SHA-256 integrity hashes. All processing occurs on "clean copies" or synthetic derivatives.
2. Synthetic Data Generation: A parameterized pipeline using the Synthetic Data Vault (SDV) framework with CTGAN (Conditional Generative Adversarial Networks) generates synthetic network traffic. This approach addresses class imbalance without post-hoc oversampling artifacts, allowing for reproducible experimentation without exposing original evidence.
3. Classification: An XGBoost binary classifier is employed for intrusion detection. XGBoost is selected for its high performance on tabular data and its compatibility with SHAP (SHapley Additive exPlanations) TreeExplainer, which provides exact, instance-level feature attributions rather than approximations.
4. Explainability: The framework integrates SHAP to generate both global summaries (ranking feature importance) and local waterfall plots (decomposing individual predictions). This maps statistical predictions to observable network behaviors (e.g., flow duration, packet rates) suitable for forensic reporting.
5. Evaluation Protocol: The study utilizes a Train-on-Synthetic, Test-on-Real (TSTR) protocol to simulate legally constrained scenarios where original evidence cannot be used for training. Three scenarios are compared: Real-to-Real (R→R), Synthetic-to-Real (S→R), and Mixed-to-Real (M→R).
6. Validation: Statistical fidelity is assessed using Kolmogorov–Smirnov (KS) testing to ensure privacy preservation (distinguishability from original data) while maintaining operational utility.

Key Contributions

• Forensic Evidence-Artifact Separation Pipeline: The first integration of synthetic data, ML-based detection, and explainability within a workflow explicitly governed by ISO/IEC 27037, 27041, 27042, and NIST SP 800-86. This operationalizes the requirement that original evidence remains unmodified while analytical artifacts are derived from synthetic or controlled copies.
• Empirical TSTR Validation: Demonstrates that models trained entirely on synthetic data can achieve high detection performance on real traffic. On CICIDS2017, the S→R scenario achieved an F1-macro of 0.96, within cross-validation variance of the R→R baseline (0.97).
• Forensic Interpretation Layer: The study maps SHAP attributions to specific forensic indicators for Brute Force, Port Scan, and Denial-of-Service (DoS) attacks. It confirms that synthetic training preserves the "attack fingerprints" necessary for expert witness testimony.
• Methodological Insight on Data Mixing: The paper identifies that mixing real and synthetic data without strict control over class distribution (M→R) leads to performance degradation (F1-macro 0.87) due to class bias, reinforcing the need for parameterized control in data augmentation.
• Operational Boundary Conditions: Cross-dataset validation reveals that synthetic-only training effectiveness is heavily dependent on feature space dimensionality. The approach is effective with high-dimensional data (e.g., CICIDS2017 with 78 features) but degrades significantly with low-dimensional datasets (e.g., Kitsune with 7 features), establishing a practical boundary of approximately 30 numeric flow-level features for synthetic-only training.

Results

• Performance: On CICIDS2017, the S→R model achieved an F1-macro of 0.96, compared to 0.97 for R→R and 0.87 for M→R. XGBoost outperformed Random Forest (0.95) and SVM (0.92) in the R→R scenario.
• Statistical Fidelity: KS testing confirmed that synthetic data is statistically distinguishable from original data (mean |KS| = 0.38 for CICIDS2017), satisfying privacy preservation requirements while retaining the distributional characteristics necessary for binary classification.
• Explainability Consistency: SHAP attributions for Brute Force, Port Scan, and DoS attacks were consistent between real and synthetic instances, validating that synthetic data preserves the behavioral patterns required for forensic analysis.
• Dimensionality Impact: On UNSW-NB15 (45 features), S→R performance dropped to F1-macro 0.52. On Kitsune (7 features), it dropped to 0.24. However, the M→R scenario recovered performance to near-baseline levels (0.98 and 0.96, respectively) in these lower-dimensional cases.

Significance
The paper claims that its primary significance lies not merely in detection accuracy, but in the creation of a defensible, reproducible, and auditable workflow for AI-assisted digital forensics. By strictly separating original evidence from analytical artifacts and utilizing explainable models, the framework addresses the "transparency barrier" that has historically hindered the legal admissibility of ML in forensic investigations. The work demonstrates that synthetic data can serve as a functional substitute for real traffic in training scenarios where privacy or chain-of-custody constraints prohibit the use of original evidence, provided that the feature space is sufficiently rich and the generation process is strictly parameterized. The authors emphasize that the framework positions ML outputs as analytical indicators to support expert judgment rather than as independent evidence of malicious intent, aligning with the epistemological limits of statistical attribution.