Technical Summary: Deep Learning for Proxy Zero-Day Intrusion Detection

1. Problem Definition and Context

The paper addresses the critical challenge of detecting Zero-Day Attacks (ZDAs)—exploits of previously unknown vulnerabilities for which no signatures or patches exist. A fundamental problem identified is that true zero-day exploits are rarely observable in public benchmark datasets. Consequently, empirical research cannot directly evaluate the discovery of unknown vulnerabilities. Instead, the field relies on proxy evaluation strategies to approximate zero-day conditions.

The authors argue that current literature often conflates these proxy evaluations with true vulnerability discovery, leading to over-claimed effectiveness. Furthermore, existing surveys on Deep Learning (DL) for Intrusion Detection Systems (IDS) are fragmented, often focusing on specific architectures, datasets, or narrow themes without integrating robustness, explainability, and deployment readiness into a unified evaluation framework. The paper posits that without standardized proxy definitions and rigorous evaluation, reported high accuracy scores may not translate to operational resilience against novel threats.

2. Methodology

The study employs a Systematic Review following PRISMA 2020 and PRISMA-S guidelines.

• Search Strategy: The authors searched five corpus-forming databases (IEEE Xplore, ACM Digital Library, Elsevier ScienceDirect, SpringerLink, Wiley Online Library) up to April 30, 2026, supplemented by Scopus and Web of Science searches in July 2026 for coverage validation.
• Selection Process:
  • Initial identification: 1,101 records (from primary databases and citation chaining).
  • Screening: 783 records screened by title/abstract; 350 full-text articles assessed.
  • Correction and Audit: A critical step involved a "corrected publication-status and eligibility audit." This process verified peer-reviewed venue status and empirical eligibility.
  • Final Corpus: The review retained 37 peer-reviewed empirical primary studies for the primary synthesis. An additional 63 records were retained as background or excluded evidence.
• Data Extraction: The authors extracted data on DL architectures, datasets, proxy strategies, performance metrics, robustness, explainability, and deployment readiness.
• Synthesis Approach: Due to the heterogeneity of reported metrics, lack of variance estimates, and inconsistent proxy protocols, the authors explicitly avoided formal meta-analysis. Instead, they performed a descriptive quantitative synthesis, aggregating point estimates (accuracy, F1-score) by architecture and dataset while noting sample sizes and ranges.
• Evaluation Framework: A novel seven-dimension framework was developed to assess studies based on: (1) DL Architecture, (2) Dataset Context, (3) Zero-Day Evaluation Strategy, (4) Performance Metrics, (5) Robustness, (6) Explainability, and (7) Deployment Readiness.

3. Key Contributions

The paper makes four primary contributions:

1. Operational Distinction and Validity Ladder: It formally distinguishes between true zero-day exploitation and four proxy strategies:

   • Unseen attack-class evaluation (holding out attack types).
   • Temporal generalization (training on past, testing on future).
   • Cross-dataset (domain-shift) evaluation.
   • Open-set/unknown-class evaluation.
     The authors present a "validity ladder" (Figure 1) ordering these proxies by their closeness to true zero-day discovery, emphasizing that all are stress tests of robustness rather than proof of vulnerability discovery.

2. Auditable Evidence Base: It provides a traceable, corrected corpus of 37 studies, distinguishing between primary empirical evidence and background/excluded records, with detailed documentation of exclusion reasons (e.g., lack of empirical validation, wrong domain).

3. Descriptive Quantitative Synthesis: It synthesizes performance data across the 37 studies, highlighting that while reported accuracies are often high (e.g., medians around 97-99% for Hybrid and Autoencoder models), these values are not directly comparable due to varying datasets and proxy definitions. It identifies that Hybrid/multi-architecture models and Unspecified/other DL categories dominate the corpus.

4. Seven-Dimension Evaluation Framework: It proposes a structured framework (Table 2) to evaluate DL-based IDSs under proxy zero-day conditions. This framework moves beyond simple accuracy to include robustness (adversarial testing), explainability (SHAP, LIME, attention), and deployment readiness (latency, memory, energy).

4. Key Results and Findings

• Architecture Trends: The corpus is dominated by Unspecified/other DL (13 studies) and Hybrid/multi-architecture models (7 studies). While Transformer and GNN architectures are discussed in broader literature, they were not present in the final eligible 37-study corpus with extractable metrics. Hybrid models (e.g., CNN-LSTM, AE-Transformer) showed the highest reported accuracies in the small quantitative subset.
• Dataset Usage: CICIDS2017 (12 studies) and UNSW-NB15 (5 studies) are the most frequently used benchmarks. However, 46% of studies used custom, proprietary, or unreported datasets, leading to significant fragmentation. Older datasets like KDD99 and NSL-KDD are still in use but are noted for potential bias.
• Proxy Strategies: Unseen attack-class testing is the most common proxy strategy. Temporal, cross-dataset, and open-set evaluations are significantly underutilized.
• Robustness: Only 35% (13 of 37) of studies explicitly evaluated adversarial robustness. Iterative and optimization-based attacks (e.g., PGD, CW) were found to cause larger performance degradation than single-step attacks (e.g., FGSM). Adversarial training is the most reported defense but incurs high computational costs.
• Explainability: Only 22% (8 of 37) of studies mentioned any explainability technique. Attention visualization (6 studies) was the most common, while SHAP (2 studies) and LIME (0 studies) were rarely reported. This indicates explainability is not yet a standard component of evaluation pipelines.
• Reporting Gaps: The review identified persistent issues: inconsistent proxy definitions, under-reporting of variance and deployment metrics (latency, memory), and a lack of cross-dataset validation.

5. Significance and Claims

The paper claims significance not in proving a specific architecture is superior, but in restructuring how the field evaluates and compares DL-based ZDA detection.

• Realism over Peak Accuracy: The authors argue that high benchmark accuracy is insufficient evidence of zero-day capability. The proposed framework shifts the focus toward reproducible, comparable, and operationally meaningful evidence.
• Proxy Validity: By establishing a validity ladder, the paper clarifies that proxy strategies are "stress tests" of robustness, not proofs of vulnerability discovery. This distinction is crucial for managing expectations in security deployments.
• Holistic Assessment: The paper highlights that current research often neglects robustness, explainability, and deployment constraints. It calls for future studies to integrate these dimensions to ensure models are not only accurate but also trustworthy and deployable in resource-constrained environments.
• Future Directions: The authors identify key research gaps, including the need for self-supervised learning to address label scarcity, federated learning for privacy, and lightweight models for edge deployment. They emphasize the need for standardized proxy protocols and adversarial robustness benchmarks to advance the field from academic exercises to operational solutions.

In conclusion, the paper asserts that progress in DL-based ZDA detection depends less on reporting another near-perfect benchmark score and more on standardizing and stress-testing the conditions under which claims of zero-day generalization are made.