💻 computer science

AE-DSP: An Explainable Intrusion Detection Framework for Zero-Day Attacks Based on SHAP Contribution-Space Modeling

This paper proposes AE-DSP, an explainable intrusion detection framework that addresses the challenge of zero-day attack detection by modeling SHAP contribution deviations within a unified space to capture both overall structural anomalies and dimension-wise semantic patterns, achieving high performance and traceability across multiple benchmark datasets.

Original authors: Yifan Liu, Zheng Liu, Wenlei Chai, Miaolin Wang, Junjie Qiu, Yi Liu

Published 2026-11-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yifan Liu, Zheng Liu, Wenlei Chai, Miaolin Wang, Junjie Qiu, Yi Liu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital world, security systems act as constant sentinels, watching the flow of data to spot intruders before they cause harm. For decades, these systems have relied on a simple principle: they learn what a known criminal looks like by studying thousands of past examples. If a new visitor matches the pattern of a known thief, the alarm sounds. However, this approach has a fatal blind spot. It cannot recognize a criminal who has never been seen before, a "zero-day" attacker who uses a completely new method. Traditional systems are like bouncers who only know the faces of people on a specific "do not enter" list; if a stranger walks in wearing a disguise they have never encountered, the bouncer might let them pass. The challenge for researchers is to build a system that can spot a stranger as dangerous, even without ever having seen that specific type of stranger before, by understanding the subtle, unnatural ways they try to interact with the network.

A team of researchers from Hebei University has developed a new framework called AE-DSP to solve this exact problem. Instead of trying to memorize the faces of every possible criminal, their system focuses on the logic of how decisions are made. They start with a standard security detector, a tool that has already learned to distinguish between normal traffic and known attacks. Rather than just looking at the final verdict this tool gives, the researchers peel back the layers to see why it made that decision. They break down every single piece of data in a network packet and ask: how much did this specific piece of information push the system toward thinking "this is an attack"? This creates a detailed map of influence for every single moment of traffic, showing exactly which features are driving the alarm.

The innovation lies in how they use this map. The researchers realized that normal traffic follows a very consistent pattern of influence; the features that matter most for a normal connection always contribute in a predictable way. When a zero-day attack occurs, it disrupts this pattern. The new system, AE-DSP, builds a reference model of what "normal" influence looks like. It then watches for two specific types of trouble. First, it looks for a general sense of wrongness, where the overall pattern of influence is so different from the norm that it stands out immediately. Second, and more importantly, it looks for specific, structured deviations. It asks: did the attack change the influence of specific features in a way that resembles the shape of known attacks, even if the specific features are different? To do this, the system learns "semantic prototypes," which are essentially archetypes of how attacks typically distort the decision-making process. These prototypes act as a library of suspicious patterns, allowing the system to recognize a new attack because it shares the same structural "fingerprint" of distortion as old ones, even if the attack itself is brand new.

The results of this approach are striking. The researchers tested their system on three major datasets containing millions of network records, including NSL-KDD, UNSW-NB15, and CICIDS2017. In these tests, the system was trained only on normal traffic and a set of known attacks, while a completely different set of unseen attacks was held back for the final test. The AE-DSP framework successfully identified these unseen threats with remarkable accuracy. On the NSL-KDD dataset, it caught 97.74% of the new attacks. On UNSW-NB15, it caught 93.50%, and on the complex CICIDS2017 dataset, it caught 92.51%. These numbers represent a significant improvement over existing methods, which often struggle to detect attacks they have never seen before. The system also maintained a high level of accuracy in identifying known attacks, ensuring that it did not become so sensitive to new threats that it started flagging normal activity as dangerous.

What makes this discovery particularly valuable is not just the high detection rate, but the clarity it provides. In many advanced security systems, when an alarm goes off, the operator is left with a black box that says "threat detected" but offers no explanation. AE-DSP changes this. Because the system tracks the specific contributions of every feature, it can trace the alarm back to the exact source of the problem. If the system flags a packet, it can point to the specific parts of the data stream that caused the deviation, such as an unusual connection state or a strange service access pattern. This allows security analysts to understand why a decision was made, turning a mysterious alert into a clear, actionable insight. The researchers found that the system's ability to explain its decisions was not just a side effect, but a core part of its success, as the logic used to explain the threat was the same logic used to detect it.

The study also addressed the practical reality of running such a system. While the process of breaking down the data to understand its influence requires significant computing power, the researchers found that the actual detection step—the part that happens after the data is analyzed—is very fast. This means the system could be deployed in real-world scenarios, perhaps running on powerful servers to analyze traffic in batches, rather than trying to stop every single packet in real-time. The team confirmed that their method is stable, producing consistent results even when the underlying computer processes are restarted or slightly altered. They also showed that the system works well across different types of network data, suggesting that the method of looking for structural deviations in decision-making is a robust way to handle the ever-changing landscape of cyber threats.

Ultimately, this work shifts the focus from memorizing the past to understanding the nature of the present. By transforming the raw data into a space where the logic of the detector is visible, the researchers created a tool that can spot the unfamiliar by recognizing the unnatural. It is a move away from simple pattern matching toward a deeper, more flexible understanding of what constitutes an anomaly. While the system still faces challenges with attacks that are extremely subtle or those that vary wildly within a single group, the results demonstrate that it is possible to build a defense that is both highly effective against new threats and transparent in its reasoning. This offers a promising path forward for cybersecurity, where the goal is not just to know the enemy, but to understand the very mechanics of the attack itself.

Technical Summary: AE-DSP Framework for Zero-Day Attack Detection

1. Problem Definition

The central challenge in Network Intrusion Detection Systems (NIDS) is the detection of zero-day attacks. Unlike known attacks, zero-day classes are unavailable during the training phase, preventing models from learning their specific class-conditional distributions or decision boundaries. Existing approaches typically rely on anomaly detection, open-set recognition, or zero-shot generalization, constructing detection criteria from raw traffic distributions, learned representations, or class-level relationships. However, these methods often underutilize sample-level evidence that directly links input features to the detector's decision process. Specifically, existing methods fail to fully exploit the multidimensional structure of how individual features contribute to a prediction, often summarizing anomaly evidence into a single score that loses directional and structural information.

2. Methodology: AE-DSP Framework

The authors propose AE-DSP (Autoencoder-Deviation Semantic Prototypes), an explainable framework that transforms post-hoc SHAP (SHapley Additive exPlanations) contributions into a unified, task-relevant contribution space. The framework operates under a strict "held-out" protocol where target zero-day classes are excluded from training and validation.

2.1 Task-Relevant SHAP Contribution Space

Instead of using raw traffic features, AE-DSP decomposes the output of a supervised base detector (LightGBM) into feature-wise contributions using TreeSHAP.

Global Top-k Selection: To ensure a fixed coordinate system, the framework selects the top $k$ feature dimensions based on their global importance (mean absolute SHAP value) calculated solely from the training set.
Representation: Each sample is represented as a vector $\mathbf{z}_i$ in this $k$ -dimensional space, where each element corresponds to the SHAP value of a specific feature. This preserves the alignment between contribution dimensions and original traffic features while focusing on task-relevant decision evidence.

2.2 Dual Anomaly Evidence Modeling

AE-DSP models two complementary forms of anomaly evidence within this contribution space:

Overall Deviation ( $E_{dev}$ ):
- Mechanism: An autoencoder (AE) is trained exclusively on the contribution vectors of normal traffic. It learns to reconstruct the typical structure of normal decision evidence.
- Detection: For a test sample, the squared reconstruction error is calculated. This error is robustly standardized using the median and interquartile range (IQR) of the normal training errors.
- Significance: A high reconstruction error indicates the sample's contribution structure deviates significantly from the learned normal baseline, regardless of the specific attack class.
Dimension-Wise Contribution Deviation ( $E_{pat}$ ):
- Mechanism: The framework calculates DeltaSHAP vectors, which represent the standardized deviation of a sample's contributions from the mean contribution baseline of normal traffic.
- Deviation Semantic Prototypes (DSPs): To capture the multimodal nature of attack behaviors, the system learns $M$ representative Deviation Semantic Prototypes from the DeltaSHAP vectors of known attacks only. These prototypes act as compact centers for different patterns of contribution deviations.
- Detection: For a test sample, the distance to the nearest DSP is calculated and converted into a similarity score. This score is robustly scaled using the distribution of similarities observed in normal training samples.
- Significance: This captures the direction, location, and relative strength of contribution changes, allowing the model to match unseen attacks against representative deviation patterns learned from known attacks.

2.3 Fusion and Explainability

Fusion: The final detection score is a weighted sum of the overall deviation evidence ( $E_{dev}$ ) and the contribution-deviation pattern evidence ( $E_{pat}$ ).
Explainability: Because the contribution space aligns with original features, the decision can be traced back. The system identifies the nearest DSP (indicating the type of deviation pattern) and the dominant DeltaSHAP dimensions (indicating which specific features drove the anomaly).

3. Key Contributions

The paper outlines three primary contributions:

Task-Relevant SHAP Contribution Space: A formulation that explicitly represents feature-to-decision relationships at the sample level, providing a detector-informed representation for unseen attacks while maintaining correspondence with original traffic features.
Normal-Reference-Driven Deviation Modeling: A strategy that characterizes unseen samples using two complementary evidence types:
- Overall Deviation: Captures departures from the normal contribution structure.
- Dimension-Wise DeltaSHAP: Preserves the structural details of contribution changes and relates them to representative patterns learned from known attacks.
Comprehensive Evaluation: Extensive experiments on NSL-KDD, UNSW-NB15, and CICIDS2017 under strict held-out protocols, including ablation studies, stability analyses, and quantitative explainability validation.

4. Experimental Results

The framework was evaluated on three datasets with varying attack types and traffic distributions.

Performance Metrics:
- NSL-KDD: F1-score of 0.9341, New Recall (zero-day detection) of 0.9774.
- UNSW-NB15: F1-score of 0.9250, New Recall of 0.9350.
- CICIDS2017: F1-score of 0.9025, New Recall of 0.9251.
Comparative Advantage: AE-DSP achieved the highest New Recall among compared methods (including generative augmentation, deep reinforcement learning, and open-set recognition approaches) across all three datasets. For instance, on NSL-KDD, its New Recall was 1.34 percentage points higher than the second-best method (ADQN).
Ablation Studies: Removing the SHAP transformation (using raw features) or removing either the AE or DSP component significantly degraded performance, confirming the necessity of the contribution space and the complementarity of the two evidence types.
Stability: Random-seed stability analysis showed low standard deviations in F1-score and New Recall. Parameter sensitivity analysis indicated the model is robust to variations in the number of top-k dimensions and prototype counts.
Explainability Validation: Masking the top DeltaSHAP dimensions caused a significant drop in New Recall (e.g., 21.20 percentage points for 10 dimensions on NSL-KDD) compared to random masking, validating that the identified features are indeed critical to the decision. DSP prototypes showed high purity and low entropy, confirming they capture structured patterns.

5. Significance and Limitations

Significance:
The paper claims that AE-DSP demonstrates that task-relevant anomaly evidence derived from supervised detectors can effectively generalize to unseen attacks. By transforming raw features into a SHAP contribution space, the framework bridges the gap between model-level discriminative knowledge and sample-level anomaly modeling. It successfully balances the detection of zero-day attacks (high New Recall) with the preservation of known attack detection (high Known Recall) and provides instance-level explanations that trace decisions back to specific feature contributions.

Limitations and Future Work:
The authors modestly acknowledge several limitations:

Feature Granularity: The current framework relies on flow-level features and may not fully capture long-term behavioral patterns or session-level context, making it less effective against stealthy attacks that strongly overlap with normal traffic in flow-level statistics.
Computational Bottleneck: The generation of TreeSHAP contributions is the dominant computational cost (approx. 94% of inference latency), limiting the framework's suitability for packet-level line-rate blocking. It is currently more feasible for GPU-supported micro-batch detection or offline auditing.
Future Directions: The authors suggest incorporating temporal, session-level, and host-level information, as well as investigating faster or approximate contribution computation methods to improve real-time applicability.