UNAD+: An Explainable Hybrid Framework for Unknown Network Attack Detection

Technical Summary of UNAD+: An Explainable Hybrid Framework for Unknown Network Attack Detection

Problem Statement
The reliable detection of previously unseen (zero-day) network attacks remains a critical challenge for Intrusion Detection Systems (IDS). While supervised learning methods perform well on known attack classes, they fail to generalize to novel threats not present in training data. Conversely, unsupervised anomaly detection methods, which learn the characteristics of benign traffic to identify deviations, are better suited for zero-day detection but often suffer from high False Positive Rates (FPR). This limits their practical deployment, as they generate excessive alerts that burden security analysts. Furthermore, existing hybrid systems often lack transparency; the internal reasoning of complex ensemble models is frequently opaque, hindering analyst trust, root-cause analysis, and compliance in regulated sectors.

Methodology
The paper proposes UNAD+, an enhanced hybrid framework that extends the original Unknown Network Attack Detector (UNAD). UNAD+ integrates three core components into a modular pipeline:

1. Benign-Only Unsupervised Ensemble with Weighted Majority Voting (WMV):

   • The first stage utilizes a heterogeneous ensemble of 100 base learners (50 Local Outlier Factor and 50 Isolation Forest models) trained exclusively on bootstrapped subsets of benign traffic.
   • Unlike the original UNAD, which used simple majority voting, UNAD+ employs Weighted Majority Voting. During validation, each base learner is assigned a weight proportional to its F1-score. During inference, predictions are weighted by these scores to determine the final ensemble output. This approach reduces the influence of weaker detectors and eliminates tie-breaking scenarios that previously biased the original system toward benign predictions.

2. Supervised Refinement Stage:

   • To address the high FPR of the unsupervised stage, a second-stage supervised classifier is introduced.
   • The ensemble's initial detections (both correctly identified attacks and benign flows) are converted into pseudo-labels. These, combined with the original benign data, form an expanded training set.
   • A Random Forest classifier, selected for its robustness and interpretability, is trained on this pseudo-labeled data. This stage learns more precise decision boundaries from traffic patterns closer to the operational detection setting, effectively correcting residual errors from the first stage.
   • A "human-in-the-loop" checkpoint is proposed between stages to allow domain experts to validate a subset of pseudo-labels before they are used for supervised training, mitigating error propagation.

3. Post Hoc Explainability Layer:

   • To enhance transparency, the framework incorporates both local and global explanation mechanisms.
   • Local Explainability: LIME (Local Interpretable Model-Agnostic Explanations) is applied to the Random Forest classifier to generate instance-level explanations, highlighting which traffic features (e.g., packet lengths, flow bytes) contributed to specific predictions.
   • Global Explainability: A Decision Tree surrogate model is trained to approximate the behavior of the Random Forest. This provides a simplified, interpretable view of the global decision logic, including dominant rules and feature interactions.

Key Contributions

• Hybrid Architecture: The paper demonstrates a coherent integration of benign-only unsupervised detection, supervised refinement using pseudo-labels, and explainability within a single framework.
• Weighted Voting Mechanism: The introduction of WMV improves ensemble decision-making by weighting base learners based on performance, thereby reducing tie-related ambiguity and benign-class bias.
• False Positive Reduction: The supervised refinement stage is shown to be a critical mechanism for drastically reducing the false positive rates inherent in unsupervised anomaly detection.
• Integrated Explainability: The framework moves beyond treating explainability as an add-on, embedding it as a structural component to support analyst trust and auditing.

Experimental Results
The framework was evaluated on two benchmark datasets: CICIDS2017 and NSL-KDD.

• Performance Metrics:
  • On CICIDS2017, the full UNAD+ framework achieved an F1-score of 98.31%, a significant improvement over the original UNAD (75.19%) and the standalone supervised classifier (90.59%). The False Positive Rate (FPR) was reduced from 11.34% (weighted ensemble) to 0.18% after refinement.
  • On NSL-KDD, the framework achieved an F1-score of 98.25%, improving upon the original UNAD (93.38%). The FPR decreased from 6.37% to 2.77%.
• Class-Level Analysis: The refinement stage significantly improved detection for difficult or rare attack classes (e.g., SSH-Patator, DoS Slowhttptest, R2L, and U2R). However, performance gains were limited for extremely rare classes where the first stage failed to generate sufficient pseudo-labeled instances for the second stage to learn from.
• Explainability Evaluation: The local explanations (LIME) were found to be plausible, highlighting meaningful network features, and stable across repeated inspections. The global surrogate models demonstrated high fidelity (>99%) to the Random Forest classifier, successfully capturing its decision logic in an interpretable format.

Significance and Claims
The authors claim that UNAD+ offers a substantial improvement over the original UNAD baseline, not merely through predictive performance, but by addressing the operational limitations of unsupervised IDSs. The paper asserts that the combination of weighted ensemble detection and supervised refinement effectively balances the need for zero-day detection with the requirement for low false positive rates. Furthermore, the integration of explainability is presented as essential for real-world deployment, enabling analysts to interpret decisions, audit system behavior, and debug classifier errors. The work concludes that while the framework is highly effective, its performance on extremely rare attack classes is contingent upon the first stage's ability to generate informative pseudo-labels, and future work should address the challenges of encrypted traffic.