ExAI5G: A Logic-Based Explainable AI Framework for Intrusion Detection in 5G Networks

1. Problem Statement

The deployment of 5G networks has expanded the attack surface for cyber intrusions, particularly within the Internet of Things (IoT) ecosystem. While deep learning-based Intrusion Detection Systems (IDS) offer high detection rates, they typically function as "black-box" models. This lack of transparency creates a trust gap for security operators, who cannot verify why an alert was triggered.

• The Trade-off: Existing Explainable AI (XAI) methods (e.g., SHAP, LIME) often provide unstable feature importance scores that are difficult to translate into actionable insights.
• The Need: There is a critical need for a system that balances high performance with logic-based interpretability, allowing security experts to verify decisions and take immediate action without sacrificing detection accuracy.

2. Methodology: The ExAI5G Framework

The authors propose ExAI5G, a four-stage framework designed to make a deep learning IDS transparent through logic extraction and natural language generation.

A. Core Architecture

1. Transformer-Based IDS:
   • A TabTransformer model is trained on 5G network traffic data (29 features spanning frame, IP, TCP, UDP, and HTTP layers).
   • It uses a 128-dimensional embedding, a learnable [CLS] token, and a 6-layer Transformer encoder with 8 heads.
   • Loss Function: Optimized using Focal Loss with class weights to handle severe class imbalance (e.g., rare attacks vs. benign traffic).
2. Feature Attribution (Integrated Gradients):
   • To understand what drives the Transformer's decisions, the framework applies Integrated Gradients (IG).
   • This calculates the contribution of each input feature to the predicted class, generating global importance rankings.
3. Logic Rule Extraction (Surrogate Model):
   • A CART Decision Tree (surrogate model) is trained on the Transformer's predictions (pseudolabels).
   • The tree is constrained (max depth=4, min leaf size=40) to ensure interpretability.
   • Rule Derivation: Each leaf node is converted into a logical "if-then" rule (e.g., IF feature_A > threshold AND feature_B < threshold THEN Class_X).
   • Metrics: The system measures Fidelity (how well the tree matches the Transformer) and Coverage (how many test cases the rules explain).
4. LLM-Mediated Explanation & Validation:
   • Generation: Four different Large Language Models (LLMs)—Qwen2.5, Llama3.1, Phi4, Gemma3—convert the logical rules and IG attributions into natural language explanations.
   • Prompt Engineering: Prompts are strictly constrained to ensure the LLM uses directional language (e.g., "high," "low") that aligns with the mathematical sign of the attribution scores.
   • Three-Part Validation Scheme:
     1. Semantic Similarity: Measures cosine similarity between the original logic rule and the LLM's text.
     2. Attribution Faithfulness: Programmatically verifies if the text description matches the positive/negative sign of the feature attribution.
     3. Actionability Score: A powerful "evaluator" LLM (Llama3.3:70b) rates the explanation on a 1–5 scale based on its utility for a security analyst (e.g., does it suggest a specific attack type or next step?).

3. Key Contributions

1. Novel XAI Framework: A system that integrates a high-performance Transformer IDS with logic rule extraction and validated natural language explanations.
2. Quantitative Validation of LLM Explanations: Introduction of a rigorous, three-part evaluation methodology (Semantic, Faithfulness, Actionability) to assess LLM-generated explanations, moving beyond qualitative assessment.
3. Performance vs. Interpretability Analysis: A direct comparison demonstrating that an interpretable system can achieve near-parity with opaque models, challenging the notion that transparency requires a significant performance penalty.
4. Human-in-the-Loop Verification: A pilot study with two security experts comparing human ratings against automated LLM-as-judge metrics to validate the reliability of the automated evaluation.

4. Experimental Results

The framework was evaluated on a public 5G/IoT intrusion dataset containing 9 classes (8 attacks, 1 benign) with over 1.9 million records.

• Classification Performance:
  • Accuracy: 99.9% (99.87% reported in results).
  • Macro F1-Score: 0.854.
  • While ensemble baselines like Random Forest achieved slightly higher F1-scores (0.989), they lack interpretability. ExAI5G prioritizes the trade-off for transparency.
• Rule Extraction:
  • Extracted 16 logical rules with 99.72% fidelity to the original Transformer model.
  • The rules cover 100% of test instances; the top 8 rules explain over 99% of cases.
• Feature Importance:
  • Temporal and volumetric features (e.g., frame.time_relative, tcp.time_relative) were identified as the primary drivers of intrusion detection, aligning with domain knowledge.
• LLM Explanation Quality:
  • Top Performers: Qwen2.5:14b and Phi4:14b achieved the highest scores.
  • Faithfulness: Up to 0.99 (near-perfect alignment with attribution signs).
  • Actionability: Scores of 4.5–4.6/5, indicating the explanations provide concrete security context and next steps.
• Human vs. Automated Evaluation:
  • Human experts rated the explanations as "moderately to highly" actionable (avg 3.6/5).
  • While human ratings were slightly more conservative than the automated LLM-evaluator, the rankings of models were largely consistent, validating the automated metrics.
• Efficiency:
  • Median inference latency of 2.48 ms per flow on a CPU, enabling real-time deployment.

5. Significance and Conclusion

The ExAI5G framework addresses a critical bottleneck in 5G security: the inability to trust opaque AI models in operational environments.

• Trustworthiness: By converting complex neural network decisions into human-readable logical rules and actionable natural language, the system bridges the gap between AI and security analysts.
• Operational Viability: The study proves that high accuracy (99.9%) does not require sacrificing transparency. The slight performance gap compared to the best black-box models is deemed an acceptable trade-off for the ability to verify and act on alerts.
• Future Impact: The framework lays the groundwork for automated security orchestration, where extracted logical rules could directly trigger firewall updates or SIEM correlations, moving from "explainable" to "actionable" AI.

Limitations & Future Work: The authors note that the current rules are specific to the dataset used and may not generalize perfectly to all 5G environments. Future work will focus on cross-dataset validation, improving surrogate model fidelity, and developing defenses against adversarial attacks that might exploit the transparency of the extracted rules.