Cognitive Threat Intelligence and Explainable Federated Security Analytics for distributed Infrastructure Systems

Technical Summary: Cognitive Threat Intelligence and Explainable Federated Security Analytics for Distributed Infrastructure Systems

Problem Statement
The rapid expansion of distributed infrastructure systems, cloud computing, Internet of Things (IoT), and edge architectures has significantly increased the cybersecurity attack surface. Traditional centralized intrusion detection systems (IDS) face critical limitations in this environment, including scalability issues, high communication overhead, delayed threat response, and significant privacy risks associated with transmitting sensitive raw network traffic to central servers. Furthermore, many existing AI-driven cybersecurity solutions operate as "black-box" systems, lacking the transparency required for analysts to understand the reasoning behind anomaly predictions and threat classifications. This opacity hinders trust, incident investigation, and regulatory compliance. There is a pressing need for a framework that simultaneously achieves privacy preservation, collaborative threat intelligence, scalability, and explainable decision-making without compromising detection accuracy.

Methodology
This study proposes and evaluates a Cognitive Threat Intelligence and Explainable Federated Security Analytics framework. The methodology employs an empirical and experimental research design utilizing benchmark cybersecurity datasets: NSL-KDD and CIC-IDS2017.

1. Federated Learning (FL) Architecture: Instead of centralizing raw data, the framework distributes the training process. Local security models are independently trained at distributed network nodes. Only encrypted model parameters and updates are shared with a central aggregator via a Federated Averaging mechanism. This ensures that sensitive raw network traffic remains local, preserving data privacy and reducing the risk of single points of failure.
2. Algorithmic Implementation: The study implements and compares several machine learning and deep learning algorithms for anomaly detection and attack classification within the federated environment. These include:
   • Machine Learning: Random Forest, XGBoost.
   • Deep Learning: Autoencoder, Long Short-Term Memory (LSTM) networks.
3. Explainable AI (XAI) Integration: To address the "black-box" limitation, the framework integrates SHAP (Shapley Additive Explanations) and LIME (Local Interpretable Model-Agnostic Explanations). These techniques generate human-understandable insights into the model's predictions, explaining which network features (e.g., source bytes, destination bytes) contributed most to specific anomaly detections.
4. Data Preprocessing: The datasets underwent cleaning, normalization, feature selection, and handling of class imbalances (particularly for rare attack types like R2L and U2R) to ensure model robustness.
5. Evaluation Metrics: Performance was assessed using standard cybersecurity metrics: Accuracy, Precision, Recall, F1-Score, ROC-AUC, detection latency, and communication overhead.

Key Contributions

• Decentralized Privacy-Preserving Detection: The study demonstrates a viable architecture for collaborative threat detection where raw data is never shared, mitigating privacy concerns inherent in centralized IDS.
• Integration of XAI with Federated Learning: It bridges the gap between high-performance federated models and the need for interpretability, providing a mechanism for security analysts to validate AI-driven decisions.
• Algorithmic Benchmarking: The research provides a comparative analysis of Random Forest, XGBoost, Autoencoder, and LSTM within a federated setting, identifying their relative strengths in distributed environments.
• Feature Importance Analysis: By applying XAI, the study identifies specific network traffic attributes (notably src_bytes and dst_bytes) that are most critical for detecting malicious activity, enhancing the transparency of the detection process.

Results
Experimental results on the NSL-KDD and CIC-IDS2017 datasets yielded the following findings:

• Model Performance: All evaluated models demonstrated high effectiveness, with average performance metrics (Accuracy, Precision, Recall) consistently exceeding 90%.
• Top Performer: XGBoost achieved the highest accuracy and the best ROC-AUC score (0.98), followed closely by LSTM (0.97) and Random Forest (~0.95). The Autoencoder model showed lower performance, attributed to the challenges of unsupervised anomaly detection.
• Attack Detection Rates: The framework achieved a detection rate of approximately 98% for Denial of Service (DoS) attacks and 95% for Probe attacks. Detection rates were lower for Remote-to-Local (R2L) and User-to-Root (U2R) attacks, reflecting the complexity and scarcity of these attack types in the datasets.
• Explainability: SHAP and LIME successfully identified feature importance, confirming that data volume metrics (src_bytes, dst_bytes) were primary indicators of anomalies.
• Communication Overhead: The study observed variable communication costs across client nodes, with some nodes (e.g., Client 4) incurring higher overhead due to larger model updates or complex local training, highlighting communication efficiency as a remaining optimization challenge.
• Confusion Matrix: The model demonstrated high reliability, with 987 True Positives and 950 True Negatives, and relatively low False Positives (35) and False Negatives (28).

Significance of the Study
The paper asserts that this research contributes to the development of intelligent, explainable, and resilient cybersecurity architectures tailored for modern distributed infrastructure. By integrating Federated Learning with Explainable AI, the proposed framework addresses the dual challenges of data privacy and decision-making transparency. The study suggests that such frameworks can reduce reliance on centralized security infrastructures, minimize sensitive data exposure, and foster trust in automated security systems. Ultimately, the work aims to support organizations in deploying scalable, transparent, and privacy-aware solutions to protect critical digital systems against evolving cyber threats.

Future Directions
The authors note that future work should focus on optimizing communication efficiency to enhance scalability and real-time performance. They also suggest exploring hybrid AI models, blockchain-enabled federated security, and adaptive threat intelligence systems for IoT and edge computing environments.