Technical Summary: Hybrid Intrusion Detection System for IoT and Cloud Environments

Problem Statement

The rapid integration of Internet of Things (IoT) devices with cloud computing infrastructures has created heterogeneous network environments that are increasingly complex and vulnerable to cyberattacks. Traditional Intrusion Detection Systems (IDS) face significant limitations in these settings:

• Signature-based systems are effective against known attacks but fail to detect zero-day or emerging threats due to their reliance on predefined signatures.
• Anomaly-based systems can detect unknown threats by identifying deviations from normal behavior but often suffer from high false positive rates, low recall, and sensitivity to threshold settings.
• Supervised Machine Learning (ML) models (e.g., Random Forest, XGBoost) achieve high accuracy on labeled data but struggle to generalize to unseen attack patterns or heterogeneous traffic distributions.
• Dataset Heterogeneity: Most existing research evaluates models on single-domain datasets (either enterprise or IoT), limiting the generalizability of the resulting models to real-world, multi-domain environments where traffic characteristics differ significantly.

The core problem addressed is the lack of a lightweight, robust IDS framework capable of simultaneously detecting known and unknown attacks across diverse cloud and IoT environments while balancing detection sensitivity (recall) with false alarm control (precision).

Methodology

The authors propose a Hybrid Intrusion Detection Framework that integrates supervised classification with anomaly-aware sequence modeling. The methodology consists of four primary stages:

1. Dataset Integration and Feature Alignment

To address dataset heterogeneity, the study merges the CICIDS2017 dataset (representing enterprise/cloud traffic) and the TON IoT dataset (representing IoT/IIoT traffic).

• Feature Alignment: A manual feature selection strategy was employed to construct a unified, compact feature space. Six common traffic-related attributes were selected: destination port, flow duration, source packets, destination packets, source bytes, and destination bytes.
• Domain Indicator: A binary feature was added to indicate the dataset origin (0 for CICIDS2017, 1 for TON IoT) to allow the model to learn from both domains without losing source context.
• Preprocessing: The pipeline included handling missing values (replacing undefined/infinite values with zero), removing duplicate records, transforming labels to binary (0 for normal, 1 for attack), and applying logarithmic transformation (log1p) followed by Min-Max normalization to handle skewness and scale differences.

2. Model Architecture

The framework employs a parallel architecture with a logical OR-based fusion mechanism:

• Supervised Component: Two ensemble learning classifiers, Random Forest and XGBoost, are trained on the labeled binary dataset to detect known attack patterns. XGBoost was configured with 100 estimators, a max depth of 6, and a learning rate of 0.1.
• Anomaly Detection Component: An LSTM Autoencoder is trained exclusively on normal traffic sequences (sequence length of 10). The architecture consists of an encoder (64 and 32 hidden units) and a decoder (32 and 64 hidden units) with a TimeDistributed Dense output layer. It learns to reconstruct normal traffic; high reconstruction errors indicate anomalies.
• Hybrid Fusion: The final decision is made using a logical OR rule: a traffic sequence is classified as an attack if either the supervised model predicts it as malicious or the LSTM Autoencoder's reconstruction error exceeds a tuned threshold (0.001014).

3. Evaluation Strategy

The models were evaluated using an 80:20 train-test split with stratified sampling. Performance was measured using Accuracy, Precision, Recall, F1-score, False Positive Rate (FPR), and False Negative Rate (FNR). Additionally, an unseen-attack experiment was conducted where specific attack families (e.g., DDoS, DoS variants) were excluded from training to test generalization capabilities.

Key Contributions

1. Hybrid Framework: The development of a unified IDS combining XGBoost-based supervised classification with LSTM Autoencoder-based anomaly detection to leverage the strengths of both approaches (high precision for known attacks and sensitivity for unknown threats).
2. Cross-Domain Dataset Integration: The creation of a feature-aligned, unified dataset merging CICIDS2017 and TON IoT. This addresses the research gap of single-domain evaluation, allowing the model to learn from heterogeneous enterprise and IoT traffic patterns.
3. Comprehensive Empirical Evaluation: A detailed comparison of standalone models (Random Forest, XGBoost, LSTM Autoencoder) against the hybrid fusion model, including an unseen-attack evaluation to assess generalization.
4. Trade-off Analysis: An investigation into the trade-offs between recall improvement and false positive control, demonstrating how hybrid fusion enhances sensitivity to novel threats at the cost of slightly increased false alarms.

Experimental Results

The proposed hybrid framework demonstrated superior performance compared to standalone models under standard evaluation settings:

• Overall Performance: The hybrid model achieved 99.45% accuracy, 99.75% precision, 99.64% recall, and a 99.70% F1-score.
• Component Performance:
  • XGBoost: Achieved high precision (99.39%) and recall (98.71%), confirming its strength in classifying known attacks.
  • LSTM Autoencoder: Achieved very high precision (99.30%) but extremely low recall (12.35%) as a standalone model, indicating it is conservative and misses many attacks when used alone.
  • Hybrid Fusion: Successfully improved the recall of the system to 99.64% by capturing the missed attacks from the anomaly detector, while maintaining high precision.
• Unseen-Attack Evaluation: When tested on attack families excluded from training, the hybrid model maintained a high recall (97.41%) compared to XGBoost (96.37%), though precision dropped (27.16%) due to the increased sensitivity of the anomaly detector. This confirms the hybrid approach's ability to detect previously unseen threats.
• Per-Attack Detection: The framework achieved detection rates exceeding 98% for high-volume attacks (DDoS, DoS) and reconnaissance attacks (PortScan), and over 94% for stealthier attacks (Backdoor, Bot).

Significance and Claims

The paper claims that the proposed hybrid framework offers a robust and scalable solution for modern cybersecurity in heterogeneous environments. Its significance lies in:

• Improved Generalization: By integrating multi-domain datasets, the system is better equipped to handle the diverse traffic patterns found in real-world cloud and IoT deployments, moving beyond the limitations of single-dataset evaluations.
• Balanced Detection: The framework effectively addresses the "recall vs. precision" trade-off. It utilizes supervised learning for accurate classification of known threats and anomaly detection to capture zero-day or emerging attacks, significantly reducing the risk of missed intrusions (false negatives).
• Practical Applicability: While acknowledging the computational cost of deep learning, the authors argue that the modular design and the use of efficient ensemble methods (XGBoost) make the system viable for dynamic network monitoring. The study emphasizes that the slight increase in false positives is an acceptable trade-off for the critical improvement in detecting unknown attacks.

The authors conclude that combining supervised learning with anomaly-aware sequence modeling is a necessary step toward building intrusion detection systems that are both accurate and adaptable to the evolving landscape of cyber threats.