Robustness Analysis of Machine Learning Models for IoT Intrusion Detection Under Data Poisoning Attacks

1. Problem Statement

The integrity of Machine Learning (ML)-based Network Intrusion Detection Systems (NIDS) in Internet of Things (IoT) environments is increasingly threatened by data poisoning attacks. As IoT ecosystems rely on data-driven inference for security, adversaries can manipulate training data to:

• Reshape model decision boundaries.
• Weaken threat detection performance.
• Embed persistent misclassification patterns (backdoors).

While deep learning and ensemble models are widely adopted for IoT security, there is a lack of systematic, unified research evaluating how specific poisoning strategies affect different ML architectures across diverse, real-world IoT datasets. Existing studies often focus on clean data or specific attack vectors, leaving a gap in understanding comparative model fragility.

2. Methodology

The study employs a controlled experimental framework to evaluate the resilience of four distinct ML classifiers against four specific poisoning techniques across three major IoT datasets.

Datasets

Three real-world, high-quality IoT cybersecurity datasets were utilized to ensure diverse traffic landscapes:

1. CICIoT2023: Contains ~1 million records from 105 devices with 33 attack types (Consumer IoT focus).
2. Edge-IIoTset: Contains ~2.2 million records with 14 IIoT attacks and 63 features (Industrial IoT focus).
3. N-BaIoT: Contains ~7 million time-series samples from Mirai and Bashlite botnet infections (Botnet focus).

Machine Learning Models

Four supervised learning algorithms representing different architectural families were tested:

• Random Forest (RF): Ensemble tree-based model (Robust baseline).
• Gradient Boosting Machine (GBM): Sequential error-corrective ensemble.
• Logistic Regression (LR): Lightweight linear classifier (common in edge devices).
• Deep Neural Networks (DNN): High-capacity hierarchical models.

Poisoning Techniques

Four adversarial strategies were applied to the training data:

1. Label Flipping: Reassigning ground-truth labels to incorrect classes to corrupt supervision.
2. Outlier Injection: Injecting extreme or noisy values to distort feature distributions.
3. Feature Impersonation: Altering feature values so malicious samples mimic benign traffic patterns.
4. Generic Synthetic Outliers: Introducing statistically abnormal, fabricated samples to shift dataset distributions.

Evaluation Metrics

Performance was measured using Accuracy, Precision, Recall, and F1-Score under both clean and poisoned conditions to quantify degradation.

3. Key Contributions

• Unified Robustness Evaluation: This is the first study to provide a comparative analysis of four major ML classifiers against four distinct poisoning techniques across three major IoT datasets simultaneously.
• Identification of Model-Specific Vulnerabilities: The research delineates clear behavioral patterns, showing that ensemble methods are significantly more resilient than linear or deep learning models under specific attack vectors.
• Dataset-Specific Insights: The study reveals that the impact of poisoning is not uniform; it varies heavily based on dataset complexity (e.g., CICIoT2023 showed higher vulnerability to feature impersonation than N-BaIoT).
• Defense Implications: The findings offer empirical evidence to guide the design of hybrid, poisoning-resilient NIDS architectures and inform regulatory frameworks for AI security.

4. Experimental Results

The experiments revealed significant disparities in model resilience:

• Baseline Performance: On clean data, Random Forest (RF) and Gradient Boosting Machines (GBM) consistently outperformed Logistic Regression (LR) and Deep Neural Networks (DNN), achieving accuracy rates often above 99% (e.g., GBM on N-BaIoT reached 99.96%).
• Impact of Label Flipping: This was the most destructive attack.
  • LR and DNN suffered severe degradation, with accuracy drops of up to 40%.
  • RF and GBM remained comparatively stable but still experienced notable performance losses.
• Impact of Outlier Injection:
  • Effects were mixed. RF and GBM maintained high performance in Edge-IIoTset and N-BaIoT.
  • LR and DNN suffered significant precision and recall degradation, particularly in the CICIoT2023 dataset.
• Impact of Feature Impersonation:
  • Primarily affected models trained on CICIoT2023, causing sharp reductions in accuracy and F1-scores for LR and DNN.
  • RF and GBM demonstrated high resilience, suggesting tree-based aggregation effectively mitigates subtle feature-level manipulations.
• Impact of Synthetic Outliers:
  • Generally caused milder disruption compared to targeted attacks.
  • LR showed moderate degradation, while ensemble models (RF, GBM) and DNNs remained largely stable.

Summary of Resilience:

1. Most Robust: Random Forest and Gradient Boosting Machines.
2. Most Vulnerable: Logistic Regression and Deep Neural Networks (particularly under label manipulation and feature impersonation).

5. Significance and Future Directions

The study underscores that data integrity is as critical as model architecture in IoT security. The findings have several implications:

• Architectural Selection: For high-stakes IoT environments, ensemble methods (RF/GBM) are preferable over linear or pure deep learning models when facing potential poisoning threats.
• Defense Strategies: The paper advocates for:
  • Adversarially robust training and continuous anomaly monitoring.
  • Feature-level validation and noise-aware preprocessing.
  • Hybrid defenses combining ensemble learning with federated learning for decentralized security.
• Policy and Compliance: The authors call for integrating resilience testing into regulatory frameworks for AI-driven IoT security.
• Future Work: Recommendations include developing real-time adversarial detection mechanisms, adaptive learning strategies for model recovery, and utilizing blockchain or differential privacy to ensure long-term data integrity.

In conclusion, this work provides a critical empirical foundation for building resilient intrusion detection pipelines, highlighting that without specific countermeasures against data poisoning, even high-performing ML models in IoT can be rendered ineffective.