SE-Enhanced ViT and BiLSTM-Based Intrusion Detection for Secure IIoT and IoMT Environments

1. Problem Statement

The rapid expansion of the Industrial Internet of Things (IIoT) and the Internet of Medical Things (IoMT) has introduced critical security vulnerabilities due to the heterogeneous, resource-constrained, and highly interconnected nature of these networks. Traditional security mechanisms often fail to detect advanced, evolving cyber threats (such as zero-day attacks) or struggle with the dynamic network topologies of IoT environments.

Key challenges identified include:

• Class Imbalance: Real-world intrusion datasets often suffer from severe class imbalance, where attack samples are scarce compared to normal traffic, leading to biased models that miss minority attack classes.
• Feature Complexity: Network traffic data contains both complex spatial dependencies (relationships between features) and temporal dependencies (sequential patterns over time), which single-architecture models often fail to capture simultaneously.
• Latency Requirements: Critical IIoT and IoMT applications require real-time detection with minimal latency to prevent system compromise.

2. Methodology

The authors propose a novel hybrid deep learning architecture named SE ViT-BiLSTM, designed to integrate spatial feature extraction with temporal sequence modeling.

A. Data Preprocessing

• Datasets: Two real-world benchmarks were used:
  • Edge-IIoTset (2022): Contains 14 attack types across 5 categories.
  • CICIoMT2024: Contains 18 attack types across 5 categories, tailored for medical IoT.
• Cleaning & Encoding: Categorical features were label-encoded, and redundant columns (e.g., timestamps) were removed. The target variable (Attack Type) was encoded for multi-class classification.
• Handling Imbalance: To mitigate class imbalance, SMOTE (Synthetic Minority Over-sampling Technique) was applied to Edge-IIoT, and RandomOverSampler was used for CICIoMT2024.
• Split: Data was split 80/20 for training and validation.

B. Model Architecture

The model operates via a dual-branch parallel architecture followed by feature fusion:

1. Input Layer: Accepts network traffic sequences (60 features for Edge-IIoT; 83 for CICIoMT2024) with a time dimension of 1.
2. Branch 1: SE-Enhanced Vision Transformer (SE-ViT):
   • Embedding: A Dense layer with ReLU activation projects inputs into a higher-dimensional space.
   • Attention Mechanism: Instead of standard Multi-Head Self-Attention, the authors integrate Squeeze-and-Excitation (SE) blocks.
     • Squeeze: Performs global average pooling to capture channel-wise statistics.
     • Excitation: Passes data through Dense layers (ReLU and Sigmoid) to generate channel-wise importance weights.
     • Recalibration: Features are rescaled by these weights to emphasize informative channels.
   • Residual Connection: Recalibrated features are added to the original input and normalized via Layer Normalization.
3. Branch 2: Bidirectional LSTM (BiLSTM):
   • Processes the input sequence in both forward and backward directions to capture temporal dependencies.
   • Uses 32 units with return_sequences=True to retain full temporal dynamics.
4. Fusion & Output:
   • Outputs from both branches are flattened and concatenated.
   • A final Dense layer with Softmax activation classifies the input into one of six (Edge-IIoT) or multiple (CICIoMT2024) attack categories.

3. Key Contributions

• Novel Hybrid Architecture: Integration of SE attention mechanisms into a Vision Transformer (ViT) combined with BiLSTM to simultaneously model spatial feature recalibration and temporal dependencies.
• Comprehensive Evaluation: Rigorous testing on two distinct, real-world datasets (Edge-IIoT and CICIoMT2024) covering both industrial and medical domains.
• Imbalance Mitigation Strategy: Systematic comparison of model performance before and after applying SMOTE and RandomOverSampler, demonstrating the critical impact of data balancing on minority class detection.
• Real-Time Performance: The model achieves high accuracy with extremely low inference latency (sub-millisecond), making it suitable for edge deployment.

4. Experimental Results

The model was trained on Google Colab (T4 GPU) for 50 epochs using Adam optimizer and categorical cross-entropy.

Performance Metrics (Post-Balancing)

Dataset	Accuracy	Loss	FPR	Latency (sec/inst)
Edge-IIoT	99.33%	0.0158	0.0013%	0.00035
CICIoMT2024	98.16%	0.0578	0.0036%	0.00014

• Impact of Balancing:
  • Edge-IIoT: Accuracy improved from 99.11% to 99.33%.
  • CICIoMT2024: Accuracy significantly improved from 96.10% to 98.16%, with a substantial reduction in False Positive Rate (FPR) and loss.
  • Class-wise Performance: Before balancing, the model struggled with minority classes in CICIoMT2024 (e.g., DoS UDP flood had 60% precision). After balancing, precision and recall for these classes rose to >90%.
• Ablation Study: The parallel architecture (Model #3) outperformed sequential designs and larger BiLSTM configurations, confirming that the specific fusion of SE-ViT and BiLSTM is optimal for this task.
• Comparison: The proposed model outperformed or matched state-of-the-art methods (including GNN-XGBoost and ResNet-BiGRU) while providing a more comprehensive set of metrics (including FPR and latency).

5. Significance and Conclusion

This study demonstrates that a hybrid deep learning approach combining SE-Attention (for feature recalibration) and BiLSTM (for temporal modeling) is highly effective for Intrusion Detection Systems (IDS) in critical IoT environments.

• Robustness: The model successfully handles class imbalance, a common hurdle in cybersecurity datasets, ensuring that rare but dangerous attacks are detected.
• Efficiency: With inference times under 0.0005 seconds per instance, the framework is viable for real-time deployment in resource-constrained IIoT and IoMT devices.
• Future Directions: The authors plan to extend the work by testing on additional datasets, optimizing the model via compression for edge devices, and integrating Explainable AI (XAI) to improve interpretability for security analysts.

In summary, the SE ViT-BiLSTM framework offers a high-accuracy, low-latency solution for securing the next generation of interconnected industrial and medical systems.