A Comparative Study of Intrusion Detection Models for DDoS and Brute Force Attack Detection Using Public Cybersecurity Datasets

Technical Summary: A Comparative Study of Intrusion Detection Models for DDoS and Brute Force Attack Detection

Problem Statement
The rapid expansion of internet-based services has escalated the risk of cyberattacks, particularly Distributed Denial of Service (DDoS) and Brute Force attacks. Traditional Intrusion Detection Systems (IDS) rely on signature-based detection, which is effective against known threats but fails to identify zero-day attacks or evolving patterns. While anomaly-based detection using machine learning offers a solution, existing literature often lacks systematic comparisons of different algorithms under identical experimental conditions. Variations in preprocessing, feature engineering, and evaluation metrics make it difficult to determine the most effective model for specific attack types. This study addresses the need for a comparative analysis of Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Artificial Neural Networks (ANN) using the CICIDS2017 dataset to identify the most reliable algorithm for detecting DDoS and Brute Force attacks.

Methodology
The research employs a quantitative approach using supervised machine learning. The study utilizes the CICIDS2017 dataset, a benchmark containing realistic network traffic with 78 numerical features and multiple attack categories. The methodology follows a structured pipeline:

1. Data Preprocessing: The dataset was cleaned by replacing infinite values with missing values (NaN) and removing records with missing data. Categorical attack labels were encoded numerically. Crucially, feature scaling (StandardScaler) was applied exclusively to the Artificial Neural Network (ANN) inputs, as tree-based models (RF and XGBoost) do not require this step.
2. Dataset Splitting: The preprocessed data was divided into training and testing subsets using an 80:20 stratified split to maintain class distribution.
3. Model Implementation: Three models were trained on the same data:
   • Random Forest: Configured with 100 decision trees and a random seed of 42.
   • XGBoost: Configured with 100 estimators, a maximum tree depth of 6, a learning rate of 0.1, and subsample/column sampling ratios of 0.8.
   • Artificial Neural Network: A deep learning model with one input layer, three hidden layers (128, 64, and 32 neurons respectively), and one output layer. It utilized ReLU activation for hidden layers, Softmax for the output, and Dropout (0.3) to prevent overfitting. The model was trained using the Adam optimizer and Sparse Categorical Crossentropy loss.
4. Evaluation: Performance was assessed using Accuracy, Precision, Recall, and F1-score. Confusion matrices were analyzed to evaluate True Positives, True Negatives, False Positives, and False Negatives.

Key Results
All three models demonstrated excellent classification performance, with accuracy rates exceeding 99%. However, distinct differences emerged:

• XGBoost achieved the highest performance across all metrics, recording approximately 99.9975% for Accuracy, Precision, Recall, and F1-score. It produced the most accurate confusion matrix with minimal false positives and false negatives.
• Random Forest yielded highly competitive results, closely following XGBoost with 99.9965% across all metrics.
• Artificial Neural Network (ANN) achieved strong results but slightly lower performance (~99.955%) compared to the ensemble methods. The authors attribute this to the model's sensitivity to feature scaling and the specific hyperparameter configuration used.

Feature importance analysis for the tree-based models revealed that backward-flow traffic characteristics (e.g., Avg Bwd Segment Size, Bwd Packet Length Max, Bwd Packet Length Std) were the most influential features in distinguishing malicious traffic from benign activity.

Key Contributions

• Systematic Comparison: The study provides a direct comparison of RF, XGBoost, and ANN under identical preprocessing, training, and evaluation conditions, addressing a gap in literature where studies often use disparate methodologies.
• Performance Benchmarking: It establishes XGBoost as the superior model for detecting DDoS and Brute Force attacks within the CICIDS2017 context, outperforming both Random Forest and ANN.
• Reproducibility: The paper offers a reproducible framework and detailed configuration parameters, serving as a reference for future research.
• Feature Insight: The analysis identifies specific backward-flow network characteristics as critical indicators for intrusion detection.

Significance and Claims
The paper claims that ensemble learning techniques, particularly XGBoost and Random Forest, are highly effective for modern intrusion detection, offering reliability with extremely low false positive and negative rates. The authors assert that these tree-based methods provide a practical balance between high classification performance and computational efficiency, making them suitable for deployment in operational network security environments.

The study concludes that while deep learning (ANN) is capable, ensemble methods may be more robust for this specific task under the tested conditions. The findings are intended to assist cybersecurity professionals and network administrators in selecting appropriate machine learning algorithms for developing intelligent IDS. The authors acknowledge limitations, noting that the study relied on a static benchmark dataset rather than live traffic and focused only on DDoS and Brute Force attacks, suggesting that future work could explore hybrid models, real-time applications, and broader attack categories.