Machine Unlearning for the XGBoost Model with Network Intrusion Datasets

Technical Summary: Machine Unlearning for the XGBoost Model with Network Intrusion Datasets

Problem Statement
Machine Unlearning (MU) has emerged as a critical technique for removing specific data points from trained models without the computational expense of full retraining, addressing privacy regulations like the GDPR's "right to be forgotten." However, existing MU research predominantly focuses on deep learning and image data, leaving a significant gap in the domain of Network Intrusion Detection (NID). NID systems rely heavily on tabular data and traditional tree-based models, such as XGBoost, which are known for their high performance in this domain. While these models are effective, they lack established unlearning mechanisms. Furthermore, NID datasets often suffer from quality issues, such as mislabeling, which necessitate the removal of specific samples to maintain model reliability. Retraining these models from scratch every time data must be deleted is computationally inefficient, creating a need for an MU approach tailored specifically to XGBoost and tabular NID data.

Methodology: XGBoost-Forget
To address this gap, the authors propose XGBoost-Forget, an unlearning approach adapted from the SISA (Sharded, Isolated, Sliced, Aggregated) framework. The methodology operates as follows:

1. Data Partitioning: The training dataset is divided into multiple independent shards.
2. Slicing and Checkpointing: Within each shard, data is further divided into smaller slices. An independent XGBoost model is trained for each shard. Training occurs incrementally across slices; after each slice is processed, the model's state (parameters) is saved as a checkpoint.
3. Inference: Final predictions are generated by aggregating the outputs from all shard models.
4. Unlearning Process: When a request to forget specific data arrives, the system identifies the shard(s) containing the target samples. Only the slices within those affected shards that include the data to be removed are retrained. Slices that do not contain the target data are reused from cache. The affected shard model is rebuilt using the remaining data and the subsequent slices, while the other shards remain untouched.

Key Contributions

• First XGBoost Unlearning Approach: This work introduces the first known unlearning method specifically designed for the XGBoost model, a dominant algorithm in tabular data analysis.
• Adaptation to Tabular NID: The study bridges the gap between MU research and Network Intrusion Detection by applying the technique to real-world tabular datasets (IoT-23 and GeNIS).
• Comprehensive Evaluation: The approach is evaluated using a suite of metrics covering model utility (Accuracy, Precision, Recall, F1-Score), unlearning efficiency (Running Time), and forgetting quality (Attack Success Rate and Jensen–Shannon Divergence).

Experimental Results
The study evaluated XGBoost-Forget on two datasets: IoT-23 (real IoT malware/benign traffic) and GeNIS (staged attack sequences). The results were compared against a baseline of full retraining (XGB-R) and the SISA framework implemented with a Neural Network (NN).

• Predictive Performance: XGBoost-Forget (XGB-FU) maintained predictive performance nearly identical to the original model and the retrained baseline. In both datasets, XGB-FU achieved Accuracy, Precision, Recall, and F1-Score values comparable to the full model, with negligible degradation.
• Efficiency (Running Time): XGBoost-Forget demonstrated significantly faster unlearning times compared to full retraining. For instance, on the GeNIS dataset, unlearning took approximately 0.5–0.7 seconds compared to ~4 seconds for full retraining. It also outperformed the SISA-NN baseline in running time across all configurations.
• Forgetting Quality:
  • Attack Success Rate (ASR): ASR was used to test the removal of "poisoned" data (backdoor triggers). When evaluating the infected shard in isolation, ASR dropped significantly after unlearning (e.g., from ~100% to ~1.5% on GeNIS), approaching the levels of the retrained baseline. This indicates effective removal of the poisoned influence. When all shards were aggregated, the ASR remained low due to the dilution effect of the uninfected shards.
  • Jensen–Shannon Divergence (JSD): The JSD metric yielded values near zero for both the original vs. unlearned and retrained vs. unlearned comparisons. The authors note that this metric did not behave as expected, likely because the small removal ratio (0.01% of data) did not induce sufficient distributional changes for JSD to capture meaningful differences.

Significance and Claims
The paper claims that XGBoost-Forget successfully demonstrates that machine unlearning can be effectively applied to traditional tree-based models on tabular data without sacrificing predictive accuracy. The primary significance lies in offering a solution that is both accurate (maintaining model utility) and efficient (significantly faster than retraining) for the specific context of Network Intrusion Detection.

The authors modestly conclude that while the approach is well-suited for large and complex tabular datasets, the behavior of evaluation metrics like JSD can vary based on dataset characteristics and the size of the removed data. They suggest that further investigation is needed to refine unlearning evaluation metrics, particularly for scenarios involving small removal ratios, as the current results indicate that JSD may not be suitable for capturing subtle changes in this specific setting. The work establishes a foundation for future MU research in non-deep learning, tabular domains.