💻 computer science

An Explainable AI Audit of Temporal Signatures in Botnetand Multi-Vector Cyber-Attacks

This paper demonstrates that while high-accuracy machine learning intrusion detectors exhibit weak cross-domain transferability due to distinct temporal signatures for botnet and multi-vector attacks, applying a multi-method explainable AI framework successfully identifies a robust, consensus set of temporal features that transforms black-box models into auditable and transferable evidence for trustworthy intrusion detection.

Original authors: Gizachew Teshome Habtamu, Shahzad Ahmed

Published 2026-08-13

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Gizachew Teshome Habtamu, Shahzad Ahmed

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard at a massive, bustling airport. Your job is to spot troublemakers among millions of travelers. In the past, guards looked at faces or checked specific items in bags. But today, the "bad guys" are so good at hiding their faces and packing their bags that the guards have started using super-smart robots (Machine Learning) to do the spotting. These robots are incredibly accurate, often catching 99% of the bad guys. However, there's a catch: these robots are "black boxes." They scream "ALERT!" but won't tell you why. Did the robot catch a thief because he was running? Because he was carrying a heavy suitcase? Or did it just get confused because he was wearing a red hat? If the robot can't explain itself, security teams can't trust it, and they can't teach it to spot new kinds of trouble.

This is where a field called "Explainable AI" (XAI) comes in. Think of XAI as a translator that forces the robot to show its work, like a math student writing out their steps on a test. The big question researchers have been asking is: Do the clues the robot uses to catch one type of criminal (like a group of hackers working together) also work for catching a different type (like a giant flood of fake traffic)? Or does the robot just memorize the specific "uniform" of the criminals it trained on? If the robot only knows the red hat, it will miss the blue-hat criminals. This paper dives into that mystery, testing whether the "clues" a robot learns are universal or just specific to the training camp.

The Great Detective Swap: Botnets vs. Floods

In this study, two researchers, Gizachew and Shahzad, decided to put seven different "robot detectives" (machine learning models) through a rigorous test. They wanted to see if these robots could spot two very different types of cyber-criminals using only a tiny, nine-item checklist of "temporal" clues.

The Two Villains:

The Botnet: Imagine a swarm of thousands of tiny, mindless drones (infected computers) all waiting for a signal from a master. They move in perfect, boring unison, like a marching band. Their "signature" is a strict, rhythmic timing.
The Multi-Vector Attack: Imagine a chaotic mob of different troublemakers. Some are throwing giant rocks (floods), some are picking locks (scans), and others are trying to guess passwords (brute force). They are loud, messy, and vary in size and speed.

The Nine Clues:
Instead of looking at the content of the messages (which is often encrypted or private), the researchers forced the robots to look only at the timing and shape of the traffic. They gave them a nine-item checklist:

How long the conversation lasted.
How much data was sent (volume).
How fast it was sent (rate).
Which way the data flowed (direction).
How big the individual "packages" of data were (packet size).
And most importantly, the time gaps between packages (Inter-Arrival Time or IAT).

The Shocking Discovery: Different Rules for Different Villains

The robots were trained on these nine clues and performed amazingly well, catching almost 100% of the bad guys in both scenarios. But when the researchers asked the robots to explain how they caught them, the results were mind-blowing.

The Botnet Catch:
When the robots caught the Botnet (the marching drones), they relied almost entirely on timing. They looked at the gaps between messages. Because the drones move in perfect rhythm, the time gaps were incredibly regular. The robots said, "Aha! The time between these messages is too perfect! It's a bot!" In fact, for the botnet detectors, the timing clues were so important that if you scrambled the timing, the robot's ability to catch the bad guys dropped to near zero.

The Multi-Vector Catch:
When the robots caught the chaotic mob (the floods and scans), they ignored the timing almost completely. Instead, they looked at size and volume. They said, "Look at this! The packets are all the same weird size," or "This person is sending way too much data in one direction!" The timing didn't matter as much because the chaotic mob moved at random speeds.

The "Black Box" Trap:
Here is the most critical finding: The researchers tried to see if the clues used for the Botnet would work for the Multi-Vector attack, and vice versa. They did a math check to see if the "importance" of the clues matched up between the two groups. The result? They didn't match at all. The correlation was practically zero (0.01).

This means that if you take a robot trained to spot Botnets and tell it to look for Floods, it will likely fail because it's looking for the wrong clues. It's like teaching a dog to fetch a ball, then expecting it to catch a frisbee just because you used the same word "fetch." The paper explicitly rules out the idea that you can just copy-paste a "feature importance" dashboard from one security system to another and expect it to work. The clues are domain-specific.

The Silver Lining: The "Robust" Clues

So, is all hope lost? Do we need a different robot for every single type of attack? Not quite. The researchers used a clever trick to find the "common ground." They asked: "What is the least important clue for both types of attacks?" No, wait—they asked the opposite: "What clues are still important even if we take the minimum score across both worlds?"

By looking for the clues that survived the toughest test (being important in both the Botnet world and the Flood world), they found a "Consensus Set" of three super-robust signatures:

Variability in Timing: How much the time gaps change (not just the average time).
Source Byte Ratio: The balance between how much data is sent versus received.
Average Packet Size: The typical size of the data chunks.

These three clues are the "universal keys." They are hard for a bad guy to fake. If a botnet tries to mess up its timing to hide, it ruins its own rhythm. If a flood attacker tries to change packet sizes to hide, they might break their own flood pattern.

What This Means for You

The paper concludes that while the "black box" robots are great at catching bad guys, we can't just trust them blindly. We need to use Explainable AI (XAI) to peek inside and see what they are actually looking at.

The study shows that:

Timing is King for Bots: If you want to catch a botnet, watch the clock.
Size is King for Floods: If you want to catch a flood, watch the volume.
Don't Copy-Paste: You can't assume a robot trained on one type of attack will know how to spot another just because it's smart.
The Safe Bet: If you want a security system that works everywhere, focus on the three "Consensus" clues (timing variability, data balance, and packet size).

The researchers are careful to say that these results are based on specific datasets (CIC-IDS2017 and CTU-13), which are like training simulations. While the robots performed perfectly in these tests (over 99% accuracy), the real world is messier. However, the method they used—forcing the robots to explain themselves and finding the clues that survive across different worlds—gives us a new, trustworthy way to build security systems that don't just guess, but actually understand what they are seeing.

Technical Summary: An Explainable AI Audit of Temporal Signatures in Botnet and Multi-Vector Cyber-Attacks

Problem Statement

While Machine Learning (ML) based Intrusion Detection Systems (IDS) achieve high accuracy on benchmark datasets, they function as "black boxes," offering little insight into the evidence driving their predictions. This opacity prevents Security Operations Centre (SOC) analysts from auditing models for shortcut learning (e.g., reliance on specific port numbers) or justifying automated responses to stakeholders. Furthermore, high within-dataset accuracy does not guarantee generalizability; models often fail when transferred to different network environments. A critical gap exists in understanding whether the features identified as important by Explainable AI (XAI) techniques constitute robust, transferable temporal signatures across distinct attack domains—specifically, between botnet traffic (characterized by periodic command-and-control) and multi-vector attack traffic (characterized by volumetric floods, scans, and brute force).

Methodology

The authors propose a three-layer XAI framework to audit seven diverse classifiers trained on a harmonized, nine-dimensional temporal feature space.

1. Datasets and Feature Harmonization

Two heterogeneous benchmarks were utilized:

CIC-IDS2017: Contains multi-vector attack traffic (DoS, DDoS, PortScan, Brute Force, Web Attacks, Bot).
CTU-13: Contains real-world botnet traffic scenarios.

To enable direct comparison, the authors engineered an identical nine-feature temporal representation computable from both datasets, stripping away payload and port-specific identifiers to focus purely on behavioral timing and volume:

duration_s, total_bytes, total_packets
bytes_per_s, packets_per_s
src_bytes_ratio (directionality)
avg_packet_size
iat_mean_s (Inter-Arrival Time mean)
iat_std_s (Inter-Arrival Time standard deviation)

2. Model Training

Seven classifiers spanning four learning paradigms were trained on both datasets using the harmonized temporal features:

Tree Ensembles: Random Forest (RF), AdaBoost (ADA), LightGBM.
Deep Learning: Deep Neural Network (DNN), Multilayer Perceptron (MLP).
Distance/Kernel: k-Nearest Neighbors (KNN), RBF-kernel SVM.

3. Explainable AI (XAI) Protocol

Three independent attribution methods were applied to audit the models:

SHAP (SHapley Additive exPlanations): Used TreeExplainer for tree models and Kernel/GradientExplainer for others.
LIME (Local Interpretable Model-agnostic Explanations): Sparse local linear surrogates.
Permutation Importance: Measuring the drop in ROC-AUC when features are shuffled.

4. Robustness Analysis

To identify transferable signatures, the authors employed a consensus robustness-floor methodology. This involved:

Calculating cross-dataset Spearman rank correlations ( $\rho$ ) to test if feature importance rankings transfer.
Computing a "robustness floor" for each feature by taking the minimum normalized importance across both datasets, all algorithms, and all XAI methods. Features with high minimum scores are considered robust signatures.

Key Results

1. Detection Performance

Using only the nine temporal features, six of the seven classifiers achieved ROC-AUC $\ge$ 0.99 on both datasets.

Tree ensembles (RF, ADA, LightGBM) achieved near-perfect scores (1.000) on CTU-13.
The reduction from ~78 native features to 9 temporal features resulted in negligible performance loss (<0.005 AUC) for non-kernel models, proving that behavioral timing is sufficient for detection without payload inspection.

2. Divergent Feature Importance

XAI analysis revealed a stark dichotomy in what models rely on for different attack types:

Multi-Vector Attacks (CIC-IDS2017): Detection is dominated by volumetric and packet-size evidence (e.g., avg_packet_size, src_bytes_ratio). These attacks manipulate packet sizes and exhibit asymmetric traffic flows.
Botnets (CTU-13): Detection is exposed almost entirely by Inter-Arrival-Time (IAT) regularity (iat_mean_s, iat_std_s). Botnet command-and-control beaconing creates low-variance timing patterns.
Cross-Dataset Correlation: The global feature importance rankings do not transfer between domains. The mean Spearman correlation ( $\rho$ ) across 21 algorithm-method pairs was +0.01, with 9 pairs showing negative correlation. A feature ranking valid for botnets is often misleading for multi-vector attacks.

3. Consensus Robust Signatures

By applying the robustness-floor methodology (taking the minimum importance across domains), the authors distilled a set of signatures that remain informative in both contexts:

IAT Standard Deviation (iat_std_s): The most robust signature, critical for botnets and relevant for slow-rate DoS/brute force in multi-vector attacks.
Source-Byte Ratio (src_bytes_ratio): Indicates directional asymmetry.
Average Packet Size (avg_packet_size): Indicates uniformity in packet generation.

These three features are behaviorally difficult for an adversary to evade simultaneously without degrading the attack's functionality (e.g., injecting jitter to hide botnets degrades C&C reliability; randomizing packet sizes disrupts flood efficiency).

Significance and Claims

The paper claims that XAI can transform black-box detectors into auditable, transferable temporal evidence. The primary contributions are:

Validation of Temporal Sufficiency: Demonstrating that compact temporal features alone support near-perfect detection, enabling privacy-preserving, payload-free IDS suitable for encrypted traffic and resource-constrained edge environments.
Exposure of Domain Specificity: Providing empirical evidence that feature importance is not universal. Relying on feature-importance dashboards from one environment (e.g., a botnet dataset) for another (e.g., a multi-vector dataset) is statistically unsound.
Methodology for Robust Signatures: Introducing a "robustness-floor" approach to extract signatures that are stable across algorithms, attribution methods, and attack domains. This moves beyond simple feature ranking to identifying behavioral invariants.
Operational Utility: The study suggests that XAI should be integrated into MLOps loops to detect dataset artifacts (such as the "Destination Port" shortcut identified in full-feature models) and to provide analysts with verifiable, behavior-based justifications for alerts, rather than opaque probability scores.

The authors conclude that while the current study is limited to testbed data, the identified consensus signatures offer a principled path toward trustworthy, transparent, and transferable intrusion detection.