🤖 AI

Advanced Persistent Threats (APT) Attribution Using Deep Reinforcement Learning

This paper presents a Deep Reinforcement Learning model for Advanced Persistent Threat attribution that, through iterative architectural and algorithmic refinements, achieved a dramatic accuracy improvement from 7% to nearly 98%, demonstrating its robust capability to recognize and attribute malware activities.

Original authors: Animesh Singh Basnet, Mohamed Chahine Ghanem, Dipo Dunsin, Wiktor Sowinski-Mydlarz

Published 2026-08-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Animesh Singh Basnet, Mohamed Chahine Ghanem, Dipo Dunsin, Wiktor Sowinski-Mydlarz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the digital world as a massive, bustling city where everyone is connected. In this city, there are invisible burglars called "hackers." Sometimes, these burglars are just opportunistic thieves looking for an easy window to break. But then there are the "Advanced Persistent Threats" (APTs). Think of these as master spies or a highly organized crime syndicate. They don't just break in and run; they sneak in, hide in the walls for years, steal secrets slowly, and leave no fingerprints. Figuring out who these spies are—whether it's a rival nation, a criminal gang, or a specific group of hackers—is called "attribution." It's like trying to identify a ghost by the way it moves through a room. This is incredibly hard because these spies use tricks to hide their tracks, like wearing masks or taking secret tunnels.

To solve this mystery, scientists have started using a special kind of computer brain called "Deep Reinforcement Learning" (DRL). You can think of DRL as a video game character that learns by playing over and over again. Unlike a regular computer program that just follows a strict rulebook, this character tries different moves, gets points (rewards) for doing well, and learns from its mistakes. Over time, it gets so good at the game that it can spot patterns humans might miss. The big question researchers are asking is: Can we teach this digital detective to look at the messy, hidden behavior of a virus and say, "Aha! This was definitely done by Group X"?

This paper takes that idea and puts it to the test. The researchers gathered a huge collection of digital "fingerprints"—over 3,500 samples of malware (malicious software) from 12 different APT groups. They didn't just look at what the files looked like; they watched what the files did inside a safe, isolated sandbox (a virtual prison for viruses). They used a tool called Cuckoo Sandbox to watch how the viruses moved, changed files, and talked to other computers.

Then, they trained their DRL "detective" to analyze these behaviors. They set up a game where the AI had to guess which of the 12 groups created each virus. If it guessed right, it got a point. If it guessed wrong, it lost a point. The AI played this game thousands of times, learning to spot the subtle, unique habits of each group. The results were impressive: the DRL model got it right 89.27% of the time on new, unseen viruses.

To see if this new detective was actually better than the old ones, the researchers compared it to five other standard computer methods (like Decision Trees and Support Vector Classifiers). The old methods were okay, getting between 71% and 82% accuracy, but the DRL detective clearly outperformed them all, reaching that 89.27% mark. The paper suggests that this approach is particularly good at handling the tricky, changing nature of these cyber-attacks, where old rules often fail.

However, the authors are careful not to call this a magic bullet. They point out that this super-smart AI is hungry for resources; it needs a lot of computing power and a massive amount of data to learn effectively. They also note that while the model is great at spotting patterns in the data they fed it, the real world is messy. The paper concludes that while DRL is a powerful new tool for catching these digital spies, we still need to make it more efficient and handle the ethical questions of using AI in security. It's a promising step forward, suggesting that by letting AI learn from the behavior of the bad guys, we might finally be able to unmask them faster and more accurately than ever before.

Technical Summary: Advanced Persistent Threats (APT) Attribution Using Deep Reinforcement Learning

Problem Statement
The paper addresses the critical challenge of attributing malware to specific Advanced Persistent Threat (APT) groups. Unlike opportunistic cyberattacks, APTs are sophisticated, long-term operations often backed by nation-states, utilizing complex evasion techniques, zero-day exploits, and intricate behavioral patterns to remain undetected. Traditional attribution methods rely heavily on manual feature engineering and static signatures, which struggle to cope with the dynamic nature of modern threats, obfuscation tactics, and malware families lacking clear evolutionary links. The authors posit that there is a need for adaptive systems capable of analyzing behavioral patterns to accurately identify the specific APT group behind an attack, a task essential for developing targeted defensive strategies and holding perpetrators accountable.

Methodology
The study proposes a novel framework utilizing Deep Reinforcement Learning (DRL) to attribute malware samples to one of 12 distinct APT groups. The methodology follows a structured experimental design:

Data Collection and Preparation:
- Dataset: The research utilizes a dataset of over 3,500 malware samples categorized into 12 APT groups (e.g., APT1, APT28, Equation Group, Gorgon Group), originating from five countries (China, Russia, North Korea, USA, Pakistan).
- Sources: Behavioral data is extracted using Cuckoo Sandbox (for dynamic analysis of file operations, registry changes, and network traffic) and VirusTotal (for aggregated antivirus detection results and metadata).
- Preprocessing: Raw reports are cleaned, integrated, and transformed. Categorical variables (APT groups) are label-encoded, and numerical features are normalized using MinMaxScaler. To address significant class imbalance (ranging from 32 to 961 samples per group), the SMOTE (Synthetic Minority Over-sampling Technique) is employed to synthesize minority class examples.
Markov Decision Process (MDP) Formulation:
The attribution problem is modeled as an MDP where:
- State Space ( $S$ ): Represents the current understanding of a malware sample, derived from a feature vector including API calls, file operations, registry modifications, and network behaviors.
- Action Space ( $A$ ): Corresponds to the decision to attribute the current malware sample to a specific APT group (12 possible actions).
- Reward ( $R$ ): A positive reward (+1) is assigned for correct attribution, while incorrect attributions yield a negative reward (-1). The agent aims to maximize cumulative rewards.
Model Architecture and Training:
- Algorithm: A Deep Q-Network (DQN) is implemented using the Stable Baselines 3 library within a Gymnasium environment.
- Network Design: The policy and target networks are Multi-Layer Perceptrons (MLP) with a net architecture of [1024, 512, 512, 256] neurons using Leaky ReLU activation functions to prevent vanishing gradients.
- Training Strategy: The agent utilizes an $\epsilon$ -greedy strategy to balance exploration and exploitation. Experience replay is used to store transitions $(s_t, a_t, r_t, s_{t+1})$ . Hyperparameters include a learning rate schedule, a discount factor ( $\gamma$ ) of 0.99, and a batch size of 256.

Key Contributions

Novel Application of DRL: This work pioneers the use of Deep Reinforcement Learning specifically for APT malware attribution, moving beyond static classification to an adaptive, decision-making framework.
Behavioral Analysis Focus: The model relies on dynamic behavioral features extracted from sandbox environments rather than static code signatures, allowing it to handle obfuscated and evolving threats.
Benchmarking: The study provides a rigorous comparison of the proposed DRL model against traditional machine learning classifiers, including Stochastic Gradient Descent (SGD), Support Vector Classifier (SVC), K-Nearest Neighbors (KNN), Multi-Layer Perceptron (MLP), and Decision Tree Classifiers.
Open Resources: The authors have made the code and the curated APT malware dataset publicly available to facilitate further research.

Results
The DRL model demonstrated superior performance compared to all baseline machine learning models:

Test Accuracy: The DRL model achieved a test accuracy of 89.27%.
Comparison: This significantly outperformed the next best model, the Decision Tree Classifier (82.64%), as well as MLP (80.65%), KNN (80.21%), SVC (77.41%), and SGD (71.47%).
Detailed Metrics: On the test set, the model achieved an average precision of 87%, recall of 85%, and an F1-score of 86%.
Group-Specific Performance: The model showed exceptional proficiency with groups like the "Equation Group" (perfect scores in training) and "APT 19" (95.8% F1-score). However, it faced slight challenges with "APT 28" (62.4% precision) and "APT 21" (93.44% recall), indicating areas for future refinement.
Training Trajectory: The model's accuracy improved dramatically during training, rising from initial low levels to near 98% on training data and stabilizing at 89.27% on the test set after 20,000 steps.

Significance and Claims
The paper claims that the integration of DRL into cybersecurity frameworks offers a transformative potential for malware attribution. By leveraging the ability of DRL to learn from trial and error and adapt to incomplete or obfuscated information, the proposed system provides a more robust and accurate method for identifying APT actors compared to traditional static models.

The authors emphasize that while the model shows high efficacy, its deployment requires significant computational resources and high-quality, diverse datasets. They advocate for a responsible approach to AI integration, suggesting future work should focus on:

Enhancing computational efficiency through architecture refinement and model pruning.
Expanding dataset diversity to improve generalization.
Addressing ethical concerns regarding data privacy and bias.
Exploring the integration of Large Language Models (LLMs) to refine reward mechanisms and optimize the DRL framework.

Ultimately, the research argues that DRL can significantly advance threat intelligence, enabling organizations to move from reactive defense to proactive, accurate attribution of sophisticated cyber threats.

Technical Summary: Advanced Persistent Threats (APT) Attribution Using Deep Reinforcement Learning

More like this