💻 computer science

The MAL Simulator: Cyber Operations Simulation based on Attack &amp; Defense Graphs

The paper introduces the MAL Simulator, a flexible cyber operations platform built on the Meta Attack Language that enables the training and evaluation of automated offensive and defensive agents, demonstrating their effectiveness through case studies in an emulated network environment.

Original authors: Jakob Nyberg, Sandor Berglund, Andrei Buhaiu, Joakim Loxdal, Pontus Johnson, Mathias Ekstedt

Published 2026-10-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jakob Nyberg, Sandor Berglund, Andrei Buhaiu, Joakim Loxdal, Pontus Johnson, Mathias Ekstedt

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital realm, security is often a game of cat and mouse played at the speed of light. Defenders build walls and watch for intruders, while attackers search for the smallest crack to slip through. To prepare for these battles, researchers use computer simulations, virtual playgrounds where they can test strategies without risking real systems. However, creating a realistic simulation is difficult. If the virtual world is too simple, the lessons learned do not apply to the complex reality of modern networks. If it is too rigid, the software cannot adapt to new types of computers or new kinds of attacks. The challenge lies in building a simulator that is flexible enough to model any system, yet structured enough to teach artificial intelligence how to think like a hacker or a guardian.

A team of researchers at the KTH Royal Institute of Technology in Sweden has addressed this challenge by developing a new tool called the MAL Simulator. This system is built on a foundation known as the Meta Attack Language, a specialized way of describing how computers, users, and software interact. Instead of hard-coding every possible scenario, the researchers designed the simulator to read these descriptions. This means the same core software can be used to simulate a small office network, a massive cloud system, or a factory control system simply by changing the description, without needing to rewrite the underlying code. The researchers used this tool to train two types of artificial intelligence agents: one designed to attack the system and find weaknesses, and another designed to defend it.

The team tested their creation in a controlled environment that mimicked a real computer network, complete with servers, workstations, and the complex connections between them. They created thirty different versions of this network, varying the number of computers and the layout of the connections, to ensure the agents were learning general skills rather than just memorizing a single map. The goal for the attacking agent was to reach specific sensitive data, while the defending agent's job was to stop the attack with the least amount of disruption to the network. In the real world, shutting down a server to stop a hacker might be effective, but it also stops the business from working, so the defender must be precise.

When the researchers put the trained attacking agent to the test, it proved to be significantly more efficient than traditional computer search methods. Standard algorithms, which often check every possibility in a systematic but blind way, took much longer and made more unnecessary moves to reach their goal. The trained agent, having learned from thousands of simulated attempts, knew exactly which path to take to compromise the system quickly. It could even handle network layouts it had never seen before, showing that it had learned the underlying logic of the attack rather than just a specific route. This suggests that artificial intelligence can learn to navigate complex digital environments more effectively than rigid, pre-programmed rules.

The results for the defending agent were more nuanced and highlighted a critical reality of cybersecurity. When the simulation was perfect, with the defender receiving clear and accurate alerts about every move the attacker made, a simple, rule-based strategy worked just as well as the trained artificial intelligence. However, the real world is rarely perfect. In the simulations that included "noise"—where the defender received false alarms or missed real ones, mimicking the confusion of a real network—the trained agent outperformed the simple rules. The artificial intelligence learned to ignore the noise and focus on the most likely threats, whereas the simple rules were easily confused by the false signals. This finding suggests that while basic rules might work in ideal conditions, intelligent systems are necessary for the messy, uncertain reality of actual cyber defense.

The most revealing test came when the researchers pitted the trained attacker against the trained defender. In this scenario, the defender's performance dropped significantly. The simple strategies and the rules-based agents were overwhelmed by the speed and adaptability of the intelligent attacker. Even the trained defender struggled, indicating that a defense strategy built only against predictable, slower attackers is not enough to stop a sophisticated, learning adversary. This outcome underscores a vital point: to build truly robust defenses, security systems must be trained against opponents that are just as smart and adaptive as they are. The researchers found that without this kind of adversarial training, defenses may appear strong in tests but fail when faced with a real, intelligent threat.

The study concludes that the MAL Simulator provides a powerful platform for developing these advanced agents. By allowing researchers to easily switch between different network types and attack scenarios, the tool helps create artificial intelligence that is flexible and resilient. The work demonstrates that while we can build agents that are better than current methods at both attacking and defending, the gap between the two is constantly shifting. As attackers become smarter, defenders must evolve in tandem. The researchers have made their simulator and the tools to build these agents publicly available, inviting others to continue the work of training systems that can protect our digital infrastructure in an increasingly complex world.

Technical Summary: The MAL Simulator

Problem Statement

The development of Automated Cyber Operations (ACO) agents, such as those for penetration testing and intrusion response, requires robust simulation environments. Existing simulators often suffer from rigidity; they are frequently hard-coded to specific network topologies or asset types, making it difficult to adapt them to new target domains without modifying the simulator's core source code. Furthermore, many simulators lack the ability to simultaneously train and test both offensive and defensive agents within a unified framework, or they fail to support the configuration of diverse system variants necessary to prevent agent overfitting. There is a specific need for a simulator that bridges the gap between risk assessment frameworks and reinforcement learning (RL) training environments, allowing for the generation of attack graphs based on explicit data models that can be altered without changing the simulation engine.

Methodology

The MAL Simulator Architecture

The authors developed the MAL Simulator, a cyber operation simulator built upon the Meta Attack Language (MAL). MAL is a modeling language framework used for threat modeling and secure system design.

Core Design: The simulator separates the simulation logic from the domain semantics. The simulator core remains fixed, while the environment is defined by a MAL language (defining asset types, relations, and attack/defense steps) and an instance model (specific system data).
Formalism: The simulator extends MAL to define a Partially Observable Markov Decision Process (POMDP) and, in multi-agent settings, a Markov Game.
- State: Defined by the set of performed attack and defense steps up to time $t$ .
- Actions: Attackers select traversable attack steps; defenders select unperformed defense steps.
- Transitions: Determined by a discrete event-driven logic where defense actions can block concurrent attack attempts, but previously performed attack steps persist.
- Observability: The simulator supports configurable observability, including False Positives (FP) and False Negatives (FN) to model imperfect Intrusion Detection Systems (IDS).
- Time to Compromise (TTC): Probabilistic distributions model the effort required to execute attack steps, introducing stochasticity to attack success.

Case Study Setup

The authors validated the simulator using an emulated network named AIR-DELIVERY-SYSTEM24 (ADS-24), implemented in the cyber range CRATE.

Data Source: Network inventory data was collected via Osquery and used to generate 30 distinct instance model variations (scenarios) using a subset of MAL called CadsLang.
Scenarios: These variations included different network sizes (105–165 assets) and topologies to test generalization.

Agent Training

Two types of agents were trained using Reinforcement Learning (RL):

Offensive Agent (Attacker):
- Goal: Reach specific data access goals while minimizing the number of steps (cost).
- Architecture: Utilized a Heterogeneous Graph Transformer (HGT) to embed the state, which is represented as a heterogeneous graph of assets, relations, and attack steps.
- Algorithm: Proximal Policy Optimization (PPO).
- Training: Trained on 15 scenarios and tested on 15 unseen scenarios.
Defensive Agent (Defender):
- Goal: Minimize the sum of costs associated with compromised hosts (confidentiality/integrity loss) and the cost of defensive actions (availability loss).
- Architecture: Utilized the Vejde relational reinforcement learning library, which uses Graph Neural Networks (GNNs) to process fact-based state representations.
- Conditions: Two variants were trained: one in a "perfect" environment (no noise) and one in a "noisy" environment (1% FP/FN rates).
- Algorithm: PPO.

Key Contributions

The MAL Simulator: A novel simulation platform that decouples the simulation engine from the target domain via the MAL modeling language. This allows the simulator to adapt to different domains (e.g., IT, Cloud, OT) by changing the language and instance models rather than the source code.
Unified Adversarial Framework: The simulator provides interfaces for both offensive and defensive agents, enabling the training of agents in the same environment and the testing of agents against one another.
Generalization via Scenario Variation: The methodology demonstrates how training on a distribution of scenario variants (different network sizes, attacker entry points, and policies) improves agent robustness to unseen configurations.
Integration with Existing Tools: The simulator offers compatibility with standard ML frameworks (e.g., Gymnasium API) and specific libraries like Vejde and HGT.

Results

Offensive Agent Performance

Efficiency: The RL attacker significantly outperformed heuristic search algorithms (BFS, DFS, Random). The RL agent achieved a mean return of -301 (higher is better, as costs are negative), compared to -717 for BFS and -797 for DFS.
Generalization: The RL agent maintained high performance on unseen test scenarios, with no statistically significant difference in returns between training and testing sets. It successfully minimized the estimated attack cost more effectively than search-based methods.

Defensive Agent Performance

Noise Sensitivity: In "perfect" (noise-free) conditions, a simple heuristic agent (immediately disabling assets upon alert) performed comparably to the RL agent.
Noisy Conditions: Under realistic noisy conditions (1% FP/FN), the heuristic agent's performance degraded significantly. The RL agent trained with noise ("Vejde w/ Noise") induced the lowest defense costs, outperforming both the noise-free RL agent and the heuristic.
Adversarial Testing: When the trained defenders were tested against the RL Attacker (which was not used during defender training), all defender agents suffered a significant drop in performance. The RL attacker was able to bypass defenses more effectively than the search-based attackers the defenders were trained against, highlighting the limitations of training defenders only against heuristic or search-based adversaries.

Significance and Claims

The paper claims that the MAL Simulator successfully addresses the rigidity of previous ACO simulators by leveraging a formal modeling language to adapt to new domains without code changes. The experiments support the claim that:

RL Agents are Effective: Learned policies can outperform manually designed heuristics and search algorithms, particularly in complex or noisy environments.
Adversarial Training is Critical: The significant performance degradation of defenders when facing an RL attacker (as opposed to heuristic attackers) underscores the necessity of testing and training both offensive and defensive agents within the same environment to ensure robustness.
Sim-to-Real Potential: By incorporating uncertainties (noise, varying topologies) and using a data-driven approach based on real network inventory, the simulator facilitates the development of agents that may generalize better to real-world systems, though the authors explicitly note that full "sim-to-real" validation in the physical network was beyond the scope of this specific work.

The authors conclude that the MAL Simulator provides a flexible, publicly available foundation for developing and evaluating automated cyber operation agents, emphasizing the importance of adversarial training to avoid over-optimizing defenses against weak or predictable attackers.