Training Versus Hiring in Security Operations Centers: Agent-Based and Reinforcement Learning Simulations of Training Duration Effects on CyberTeam Performance

Technical Summary: Training Versus Hiring in Security Operations Centers

Problem Statement
Security Operations Centers (SOCs) face a critical resource allocation dilemma: should organizations invest in training existing personnel or recruit additional staff? While prior computational research has examined static team composition (structure, size, and cohesion) using agent-based modeling, it has largely failed to address the temporal dynamics of team development. Existing studies measure performance at single time points, leaving a gap in understanding how training duration affects learning curves, the point of diminishing returns, and the comparative effectiveness of trained small teams versus larger untrained teams. This study addresses these gaps by incorporating temporal dynamics into the Collaborative Cyber Team Formation (CCTF) framework to simulate how cyber teams develop strategies through sustained training.

Methodology
The authors employed agent-based modeling using NetLogo, enhanced with Reinforcement Learning (RL) to simulate human team learning trajectories. The study did not investigate RL as a subject of improvement but utilized it as a modeling tool to compress months or years of human team practice into thousands of computational episodes.

• Simulation Environment: The environment models a cyber-attack and defense scenario on scale-free (Barabási-Albert) 60-node networks. Attackers aim to compromise a central high-degree router, while defenders protect the network. The simulation runs for up to 1,000 ticks per episode, terminating upon central compromise or timeout.
• Agent Types:
  • Untrained (Baseline): Heuristic agents following fixed, non-adaptive rules (e.g., scouts gather intelligence, exploiters attack known vulnerabilities). These represent teams without sustained training.
  • Trained (RL): Agents utilizing Rainbow Deep Q-Network (DQN) architecture. These agents learn through experience, receiving rewards for effective actions (infection, central compromise) and penalties for redundant actions. They share policies within roles (scouts share one policy, exploiters share another) to simulate collective tactical knowledge.
• Training Protocol: The RL agents were trained for up to 5,000 episodes using a progressive curriculum that increased difficulty (from "very easy" to "mixed/hard") at episodes 200 and 750.
• Evaluation: Performance was measured at specific checkpoints (100, 500, 1,000, 2,000, and 5,000 episodes) across a 12-cell environment grid varying network vulnerability and defender configurations. Key metrics included win rate (strategic effectiveness) and time-to-win (tactical speed).
• Analysis: The study calculated Return on Investment (ROI) based on win-rate gains per episode and determined Equivalent Baseline Team Size (EBTS)—the number of untrained attackers required to match the win rate of a trained team.

Key Contributions

1. Characterization of Learning Trajectories: The study maps the non-linear learning curve of cyber teams, identifying three distinct phases of development.
2. Quantification of Optimal Investment: It identifies the specific training duration that yields the highest marginal returns, providing a data-driven basis for workforce planning.
3. Effectiveness Equivalence: It establishes quantitative thresholds comparing the efficiency of trained small teams against larger untrained teams, offering evidence for "training vs. hiring" decisions.

Results
The computational experiments yielded three primary findings:

1. Non-Linear Learning Trajectory: Performance improves through three phases:

   • Exploration (100–500 episodes): Minimal improvement; win rates range from 0.00 to 0.15. Teams are discovering basic mechanics but lack stable strategies.
   • Skill Acquisition (500–2,000 episodes): Rapid gains; win rates jump from ~0.15 to ~0.57. This is the period of highest learning velocity.
   • Refinement (2,000–5,000 episodes): Continued improvement but with diminishing returns; win rates plateau between 0.93 and 0.95.

2. Optimal Training Investment: The peak ROI occurs between 500 and 1,000 episodes, where teams gain approximately 6.7–8.3 percentage points in win rate per 100 episodes. Beyond 2,000 episodes, marginal gains drop significantly (by ~88–94% compared to the peak), though performance continues to improve slightly.

3. Efficiency Advantages (Trained vs. Untrained): Trained teams achieve substantial efficiency advantages. A team of 5 trained attackers (plus 3 scouts) achieves performance levels equivalent to untrained teams requiring 15 to 35 attackers.

   • In favorable conditions, 5 trained attackers perform equivalently to roughly 25–30 untrained attackers.
   • This represents a 3× to 7× efficiency multiplier, meaning a single trained attacker performs the work of 3 to 7 untrained personnel.
   • Consequently, a full 5,000-episode training program for a small team can substitute for the recruitment of roughly 20–25 additional personnel.

Significance and Claims
The paper claims to provide quantitative evidence for cybersecurity workforce planning, moving beyond static structural analysis to dynamic learning processes. The authors assert that:

• Training Duration is Critical: Organizations that stop training too early (before the 500-episode mark) forfeit high-yield improvements, while those training indefinitely past 2,000 episodes experience diminishing marginal returns.
• Training as a Substitute for Hiring: The substantial efficiency gains (3–7×) support training-focused workforce strategies over recruitment-focused expansion. The study suggests that investing in sustained training allows organizations to maintain high effectiveness with smaller teams, potentially reducing the need for large-scale recruitment.
• Methodological Innovation: The paper demonstrates that Reinforcement Learning can serve as a valid modeling tool to simulate human team learning trajectories that would otherwise require prohibitively expensive or time-consuming longitudinal human subject experiments.

The authors conclude that recognizing the distinction between structural factors (team size) and temporal factors (training duration) is essential for informed resource allocation. They suggest that developing expertise through sustained training may be a more sustainable strategy for maintaining cyber defense capabilities than attempting to match adversary sophistication solely through workforce expansion.