Technical Summary: Mastermind – Strategy-Grounded Learning for Repository-Scale Vulnerability Reproduction

1. Problem Characterization

Repository-scale vulnerability reproduction is a demanding software engineering (SE) task requiring an autonomous agent to inspect a codebase, infer the input grammar for a vulnerable path, construct a proof-of-concept (PoC), and verify that the crash occurs only on the vulnerable build (not the patched one). While recent Large Language Model (LLM) agents have improved in executing commands (navigating repos, editing files, submitting PoCs), they frequently fail due to strategic errors rather than execution errors. Agents often waste rollouts exploring unproductive directions or fail to revise hypotheses based on feedback.

The paper identifies that the primary bottleneck is strategy selection (deciding what to try next) rather than low-level execution capability. Standard approaches like independent sampling (Best-of-N) suffer from diminishing returns, and simply adding more context (e.g., ground-truth patches) does not automatically translate into an executable investigation plan. The authors argue that the "strategy"—a compact, natural-language plan for inspection, input construction, and validation—is the appropriate unit of learning, distinct from the full, noisy, and expensive execution trajectory.

2. Methodology: The Mastermind Framework

Mastermind is a dual-loop framework that separates transferable strategy learning from task-specific experience. It decouples the "Planner" (which decides the approach) from the "Executor" (which performs the actions).

Core Components

• Curator: Maintains task-local experience. It activates relevant prior strategy/outcome records (e.g., file locations, failed hypotheses, parser behaviors) before each attempt and appends new records after verification. This keeps volatile, task-specific facts outside the model weights.
• Planner: A trainable policy that emits a compact, natural-language Strategy. Given the task and curator experience, the Planner describes where to inspect, what input structure to target, and how to validate the PoC. It does not issue shell commands; it chooses the approach.
• Executor: A frozen acting substrate (e.g., GPT-5.5, GPT-5.4 mini, GLM 5.1) that instantiates the Planner's Strategy into repository actions and PoC submissions. Keeping the Executor frozen isolates gains to better planning rather than improved command generation.
• Verifier: Provides executable ground truth via the CyberGym benchmark. It validates rollouts, assigns milestone scores, and routes feedback to both the Curator and the Planner (during training).

Learning Mechanism

Mastermind employs a dual-loop training process:

1. Experience Loop: Task-local evidence (strategies and outcomes) is accumulated in the Curator. Subsequent attempts are conditioned on this concrete, task-specific history.
2. Policy Loop: The Planner is trained using Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO).
   • SFT: Warm-starts the Planner to learn the pattern of revising strategies based on feedback.
   • GRPO: Optimizes the Planner using dense milestone rewards derived from CyberGym. To ensure diversity, the training uses slot-conditioned sampling, where each group of candidate strategies is conditioned on different hypotheses (e.g., "parser-format," "bounds-allocation," "lifetime-state").
   • Reward Function: The reward is based on a milestone hierarchy (0–7) ranging from locating source code to full dual-build reproduction. It includes a penalty for long strategies to discourage full trajectory narration, keeping the learning unit compact.

3. Key Contributions

1. Strategy Bottleneck Identification: The paper demonstrates through controlled experiments that strategy quality causally affects reproduction outcomes. A fixed executor with a "Soft Oracle" strategy achieves a 39.5% pass rate, compared to 23.5% with no strategy, proving that strategy selection is the dominant bottleneck.
2. Mastermind Framework: A novel dual-loop architecture that trains on compact natural-language strategies rather than full execution trajectories. This makes reinforcement learning practical for long-horizon SE agents by decoupling the policy update from the expensive executor.
3. Transferable Learning: The framework shows that a Planner trained on one executor (GPT-5.4 mini) can significantly improve the performance of different, frozen executors (GPT-5.5, GLM 5.1) without retraining, demonstrating that learned strategies are transferable across execution backbones.

4. Experimental Results

The framework was evaluated on CyberGym, a benchmark of 1,507 real-world vulnerability reproduction tasks. The study used 260 tasks for training and 200 held-out tasks for evaluation.

• Performance with GPT-5.5: Mastermind achieved an 84.5% pass rate (169/200 tasks), significantly outperforming:
  • Independent Best-of-8 sampling: 63.0%
  • Iterative task-local improvement (no learned planner): 77.0%
  • Open-book context (Level 3 ground truth): 60.0%
• Transferability: The same Planner improved GPT-5.4 mini from 45.0% to 60.0% and GLM 5.1 from 58.5% to 71.0%.
• Efficiency: Mastermind achieved these results with fewer rollouts and lower computational cost (estimated $8 per pass for GPT-5.5 vs. $29 for Best-of-8) due to early stopping and better strategy selection.
• Ablation Studies: Removing the dual-loop components confirmed that both the learned Planner (Policy loop) and the task-local experience (Experience loop) are necessary for peak performance. The "No dual loop" baseline (independent Best-of-8) achieved only 63.0% on GPT-5.5.

5. Significance and Claims

The paper argues that for repository-scale SE agents, choosing the approach is harder than executing it. Mastermind demonstrates that learning high-level strategies is an effective and transferable mechanism for improving these agents.

• Strategic Abstraction: By treating strategy as the learning unit, the system avoids the prohibitive cost of training on full execution traces while still capturing the reasoning patterns necessary for complex debugging.
• Separation of Concerns: The framework correctly assigns "transferable reasoning patterns" (e.g., when to inspect source) to the Planner's weights and "volatile task-specific facts" (e.g., specific file paths) to the Curator's memory.
• Practical Impact: The results suggest that future SE agents should prioritize strategy-level learning and explicit experience management over merely scaling up execution models or adding more context.

The authors maintain a modest stance, noting that while strategy-level learning is highly effective for vulnerability reproduction, its generalizability to other SE tasks (like maintainability or human interpretability) remains an open question. They also emphasize safety, noting that the framework operates within sandboxed environments and improves strategy selection on existing executors rather than creating new execution capabilities.