Technical Summary: Socratic-SWE

Problem Statement

Large Language Model (LLM)-driven software engineering (SWE) agents have emerged as a critical benchmark for real-world intelligence, requiring long-horizon interaction with code repositories for bug localization, repair, and verification. However, the training of these agents is bottlenecked by the scarcity of high-quality SWE tasks. Existing synthetic data methods typically rely on static mutation, bug injection, or fixed rewriting procedures. These approaches generate task distributions that are independent of the agent's own training progress and specific weaknesses, leading to a misalignment between the curriculum and the model's actual capability gaps. Furthermore, while reinforcement learning (RL) training produces valuable "solving traces" (records of agent behavior, tool use, and failures), current methods discard these traces after extracting scalar rewards, failing to utilize them to evolve the training curriculum.

Methodology: Socratic-SWE

The authors propose Socratic-SWE, a closed-loop, self-evolving framework that transforms historical solving traces into structured training signals. The system operates through a co-evolutionary loop involving a shared policy $\pi_\theta$ that alternates between two roles: a Generator and a Solver.

1. Agent Skill Registry (Trace Distillation)

Instead of discarding traces, Socratic-SWE distills them into an Agent Skill Registry ($S$). This process involves three stages:

• Trace Collection: The Solver interacts with seed repositories, generating a corpus of traces ($T$), including both successful and failed trajectories.
• Skill Extraction: A distillation model ($M_{distill}$) analyzes these traces to identify recurring behavioral patterns. For successful traces, it extracts generalizable strategies; for failed traces, it summarizes failure modes and corrective principles.
• Registry Construction: Candidate skills are deduplicated and filtered to form a structured registry. Each skill is a structured document containing a name, description, applicability conditions, and an ordered list of operations.

2. Skill-Guided Task Generation

The Generator role uses the Agent Skill Registry to construct targeted repair tasks in real code repositories.

• Adaptive Proposals: The Generator samples a skill $s \in S$ and a repository $r$ to propose a task $\tau$ and verification signal $v$. This process is conditioned on evidence ($E_t$) collected from the Solver's previous performance, ensuring tasks target current capability gaps.
• Staged Validation: Candidate tasks undergo a four-stage execution-based validation pipeline in a sandboxed environment:
  1. Format: Syntactic validity.
  2. Grounding: Verification that task artifacts exist in the repository.
  3. Execution: Stability and reproducibility of the verification suite.
  4. Semantics: Confirmation that the task distinguishes between failing and repaired states.
• Gradient-Aware Reward: To ensure tasks are not just valid but useful, the Generator is rewarded based on solver-gradient alignment. The system computes a target gradient direction ($G_v$) from a held-out set of trusted validation tasks. The reward for a candidate task is the cosine similarity between the task-induced solver gradient ($g_\tau$) and $G_v$. This prioritizes tasks that drive the solver in a direction consistent with trusted improvements.

3. Repository Repair Solver

The Solver role trains on the accepted curriculum of tasks. It interacts with the repository sandbox to generate patches. The reward function ($r_S$) is multi-component, rewarding:

• Full suite pass (all tests passing).
• Partial repair rates.
• Regression avoidance (preserving originally passing tests).
  The Solver is optimized using GDPO (Grouped Direct Policy Optimization) to handle the heterogeneous reward components effectively.

Key Contributions

1. Trace-Driven Self-Evolution Paradigm: The paper demonstrates that solving traces, typically discarded after reward computation, can serve as a reusable substrate for self-evolving agents. Socratic-SWE converts historical behavior into structured skills that guide a closed trace-skill-task loop.
2. Gradient-Aware Curriculum Optimization: The authors introduce a curriculum mechanism that combines skill-conditioned generation with execution-based validation and a solver-gradient alignment reward. This ensures generated tasks are both verifiable and aligned with trusted improvement directions.
3. Superior Empirical Performance: The framework achieves state-of-the-art results among self-evolving baselines under identical compute budgets, demonstrating strong self-evolution and transfer capabilities.

Experimental Results

The framework was evaluated on four benchmarks: SWE-bench Verified, SWE-bench Lite, SWE-bench Pro, and Terminal-Bench 2.0. The experiments used a fixed compute budget of 36,000 training instances (12k per iteration over 3 iterations) and compared against five self-evolving baselines (SPIRAL, R-Zero, Absolute-Zero, Socratic-Zero, SSR) and a base agent.

• Performance: After three iterations, Socratic-SWE achieved 50.40% on SWE-bench Verified, a +7.80 point improvement over the base agent and +3.40 over the strongest baseline (SSR). It also reached 36.67% on Lite, 22.85% on Pro, and 14.61% on Terminal-Bench 2.0.
• Scaling: Unlike baselines that plateau or regress after 2 iterations, Socratic-SWE showed consistent gains through the third iteration. Extended scaling to 5 iterations showed it reaching a higher ceiling (52.00%) compared to SSR (48.00%).
• Ablation Studies: Removing the Skill Registry caused the largest performance drop (-4.20 points), confirming that skill-guided curriculum design is the primary driver. Replacing trace distillation with manual skills or using standard GRPO instead of GDPO also resulted in significant degradation.

Significance and Claims

The paper claims that Socratic-SWE provides a scalable substrate for self-evolving SWE agents by leveraging the agent's own solving traces to continuously adapt its training curriculum. The authors argue that this approach overcomes the limitations of static, model-agnostic data synthesis by creating a dynamic, model-aware loop where:

• Traces are transformed into skills.
• Skills guide the generation of targeted tasks.
• Generated tasks produce new traces for further evolution.

The results suggest that solving traces contain sufficient signal to drive sustained self-improvement without external annotation, effectively delaying the saturation of learning curves observed in other self-play methods. The framework is particularly effective because it utilizes the rich, diagnosable failure modes inherent in SWE traces, combined with deterministic execution-based verification.

Limitations

The authors acknowledge several limitations:

• Closed-World Setting: The evaluation relies on a fixed pool of seed repositories. As the Skill Registry becomes more complete, the system faces diminishing returns due to redundant training signals.
• Validation Dependence: The method relies on a held-out validation set to define the generator-gradient alignment reward, which may limit robustness if the validation distribution does not represent the target deployment setting.
• Execution Requirements: The approach assumes executable verification and sandboxed interaction, meaning gains may not directly transfer to settings lacking reliable tests or deterministic execution.
• Scope: Evaluation is currently limited to specific SWE and terminal-agent benchmarks; broader transfer to other languages or workflows remains to be established.