Technical Summary: OPENSKILL: Open-World Self-Evolution for LLM Agents

1. Problem Definition: Open-World Self-Evolution

Current self-evolving LLM agents typically rely on a "usable learning loop" that presupposes access to curated skills, successful execution trajectories, or explicit verifier signals (e.g., rewards, ground-truth answers) during the learning phase. However, real-world open-world deployments often provide only a seed task prompt and access to external resources, lacking initial skills or supervision signals.

This paper defines Open-World Self-Evolution as a setting where an agent must:

1. Start with only a task instruction ($I_i$) and an execution environment ($E_i$).
2. Access open-world resources (documentation, repositories, web pages) but exclude hidden target answers, rewards, or verifier outputs.
3. Construct both its skills (reusable behavioral artifacts) and its own verification signals from scratch.
4. Improve performance without target-task supervision during the evolution process, reserving ground-truth evaluation solely for final assessment.

The core challenge is that facts alone are insufficient; the agent must infer what to learn and build a coupled learning loop where the same open-world evidence supplies both the knowledge to be learned and the supervision for refining that knowledge.

2. Methodology: The OpenSkill Framework

OpenSkill is a three-stage pipeline designed to bootstrap this self-evolution loop without leaking target-task supervision. It operates under a strict "leakage barrier" that prevents ground-truth ($T^{GT}_i$) from entering the skill construction phase.

Stage 1: Open-World Knowledge Acquisition

The agent queries open-world resources to acquire two distinct types of knowledge:

• Skill Construction Knowledge ($k_i$): Retrieves background concepts, best practices, API documentation, and citations relevant to the task.
• Verification Knowledge ($k^v_i$): Retrieves independently verifiable anchors, such as reference values from official docs, statistical invariants of known datasets, cross-validation procedures, and domain standards.
• Skill Planning: A structured plan ($p_i$) is synthesized, defining the skill architecture, key procedures, and domain rules.
• Leakage Prevention: All queries are filtered to exclude benchmark names and identifiers that could lead to target answers.

Stage 2: Leakage-Free Skill Evolution

The agent iteratively generates and refines skills using a Virtual Verifier:

• Virtual Test Generation: Instead of using ground-truth tests, the system generates a virtual test suite ($\tilde{T}_i$) grounded in the verification knowledge ($k^v_i$). These tests consist of deterministic assertions (e.g., checking row counts, metric ranges, or output formats) that do not require knowing the target answer.
• Iterative Refinement: The agent executes the task with the current skill set. If the virtual pass rate ($\tilde{r}$) is less than 1, a Diagnostic Retriever analyzes failures to distinguish between:
  • Implementation Bugs: Fixed via self-correction.
  • Knowledge Gaps: Triggering targeted retrieval of new open-world knowledge.
• Termination: The loop continues until the virtual pass rate reaches 1.0 or a maximum iteration budget ($J=3$) is exhausted. The final skill set ($\hat{S}^*_i$) is the state of the skill at loop termination.

Stage 3: Zero-Shot Target Evaluation

The refined skill artifact is deployed to a target agent (which may differ from the construction agent) in a zero-shot setting. Performance is measured against the hidden ground-truth test suite ($T^{GT}_i$), which was never accessed during the evolution process.

3. Key Contributions

1. Definition of Open-World Self-Evolution: The paper formalizes a setting where agents must build skills and verification signals entirely from open-world resources without target-task supervision.
2. The OpenSkill Framework: A novel pipeline that bootstraps self-evolution by synthesizing transferable skills and a self-built virtual verifier grounded in open-world facts rather than hidden answers.
3. Empirical Validation: Demonstration that OpenSkill achieves superior automated pass rates across multiple benchmarks and model families while adhering to the no-supervision constraint.

4. Experimental Results

The framework was evaluated on three benchmarks (SkillsBench, SocialMaze, ScienceWorld) using two target agents (Opus 4.6 and GPT 5.2).

• Performance: OpenSkill achieved the best automated overall pass rate on all benchmarks. On SkillsBench, it reached 43.6% (Opus 4.6) and 42.1% (GPT 5.2), outperforming the strongest closed-world baselines (Skill-Creator and CoT) by +8.9 and +8.8 percentage points, respectively. It consistently approached the human-curated upper bound.
• Cross-Model Transferability: Skills generated by OpenSkill using Opus 4.6 were deployed directly onto weaker models (Haiku 4.5, Qwen 3 Coder, DeepSeek V3, Mistral Large 3) without retraining. OpenSkill-generated skills consistently yielded the highest rewards across all target models, improving over the no-skill baseline by 5.5%–14.8%.
• Verifier Quality: The self-built virtual verifier, despite never accessing ground-truth tests, achieved 80.5% recall and 60.7% agreement with ground-truth outcomes. It covered 88.9% of ground-truth test intents across sampled tasks.
• Ablation Studies:
  • Iteration Count: Performance peaked at 3 refinement iterations; excessive iterations led to overfitting to virtual feedback.
  • Component Contribution: Both the open-world query module and the virtual verifier provided significant, complementary gains over a parametric-only baseline.

5. Significance and Claims

The paper claims that OpenSkill demonstrates a viable path toward agents that can continue improving after deployment in environments where curated skills and reliable feedback are unavailable. By decoupling skill acquisition from target-task supervision and anchoring verification in independently verifiable open-world facts, OpenSkill enables:

• Scalability: Skills are explicit artifacts, not model weights, allowing them to be transferred across different model families.
• Groundedness: Skills are derived from external documentation and repositories, mitigating reliance on parametric knowledge which may be outdated or insufficient for niche domains.
• Supervision-Free Learning: The framework proves that a self-improvement loop can be constructed using only a task prompt and open-world resources, without needing a "gold answer" during the training phase.

The authors acknowledge limitations, including potential noise in open-world sources, the risk of virtual tasks being too easy (overestimating skill quality), and the computational cost of open-world research compared to closed-world generation. However, the results suggest that open-world acquisition combined with leakage-free verification is a robust mechanism for autonomous agent evolution.