🤖 AI

Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries

This survey presents a comprehensive taxonomy and lifecycle framework for dynamic agent skill libraries, analyzing 124 papers to define how reusable procedures evolve through evidence collection, verification, maintenance, and governance while identifying critical gaps in current benchmarks and reporting standards.

Original authors: Yubo Li

Published 2026-07-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yubo Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your favorite video game character, but instead of just having a fixed set of moves, they have a magical backpack that keeps growing. Every time they beat a level, they don't just get a high score; they write down a new "cheat code" or a "secret move" and stuff it into that backpack. This is what Dynamic Agent Skills are all about.

According to this paper, published in July 2026, AI agents (smart computer programs) are moving past just chatting with you. They are starting to do real work: browsing the web, fixing software, and even controlling robots. But here's the problem: if an agent has to figure out how to do a task from scratch every single time, it's slow and clumsy. The solution? A Skill Library. This is a collection of reusable "cheat codes" (like code functions, written instructions, or workflow maps) that the agent can pull out, use, and then improve for next time.

The Backpack That Learns to Organize Itself

The big discovery in this paper is that these backpacks (libraries) can't just be static piles of notes. If you just keep adding new notes without ever cleaning up, the backpack gets so heavy and messy that the agent forgets where anything is.

The authors surveyed 124 papers from 2023 to 2026 and found that the smartest systems treat their skill library like a living, breathing ecosystem. They don't just add things; they have a whole lifecycle:

Evidence: The agent tries something and sees if it worked.
Proposal: It suggests a new skill or an update to an old one.
The Gatekeeper (Verification): This is crucial. Before a new skill enters the library, it has to pass a test. Is it safe? Does it actually work?
Admission: If it passes, it gets in.
Maintenance: This is the part people often forget. The system must prune (cut out) old, useless skills, merge duplicates, and repair broken ones.

The paper argues strongly against the idea that you can just let an agent write skills and never check them. They found that libraries filled with unverified, self-generated skills often perform worse than if the agent had no library at all. It's like letting a chaotic friend write your entire textbook; it might look impressive, but it's full of errors.

The "Flat Library" Trap

One of the most interesting findings is what happens when the library gets too big. The paper measured this carefully. When a library has about 16 to 64 skills, the agent finds what it needs easily. But once you hit around 128 skills, something weird happens: the agent starts getting confused. It's like walking into a library with 100 books on a shelf; you can find what you need. But if you dump 256 books on that same shelf, you spend more time looking for the right one than actually reading it.

The paper suggests that flat retrieval (just searching a giant list) breaks down at this moderate size. The solution isn't just to add more computer power; it's to organize the library better, like using a hierarchy (folders within folders) or a map (graphs) to help the agent navigate.

Who Benefits the Most?

Here's a fun twist: the paper suggests that weaker AI models might actually benefit more from these dynamic skill libraries than the super-smart ones. Think of it like this: a genius student might not need a cheat sheet to pass a test, but a student who is struggling can use a well-organized cheat sheet to catch up and perform surprisingly well. However, the authors are careful to say this is a "deployment-facing hypothesis" based on converging evidence, not a hard-and-fast rule for every single situation.

The Danger Zone: Safety and "Skill Injection"

The paper also sounds a serious alarm about safety. Because these skills are stored as files that the agent reads, they can be hacked.

Prompt Injection: A bad actor could write a skill that looks helpful but secretly tells the agent to steal passwords or delete files.
Supply Chain Poisoning: Just like downloading a virus with a free app, an agent could download a "skill package" that is actually malicious.
The "Harmful but Valid" Skill: A skill might be perfectly written and useful, but it could be used to do something dangerous (like a recipe for a bomb).

The paper argues that safety can't just be an afterthought. It needs to be part of the lifecycle. You need scanners to check skills before they enter the library, and you need a way to "roll back" (undo) a skill if it starts causing trouble later.

What's Still a Mystery?

The paper is very honest about what we don't know yet.

Scaling: We know flat libraries break around 128 skills, but we don't have a perfect solution for libraries with 10,000 to 1,000,000 skills yet. We have hints that graphs and hierarchies help, but it's not a solved problem.
The "Echo Chamber" Risk: There's a worry that if an AI keeps teaching itself from its own notes, it might start repeating its own mistakes over and over, getting worse instead of better. The paper calls this "parametric collapse" and says we need to test this over very long periods to see if it actually happens.
Portability: Moving a skill from one AI agent to another is tricky. It's like trying to use a tool designed for a specific robot arm on a completely different robot; the parts might not fit.

The Bottom Line

The main takeaway is that skills are not just static tools; they are evolving artifacts. To build truly smart agents, we need to stop treating skill libraries like a simple list of instructions and start treating them like a managed database that requires constant cleaning, verifying, and organizing.

The authors suggest that the future of AI isn't just about making the brain bigger; it's about building a better backpack and teaching the agent how to keep it tidy. They propose a new set of rules for how we should test these systems: don't just look at the final score; look at how the library grew, how many mistakes were fixed, and how often the agent had to throw things away.

In short: Don't just let the AI write; let it edit, verify, and clean up its own notes. That's the secret to making it truly useful.

Technical Summary: Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries

Problem Statement

Large Language Model (LLM) agents are transitioning from chat-based assistance to operational workflows involving web browsing, software engineering, and domain-specific tasks. The primary bottleneck in these settings is not the model's ability to reason about a single task, but its capacity to reuse procedural knowledge across recurring tasks, tools, and users. While early systems treated "skills" as static prompt snippets or fixed tool collections, recent literature (2023–2026) has shifted toward dynamic skill libraries: external, editable artifact stores that evolve as agents accumulate evidence.

The field currently suffers from a lack of shared vocabulary and evaluation standards. The term "skill" is overloaded, referring to structurally distinct artifacts ranging from executable code and natural language (NL) heuristics to parametric adapters and memory traces. Furthermore, most benchmarks report only terminal task success, obscuring critical lifecycle phenomena such as skill inflation, retrieval degradation as libraries grow, and the gap between skill usage and actual utility. This paper addresses the need to treat skill libraries not as static repositories, but as learning systems where the artifact store itself ( $L_t$ ) changes over time ( $L_t \to L_{t+1}$ ) through evidence-driven updates.

Methodology

The authors conducted a systematic audit of 124 papers published between 2023 and May 31, 2026. The corpus includes method papers, benchmarks, infrastructure studies, and safety analyses. The review protocol excluded classical option discovery and generic tool-use papers unless they produced externally invocable skill artifacts or evaluated their lifecycle.

The survey is structured around three primary analytical tools:

A Six-Sense Taxonomy: Distinguishing the structural forms of "skills" (Table 1).
A Lifecycle Architecture: An eight-stage framework for how dynamic systems manage artifacts.
A Lightweight Notation: A skill-record schema and a ten-operator vocabulary to standardize comparisons of library updates.

The Six Senses of "Skill"

The paper categorizes skills into six distinct senses based on five dynamic properties (executability, editability, portability, inspectability, and verification handles):

Executable Code: Programs or contracts (e.g., Voyager, LATM, SAGE).
NL Heuristics: Procedural lessons or critiques (e.g., ERL, RetroAgent).
SKILL.md Packages: Progressive disclosure directories containing metadata, instructions, and code (e.g., MetaClaw, Trace2Skill).
Parametric Skills: Weight deltas or adapters (e.g., SkillsCrafter, SKILL0).
Memory/Trajectory: Episodic traces with invocable interfaces (e.g., SimpleMem).
Capability Labels: Declarative roles (boundary cases).

The Lifecycle Architecture

The paper proposes a reference architecture for dynamic skill systems comprising eight recurring stages:

Evidence Acquisition: Gathering trajectories, rewards, or failure traces.
Proposal: Converting evidence into candidate artifacts or edits.
Verification & Admission: Deciding if a candidate enters the library (the critical selection mechanism).
Organization & Storage: Assigning artifacts to flat indices, hierarchies, or graphs.
Retrieval & Composition: Selecting skills for future actions.
Maintenance & Repair: Pruning, merging, or refining artifacts to prevent degradation.
Distillation & Portability: Moving knowledge between external artifacts and model weights.
Governance & Provenance: Auditing lineage, authorship, and rollback capabilities.

Notation and Operators

The authors introduce a skill-record schema $S_t = \langle C_t, \pi_t, T_t, R_t, \phi_t, \nu_t, \prec_t \rangle$ , where $\phi_t$ represents the edit operator, $\nu_t$ the verification handle, and $\prec_t$ the lineage relation. Library transitions are defined by a vector of operators $\vec{u}_t$ drawn from a ten-element vocabulary: Add, Refine, Merge, Split, Prune, Distill, Abstract, Compose, Rewrite, and Rerank.

Key Contributions

Conceptual Clarification: The six-sense taxonomy resolves ambiguity by mapping diverse "skill" implementations to their structural properties and dynamic capabilities.
Lifecycle Framework: The paper reframes dynamic skills as editable learning-system artifacts, providing a unified architecture for comparing systems that differ in which lifecycle stages they implement.
Standardized Vocabulary: The ten-operator vocabulary allows for precise comparison of update mechanisms without elevating specific methods into separate contributions.
Evidence-Graded Patterns: The survey synthesizes seven recurring patterns supported by varying levels of evidence (from controlled ablations to architectural corroboration).
Safety and Governance Analysis: The paper identifies eight specific safety surfaces unique to dynamic skill stores, treating safety as a lifecycle stage rather than a post-hoc filter.

Key Findings and Evidence-Graded Patterns

The survey distills seven patterns, graded by evidence strength (A–D):

Admission Gates Matter (Grade A/B): Curated skills that pass verification (e.g., execution tests, judge-LLM rubrics) significantly outperform unverified self-generated skills. Libraries that admit any proposal suffer from redundancy and drift.
Verifier Quality in RL (Grade B): In skill-aware Reinforcement Learning, the quality of the verifier or reward-shaping signal is often more decisive than the RL algorithm itself.
Flat Retrieval Degradation (Grade B/C): Flat retrieval performance drops sharply at moderate library sizes (roughly 64–128 skills) due to signal-to-noise collapse. Hierarchical or graph-based storage mitigates but does not eliminate this.
Weaker Backbone Gains (Grade C): Weaker base models often show larger relative gains from dynamic skills than stronger models, though this depends on library content and routing reliability.
Focused Libraries (Grade B/C): Libraries curated to narrow task distributions often outperform comprehensive ones, as focused libraries reduce distractor load.
Maintenance is Load-Bearing (Grade B): At moderate-to-large scales, systems with negative operators (Prune, Merge) outperform monotonic-growth systems. Maintenance cost amortizes over time, but short horizons may not justify it.
Write-Time Abstraction (Grade B/C): Abstraction and distillation performed during skill authoring (write-time) generally yield stronger performance than read-time summarization alone.

Safety and Governance

The paper highlights that dynamic skills introduce a "software supply chain" problem. Key safety surfaces include:

Prompt Injection: Malicious instructions embedded in skill documentation or code.
Supply-Chain Poisoning: Vulnerabilities in public registries (e.g., 26.1% of skills in one study contained vulnerabilities).
Credential Leakage: Secrets exposed through the interaction of text, code, and stdout.
Attribution Loss: The inability to trace which artifact contributed to a decision after merges or splits.
The authors argue that defensible deployments require primitives like admission-time scanners, request-conditioned invocation audits, and operator-granular provenance logs.

Significance and Limitations

Significance:
The paper claims to provide the first unified framework for treating skill libraries as evolving learning objects rather than static prompt collections. By establishing a common language (taxonomy, operators, lifecycle stages), it enables the comparison of heterogeneous systems and identifies specific gaps in current research, particularly regarding maintenance, safety, and benchmark realism. It argues that the field must move beyond "skill usage" metrics to "skill utility" and "library trajectory" metrics.

Limitations:
The authors explicitly note several limitations:

Non-Exhaustive Corpus: The survey relies on snowballing and arXiv searches; unpublished systems or non-English papers may be missed.
Heterogeneous Evidence: Comparisons are architectural rather than causal; pooled effect sizes are impossible due to varying backbones, contexts, and evaluators.
Unresolved Scaling: While hierarchy and graphs delay retrieval degradation, the paper does not claim to have solved scaling to marketplace sizes (10,000+ skills).
Modest Claims: The survey does not propose a new algorithm or a single "best" verifier but rather organizes existing design patterns to guide future research.

Open Problems

The paper concludes with eight open problems, including:

Compositional verifiers for code skills.
Admission mechanisms under non-stationary task distributions.
Principled maintenance schedules derived from cost models.
Preventing parametric collapse under repeated distillation.
Retrieval scaling beyond moderate library sizes.
Cross-library skill portability and compatibility modeling.
Integrated governance, provenance, and attribution systems.
Benchmarks that honestly report library trajectories and usage-vs-utility gaps.

The authors emphasize that future progress requires community coordination on infrastructure, governance, and trajectory-aware benchmarks, rather than isolated algorithmic improvements.