🤖 AI

SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

This paper presents a comprehensive Systematization of Knowledge (SoK) on agentic skills, introducing dual taxonomies for their design and representation, analyzing critical security risks illustrated by the ClawHavoc campaign, and evaluating their impact on agent reliability to outline future challenges for robust, verifiable autonomous systems.

Original authors: Yanna Jiang, Delong Li, Haiyu Deng, Baihe Ma, Xu Wang, Qin Wang, Guangsheng Yu

Published 2026-02-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yanna Jiang, Delong Li, Haiyu Deng, Baihe Ma, Xu Wang, Qin Wang, Guangsheng Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a brilliant but very forgetful personal assistant (the AI Agent). This assistant is incredibly smart and can read, write, and reason better than almost anyone. However, they have a major flaw: they have no long-term memory for how to do things.

Every time you ask them to fix a leaky faucet, bake a cake, or debug a computer program, they have to figure out the entire process from scratch, as if they've never done it before. They might get it right once, but if you ask again tomorrow, they have to re-learn the steps. It's inefficient, slow, and prone to mistakes.

This paper is about giving that assistant a "Skill Library."

Instead of asking the AI to "think" through every single step every time, we give it a collection of pre-packaged, reusable instructions called Agentic Skills. Think of these not as simple tools (like a hammer), but as recipes or standard operating procedures that the AI can pull off the shelf, read, and execute perfectly.

Here is a breakdown of the paper's key ideas using everyday analogies:

1. What is a "Skill"? (The Recipe Card)

In the past, AI agents just used "tools" (like a search engine or a calculator). A tool is like a single hammer: you hit a nail, and it's done.
A Skill is a full recipe.

The Tool: "Hammer."
The Skill: "How to build a birdhouse." This includes: Check if you have wood (Condition), Nail the pieces together in this order (Policy), Stop when the roof is on (Termination), and Hand me the finished box (Interface).

The paper argues that for AI to be truly useful, it needs these full recipes, not just hammers.

2. The Lifecycle: How Skills Grow Up

The paper maps out the entire life of a skill, from birth to retirement, like a plant:

Discovery: Noticing that you keep doing the same thing (e.g., "I always have to format this report").
Practice: Trying it out, failing, and fixing the recipe until it works.
Distillation: Writing the final, perfect recipe down so it's short and clear.
Storage: Putting the recipe in a library (a digital cookbook).
Retrieval: When you need to format a report, the AI looks up the recipe.
Execution: The AI follows the recipe.
Update: If the software changes, the recipe is updated or thrown away.

3. The Seven Ways to Organize Skills (Design Patterns)

The authors found that systems organize these recipe books in seven different ways:

The Menu (Metadata): The AI sees a list of skill names. It only reads the full recipe if it decides it needs that specific dish. (Saves memory).
The Code Script: The skill is a computer program. It's precise and reliable, like a robot arm following exact coordinates.
The Strict Manager: The skill forces the AI to follow a rigid order (e.g., "You must test the code before you save it"). No skipping steps allowed.
The Self-Improving Library: The AI writes its own new recipes based on what it learns. (Risky, but scalable).
The Hybrid: A mix of human-readable instructions and computer code.
The Meta-Skill: A "super-recipe" that teaches the AI how to write new recipes.
The App Store: A marketplace where people share skills. You can download a "Tax Filing Skill" from a stranger.

4. The Big Danger: The "ClawHavoc" Attack

This is the scary part of the paper. The authors analyzed a real-world disaster called ClawHavoc.

The Scenario: Imagine a popular "App Store" for AI skills (called ClawHub).
The Attack: Hackers uploaded 1,200 fake "skills." They looked like helpful tools (e.g., "Crypto Wallet Manager"), but they were actually viruses.
The Result: When users downloaded these skills, the AI agents didn't just fail; they stole passwords, drained crypto wallets, and leaked private data.
The Lesson: Because AI agents can execute code and instructions, a bad skill is like giving a stranger the keys to your house and telling them, "Here, use this key to clean the kitchen." They might clean the kitchen, but they'll also steal your jewelry.

The paper proposes a Trust Tier System to fix this:

Tier 1: Just look at the name.
Tier 2: Read the instructions (but don't let them touch anything).
Tier 3: Let them do small tasks, but ask for permission first.
Tier 4: Let them run wild (only for skills you trust 100%).

5. The Golden Rule: Curated vs. Self-Made

The paper did a massive experiment to see what works best.

Curated Skills (Human-made): When humans wrote the recipes, the AI got 16% better at its jobs.
Self-Generated Skills (AI-made): When the AI tried to write its own recipes, it actually got worse (by about 1%).

The Analogy: It's like a student trying to write their own textbook. Sometimes they get it right, but often they make up rules that don't exist. A human expert writing the textbook is much more reliable.

Summary

This paper is a guidebook for building reliable AI assistants.

The Good: Giving AI "Skills" (recipes) makes them faster, smarter, and more consistent.
The Bad: If you let anyone upload skills (like an App Store), hackers can poison them.
The Solution: We need strict rules, human oversight, and a way to verify that a skill is safe before the AI uses it.

In short: Don't just give your AI a hammer. Give it a toolbox, but make sure the toolbox is locked, and only you hold the key.

1. Problem Statement

Large Language Model (LLM) agents have evolved from single-turn question answering to multi-step autonomous systems. However, a fundamental inefficiency persists: procedural knowledge is ephemeral. Agents currently re-derive execution strategies from scratch for every new task instance, even if they have solved similar problems before. This "context window amnesia" prevents agents from retaining learned procedures, leading to inconsistent performance, high token consumption, and an inability to scale reliably across long-horizon workflows.

While existing research focuses on tools (atomic primitives) or plans (one-time reasoning scaffolds), there is a lack of systematic understanding of Agentic Skills: reusable, callable modules that encapsulate procedural knowledge with explicit applicability conditions, termination criteria, and governance. The paper addresses the gap in defining, categorizing, and securing these skills across their full lifecycle.

2. Methodology

The authors conducted a Systematization of Knowledge (SoK) study using a rigorous systematic review process:

Literature Search: Searched six major databases (Google Scholar, arXiv, IEEE, etc.) from Jan 2020 to Feb 2025, supplemented by a concurrent 2026 benchmark (SkillsBench).
Corpus: Selected 65 papers for detailed analysis, with 24 systems analyzed in depth.
Taxonomy Development: Used an iterative bottom-up approach to derive design patterns and an orthogonal representation $\times$ scope taxonomy.
Case Studies: Anchored security and evaluation analyses with real-world data, specifically the ClawHavoc supply-chain attack and the SkillsBench benchmark.

3. Key Contributions

A. Formal Definition of Agentic Skills

The paper defines an agentic skill as a four-tuple $S = (C, \pi, T, R)$ :

$C$ (Applicability Condition): A predicate determining if the skill is appropriate for the current context ( $O \times G$ ).
$\pi$ (Executable Policy): The logic mapping observations to actions (can be NL, code, or learned policy).
$T$ (Termination Condition): Specifies when the skill completes successfully or fails.
$R$ (Reusable Interface): The metadata contract (name, params, returns) allowing programmatic invocation.

This distinguishes skills from tools (atomic, no internal logic), plans (ephemeral, non-reusable), and memory (declarative facts vs. procedural "know-how").

B. The Skill Lifecycle Model

The authors map the skill journey through seven stages:

Discovery: Identifying recurring patterns.
Practice/Refinement: Iterative improvement via trial-and-error.
Distillation: Compressing trajectories into compact representations.
Storage: Persisting with indexing and versioning.
Retrieval/Composition: Selecting and chaining skills at runtime.
Execution: Running within sandboxed constraints.
Evaluation/Update: Monitoring performance and retiring obsolete skills.

C. Seven Design Patterns

The paper categorizes how skills are packaged and executed into seven patterns:

Metadata-Driven Progressive Disclosure: Loading full instructions only upon selection (token efficient).
Code-as-Skill: Executable scripts (Python/Shell) offering determinism.
Workflow Enforcement: Hard-gated processes (e.g., Test-Driven Development) to prevent hallucination.
Self-Evolving Libraries: Automated generation and validation of new skills (high risk of "skill debt").
Hybrid NL+Code Macros: Combining human-readable logic with executable steps.
Meta-Skills: Skills that generate or modify other skills.
Plugin/Marketplace Distribution: Community-driven distribution (e.g., OpenClaw, GPT Store).

D. Representation $\times$ Scope Taxonomy

Representation: Natural Language, Code, Tool Macros, Learned Policies, and Hybrids.
Scope: Single-tool, Multi-tool, Web, OS/Desktop, Software Engineering, and Robotics.
Insight: Most current systems cluster in "Code + Self-Evolving" for Game/SWE domains, leaving large gaps in Policy-based skills and formal workflow enforcement.

E. Security and Governance Analysis

The paper introduces a Trust-Tiered Execution Model (T1: Metadata only $\to$ T4: Autonomous execution) and analyzes six threat vectors, including poisoned retrieval, malicious payloads, and "confused deputy" attacks.

Case Study (ClawHavoc): Analyzed a campaign where 1,184 malicious skills infiltrated the OpenClaw marketplace. These skills exfiltrated API keys, crypto wallets, and browser credentials. The attack exploited metadata poisoning (Pattern 1) and code injection (Pattern 2), highlighting that traditional binary scanners fail to detect NL-based prompt injections within skill policies.

F. Evaluation Framework

The authors propose evaluating skills on Correctness, Robustness, Efficiency, Generalization, and Safety.

Key Finding (SkillsBench): Curated (human-authored) skills improved agent pass rates by +16.2 percentage points on average. Conversely, self-generated skills degraded performance by -1.3 pp, indicating that current models cannot reliably author the procedural knowledge they need.
Domain Variance: Skills provided massive gains in Healthcare (+51.9 pp) and Manufacturing, but minimal gains in Software Engineering (+4.5 pp), suggesting skills are most valuable where pre-training data lacks procedural grounding.

4. Results and Significance

Paradigm Shift: The paper establishes "Agentic Skills" as a distinct, critical layer between tools and agents, moving beyond simple tool calling to reusable, governable procedural modules.
Quality vs. Scalability Trade-off: The research highlights a critical tension: Human curation yields high-reliability skills but does not scale, while Self-evolving libraries scale but currently degrade performance due to lack of verification.
Security Imperative: The ClawHavoc case study proves that skill marketplaces are high-value attack surfaces. The paper argues that governance must move beyond simple code scanning to tuple-level auditing (checking $C, \pi, T, R$ ) and runtime behavioral monitoring.
Compute Equalization: The study suggests that smaller models equipped with curated skills can outperform larger models without them, positioning skills as a cost-effective alternative to scaling model parameters.

5. Conclusion

The paper concludes that for LLM agents to become robust, verifiable, and certifiable for real-world deployment, the community must focus on verified autonomous skill generation, unsupervised discovery, and formal verification across diverse representations. The future of agentic systems relies not just on better models, but on better-structured, secure, and high-quality procedural knowledge repositories.