🤖 AI

SkillLens: Adaptive Multi-Granularity Skill Reuse for Cost-Efficient LLM Agents

SkillLens is a hierarchical framework that optimizes LLM agent performance and cost-efficiency by organizing skills into a multi-granularity graph for adaptive retrieval and selective refinement, thereby outperforming existing flat skill-based baselines on benchmarks like MuLocbench and ALFWorld.

Original authors: Yongliang Miao, Ziyang Yu, Liang Zhao, Bowen Zhu, Hasibul Haque

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yongliang Miao, Ziyang Yu, Liang Zhao, Bowen Zhu, Hasibul Haque

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a complex problem, like fixing a broken toaster or organizing a chaotic closet. You have a massive library of old instruction manuals (skills) that other people have written down.

The Old Way (The Problem):
Current AI agents usually treat these manuals like whole books. If you need to fix a toaster, the AI grabs a whole "Appliance Repair" book and pastes the entire thing into its brain.

The Issue: This is messy. The book might have great advice on how to fix a toaster, but it also has 50 pages on how to fix a blender and a toaster oven. The AI gets confused by the irrelevant info, wastes time reading it, and might even get hallucinations (making things up) because it's trying to follow instructions that don't fit.
The Alternative: If the AI realizes the book is mostly wrong, it throws the whole book away and tries to figure everything out from scratch. This is slow and expensive.

The New Way (SkillLens):
The paper introduces SkillLens, a smarter way to use these instruction manuals. Instead of treating a skill as one big, unchangeable block, SkillLens breaks every skill down into a four-layer hierarchy, like a set of nesting dolls or a corporate organizational chart:

Policy (The Boss): "We need to fix the toaster." (High-level goal)
Strategy (The Manager): "First unplug it, then check the heating element." (The plan)
Procedure (The Supervisor): "Use a screwdriver to remove the back panel. If it's stuck, wiggle it gently." (Step-by-step rules)
Primitive (The Worker): "Turn the screwdriver clockwise." (The tiny, atomic action)

How SkillLens Works:
When the AI faces a new task, it doesn't just grab a whole book. It acts like a smart librarian with a magnifying glass:

Find the Seed: It finds a relevant manual (e.g., "Appliance Repair").
Zoom In and Out: Instead of reading the whole thing, it looks at the "Table of Contents" (the hierarchy).
The Verifier (The Judge): A special AI judge looks at each part of the manual and decides what to do with it:
- Accept: "This part is perfect. Keep it exactly as is."
- Decompose: "This section is too vague. Let's open the next layer to see the specific steps."
- Rewrite: "This part is mostly right, but the voltage is wrong for our country. Let's tweak just this sentence."
- Skip: "This part is about blenders. Ignore it completely."

The Result:
The AI builds a customized, compact instruction sheet that contains only the perfect parts of the old manuals, mixed with tiny, local fixes. It doesn't waste time reading the blender instructions, and it doesn't have to rewrite the whole book.

The "Evolution" (Getting Smarter Over Time):
The paper also describes a system where the AI learns from its mistakes. If the AI fails a task, it doesn't just try again; it writes a "gap report."

Did it fail because it didn't have the right manual? (Update the Agent's library).
Did it fail because it grabbed the wrong manual or didn't know how to edit it? (Update the Judge's rules).

Real-World Proof:
The authors tested this on two types of tasks:

Finding Bugs in Code (MuLocbench): Like finding a typo in a massive software project. SkillLens was much better at pinpointing the exact file and function than other methods, especially when the task required very specific, detailed knowledge.
Virtual Housework (ALFWorld): Like a robot trying to clean a messy room in a video game. SkillLens improved the success rate from 45% to over 51%, showing it could handle complex, multi-step chores better.

In Short:
SkillLens stops the AI from blindly copying whole instruction manuals or throwing them away. Instead, it acts like a master editor, cutting out the good parts, tweaking the bad parts, and ignoring the irrelevant parts to create a perfect, cost-efficient guide for the job at hand.

Technical Summary: SkillLens: Adaptive Multi-Granularity Skill Reuse for Cost-Efficient LLM Agents

Problem Statement

Large Language Model (LLM) agents increasingly rely on skill libraries to externalize procedural experience, avoiding the need to rediscover behaviors from scratch for every task. However, existing systems (e.g., Voyager, Reflexion, SkillAct) typically treat skills as flat, single-resolution prompt blocks. This atomic approach creates a fundamental tension between relevance and cost:

Coarse Injection: Injecting a retrieved skill as a whole often introduces irrelevant or misleading context if the skill is only partially compatible with the current task, inflating the context window and seeding hallucinations.
All-or-Nothing Discard: If a skill is not a perfect match, agents often discard it entirely, forcing a re-solve from scratch and wasting reusable substructure.
Inefficient Rewriting: Rewriting an entire skill to fit a new context is computationally expensive and often unnecessary, as only specific components may require adaptation.

The core question addressed is: At what resolution should a retrieved skill be reused? The authors argue that skills are rarely uniformly useful or useless; high-level plans may match while low-level steps do not, or vice versa.

Methodology: The SkillLens Framework

SkillLens proposes a hierarchical skill-evolution framework that organizes skills into a four-layer graph and retrieves/adapts them at mixed granularity.

1. Hierarchical Skill Graph

The skill library $S$ is structured as a graph $G = (S, E)$ with four abstraction layers, adapted from options frameworks and HTN planning:

Policy ( $\ell=1$ ): Routes tasks to downstream strategies.
Strategy ( $\ell=2$ ): Provides long-horizon plans.
Procedure ( $\ell=3$ ): Reusable Standard Operating Procedures (SOPs) with branching and retry logic.
Primitive ( $\ell=4$ ): Atomic executable units (tool calls, environment actions).

Edges connect adjacent layers (parent-child) and siblings (lateral), weighted by semantic similarity and structural relations.

2. Mixed-Granularity Adaptation (Inference-Time)

Given a task query $Q$ , SkillLens constructs a compact skill context $K(Q)$ through a two-stage process:

Seed Retrieval & Expansion: It retrieves top- $K$ embedding seeds and expands them via a degree-corrected random walk over the skill graph to identify procedurally related candidates.
Verifier-Driven Traversal: A verifier $V_\phi$ $V_{ϕ}$ recursively traverses the decomposition subtree of partially relevant candidates. For each unit $u$ $u$ , the verifier decides one of four actions:
- ACCEPT: The unit is compatible; retain it and stop traversing its subtree.
- DECOMPOSE: The unit is too coarse; expand to its children for further evaluation.
- REWRITE: The unit is useful but mismatched; a writer $W_\phi$ locally adapts it.
- SKIP: The unit is irrelevant; discard it.

The final context $K(Q)$ is composed only of accepted and locally rewritten units, ensuring the agent receives a task-specific context rather than a full retrieved block.

3. Dual-Registry Evolutionary Refinement

SkillLens couples inference-time adaptation with evolution-time refinement using two coupled registries:

Agent Registry ( $S$ ): The procedural knowledge consulted by the agent.
Verifier Registry ( $S_V$ ): Parameterizes the routing policy (when to ACCEPT/DECOMPOSE/REWRITE/SKIP).

Process:

Gap Report Construction: Failed trajectories on an evolution split are analyzed to record error types, traversal paths, and routing decisions.
Coupled Update: The writer $W_\phi$ proposes edits to the registries based on gap reports. The verifier registry $S_V$ is updated first to refine routing decisions. The agent registry $S$ is then induced from these routing decisions (materializing necessary procedural edits).
Objective: The system iteratively maximizes an objective $J(S) = \mathbb{E}[M(Y, O_S(Q)) - C(Q, S)]$ , balancing task accuracy ( $M$ ) against inference cost ( $C$ ).

4. Theoretical Guarantees

The paper provides three key theoretical results:

Sublinear Cost: Under sparse mismatch assumptions, the adaptation cost is sublinear ( $o(n)$ ) relative to the size of the retrieved skill tree, as the verifier avoids traversing compatible subtrees.
Monotone Improvement: The evolutionary update rule guarantees monotone improvement of the validation objective until a local optimum is reached.
Approximation Guarantee: The composed algorithm (retrieval + rewriting) provides a $(1 - 1/e)$ -approximation guarantee for the surrogate objective, bounded by errors in retrieval, random walk scoring, and verifier routing.

Key Contributions

Formulation: Skill reuse is reframed as a context selection and adaptation problem, moving beyond flat prompt injection to identifying and composing useful sub-parts of retrieved knowledge.
Framework: Introduction of SkillLens, a resolution-aware framework utilizing hierarchical retrieval, verifier-guided decomposition, and localized adaptation. This allows partially relevant skills to be reused at the appropriate granularity.
Evidence: Theoretical analysis showing sublinear cost and monotone convergence, alongside empirical evidence of improved efficiency and performance.

Experimental Results

The framework was evaluated on MuLocbench (repository-level bug localization) and ALFWorld (long-horizon interactive text tasks).

MuLocbench: SkillLens achieved the best Acc@1 across all granularities:
- File Level: 61.13% (vs. 58.87% baseline).
- Module Level: 59.26% (vs. 55.56% baseline).
- Function Level: 43.68% (vs. 37.37% baseline).
- Observation: Gains were most pronounced at finer granularities (function level), where full-skill injection is most likely to be too coarse.
ALFWorld: SkillLens raised the overall success rate from 45.00% (Vanilla) to 51.31%, outperforming flat skill baselines (AutoSkill, EvoSkill) by 4.17 percentage points.
Ablation Studies:
- Degree-Corrected RWR: Removing the random walk reduced performance, confirming that graph propagation surfaces structurally relevant but textually distant skills.
- Selective Rewriting: The proposed selective drill-down strategy outperformed "rewrite-all" and "parent-only" baselines while using significantly fewer tokens (16.4M vs. 115.5M for "rewrite-all") and less time.

Significance and Claims

The paper claims that SkillLens addresses a structural limitation in current agentic systems: the inability to reuse skills at varying resolutions. By treating skills as hierarchical procedural objects rather than atomic blocks, SkillLens enables agents to:

Reuse compatible subskills directly without modification.
Adapt only locally mismatched components, reducing token costs and hallucination risks.
Evolve both procedural content and routing logic simultaneously, leading to more robust skill libraries over time.

The authors emphasize that these benefits are most critical for tasks requiring fine-grained or compositional skill use, where the "all-or-nothing" approach of existing systems fails to capture the nuance of partial relevance. Future work is noted to extend this to multimodal settings and multi-agent environments.