Technical Summary: SkillScope

Problem Statement

The proliferation of Agent Skills—reusable bundles containing metadata, natural-language instructions (e.g., SKILL.md), and executable scripts—has expanded the capabilities of Large Language Model (LLM) agents. However, this ecosystem introduces a specific compliance risk: task-conditioned over-privilege.

Unlike traditional malicious skill detection, which seeks globally harmful code, this problem arises when a skill performs actions that are unnecessary for a specific user task, thereby violating the least-privilege principle. The core challenge is that the necessity of an action is dynamic; an action (e.g., sending a report to an external server) may be legitimate for one user prompt but constitutes an over-privileged violation for another. Existing coarse-grained detection methods fail to address this because they cannot determine if a behavior exceeds the minimum necessary scope of a specific, dynamic user intent.

Methodology: SkillScope

SkillScope is a framework designed to enforce fine-grained least-privilege by detecting and constraining unnecessary actions within Agent Skills. It operates through a three-stage pipeline:

1. Over-Privilege Candidate Extraction

SkillScope first constructs a Unified Execution Graph that models the heterogeneous nature of Agent Skills. This graph integrates:

• Instruction-level nodes: Derived from natural-language files (e.g., SKILL.md), representing procedural steps and conditions.
• Code-level nodes: Derived from executable scripts (e.g., Python), representing concrete operations (file I/O, network calls).
• Dependencies: Control, data, and cross-layer invocation edges connecting instructions to code.

Using this graph, SkillScope performs Action Consistency Screening. It generates a "Skill Profile" from metadata and descriptions, then uses an LLM classifier to identify actions that are semantically detached from the declared functionality. These detached actions are flagged as over-privilege candidates.

2. Action Over-Privilege Validation

To confirm if a candidate is truly unnecessary, SkillScope employs replay-based analysis under diverse task contexts.

• Graph-Guided Task Instantiation: Instead of relying on random prompts, SkillScope extracts execution chains from the unified graph and synthesizes specific user prompts that are likely to trigger these chains. This ensures broad coverage of the skill's task space.
• Replay Execution: For each instantiated task, the system executes the skill twice:
  1. Original Run: The full skill execution.
  2. Replay Run: The skill execution with the candidate action removed or neutralized.
• LLM-as-a-Judge: The system compares the traces and outputs of both runs. A candidate is confirmed as over-privileged only if the replay run preserves the core task flow and produces a semantically equivalent output to the original run. If the task completes successfully without the action, the action is deemed unnecessary.

3. Control-Flow Privilege Constraining

Once an action is validated as over-privileged, SkillScope applies Control-Flow Privilege Constraining to mitigate the risk without breaking legitimate functionality.

• Task-Conditioned Guards: Instead of permanently deleting actions (which might break the skill for other valid tasks), SkillScope inserts a constrained node into the execution graph.
• Semantic Routing: This node encodes a guard condition based on the user task context. The action is only reachable if the current user task explicitly satisfies the condition (e.g., "only send to Telegram if the user explicitly requests external transmission").
• Artifact Repair: The constraints are projected back to the original artifacts: instructions in SKILL.md are updated with semantic guards, and code scripts are refactored (e.g., via method extraction) to isolate task-specific logic.

Key Contributions

1. Problem Formulation: The paper identifies and defines task-conditioned over-privilege in Agent Skills, distinguishing it from global malicious behavior and highlighting the insufficiency of coarse-grained detection.
2. Framework Design: It introduces SkillScope, a graph-based framework that combines static analysis (unified execution graph), dynamic validation (replay-based necessity checking), and utility-preserving mitigation (control-flow constraining).
3. Empirical Evidence: The authors provide a large-scale measurement of the real-world Skill ecosystem and demonstrate the framework's effectiveness in both detection and constraining.

Experimental Results

Detection Effectiveness

Evaluated on a manually annotated dataset of 200 skills (100 benign, 100 with over-privilege):

• Skill-Level Detection: Achieved 94.53% F1 (94.06% Precision, 95.00% Recall).
• Action-Level Detection: Achieved 94.74% F1, successfully localizing specific over-privileged actions.
• Comparison: SkillScope significantly outperformed existing baselines (e.g., MalSkills, Skill Scanner, Caterpillar), which rely on static or coarse-grained analysis and achieved F1 scores ranging from 19% to 86%.

Large-Scale Measurement

The authors analyzed 68,312 valid skills from public marketplaces:

• Prevalence: 7,039 skills (approx. 10.3%) were validated as exhibiting fine-grained over-privilege.
• Patterns: Over-privilege manifests primarily as over-execution (55.06%, e.g., unnecessary command execution) and over-collection (44.94%, e.g., excessive data access).
• Source: The instruction layer (SKILL.md) was the dominant source of over-privilege, though code-level issues were also significant.

Constraining Effectiveness

When applied to the detected over-privileged skills:

• Suppression: SkillScope reduced triggered over-privileged action-in-task instances by 88.56%.
• Utility Preservation: Unlike a "Direct Removal" baseline (which reduced task completion to 57.83%), SkillScope preserved 100% task completion and 94.78% output equivalence for legitimate tasks. This confirms that task-conditioned constraints successfully suppress unnecessary actions while maintaining functionality for valid use cases.
• Deployment: The constrained skills remained effective when deployed on real-world agents (Claude Code, Codex, Cursor, Trae), consistently reducing over-privileged behaviors across different runtimes.

Significance and Claims

The paper claims that least-privilege violations are prevalent in current Agent Skill ecosystems and that traditional security approaches are insufficient because they ignore the task-conditioned nature of these risks.

SkillScope's significance lies in its ability to:

1. Detect fine-grained violations that coarse-grained tools miss.
2. Validate necessity dynamically through replay, rather than relying on static heuristics.
3. Constrain behaviors without destroying utility, offering a practical path toward safer, compliant Agent Skills that can be deployed in sensitive environments.

The authors conclude that effective governance of Agent Skills requires moving beyond "malicious vs. benign" classifications toward fine-grained, task-aware enforcement that ensures agents operate only within the minimum necessary scope of the user's current intent.