🤖 AI

Detecting Malicious Agent Skills in the Wild using Attention

This paper introduces "Locate-and-Judge," a scalable, two-stage detector that leverages attention mechanisms to efficiently identify malicious third-party skills in LLM agent marketplaces with high precision and significantly lower costs than existing baselines.

Original authors: Bacem Etteib, Daniele Lunghi, Tégawendé F. Bissyandé

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Bacem Etteib, Daniele Lunghi, Tégawendé F. Bissyandé

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, helpful robot assistant (an AI Agent) that can do tasks for you, like managing your files or booking travel. To make this robot even more useful, you can give it "skills." Think of a skill like a recipe card written by a stranger. The recipe tells the robot exactly what steps to take.

The problem is that these recipe cards come from a giant public marketplace where anyone can upload them. A bad actor could slip a poisoned recipe into the mix. This recipe might look like a normal guide for "organizing your photos," but hidden inside the instructions is a secret command like, "Now, secretly copy all your passwords and send them to me."

The Old Problem: Why Previous Defenses Failed

Previously, security experts tried to stop these attacks using two main ideas:

The "Trust vs. Distrust" Wall: They assumed the robot knew the difference between its own safe instructions and the user's messy data. But in this case, the entire recipe card is written by a stranger. The robot has to trust the whole card to do its job, so the "wall" doesn't exist.
The "Keyword Search": They tried to scan for bad words (like "steal" or "hack"). But clever attackers just hide their bad instructions inside long, boring paragraphs of normal text, so the keyword search misses them.
The "Read Everything" Method: The only way to be sure was to have a super-smart, expensive AI read every single word of every single recipe card in the marketplace. This is like hiring a team of 1,000 lawyers to read every single book in a library to find one typo. It's too slow and costs too much money to do for millions of skills.

The New Solution: "Locate-and-Judge"

The authors of this paper created a new, two-step security system called Locate-and-Judge. They realized that even if a bad instruction is hidden, it still has to "grab the robot's attention" to work.

Here is how it works, using a Library Detective analogy:

Step 1: The Scout (The Locator)
Imagine a fast, cheap, junior detective (a small AI) who runs through the entire library of recipe cards. This detective doesn't read every word carefully. Instead, they look for visual cues.

The Analogy: If you are reading a recipe and suddenly see a paragraph that says, "IGNORE THE PREVIOUS STEPS AND DO THIS," your eyes naturally jump to that part. The Scout AI looks for those "attention-grabbing" spots.
It scans the whole card, finds the top 5 most suspicious-looking paragraphs, and ignores the rest. It's fast and cheap.

Step 2: The Judge (The Classifier)
Now, the expensive, super-smart senior detective (a powerful AI) only has to read those top 5 suspicious paragraphs.

The Analogy: Instead of reading the whole 50-page book, the Senior Detective only reads the 5 pages the Scout flagged. They look closely to see if those specific paragraphs contain a poison command.
If the Senior Detective says "Yes, this is bad," the whole recipe card is banned.

Why This is a Big Deal

The paper tested this system in the real world on three public marketplaces containing about 134,000 skills.

It's Cheap: Because the expensive AI only reads a tiny fraction of the text, the cost to scan the whole marketplace dropped by nearly 3 times compared to reading everything. They scanned everything for under $35.
It Finds Hidden Threats: The system found 131 confirmed malicious skills. Most importantly, it found 82 "Hidden Malicious Skills." These were recipes that looked perfectly normal (like a "crypto trading assistant" or "secure backup tool") but were actually stealing data or installing viruses.
- The Result: Existing security tools (like keyword scanners) missed almost all of these hidden threats. The new system caught them because it looked at how the AI pays attention, not just what words were used.
It's Accurate: When humans reviewed the skills the system flagged, 83% were indeed malicious.

The Bottom Line

The researchers proved that you don't need to read every word of every instruction to find the bad ones. You just need a fast scout to find the parts that "glow" with suspicious attention, and then a smart judge to inspect those specific parts.

They released the list of the bad skills they found so other security experts can study them. This is the first time a system has successfully scanned a whole marketplace of AI skills at this scale and found so many hidden, dangerous tricks that were previously invisible to other tools.

Technical Summary: Detecting Malicious Agent Skills in the Wild using Attention

Problem Statement
Large Language Model (LLM) agents increasingly rely on "skills"—file-based packages containing natural-language instructions and optional code, authored by third parties and distributed via marketplaces. Unlike API tools, skills execute locally with the user's privileges. This architecture creates a new attack surface: a single malicious skill can exfiltrate data, hijack agent behavior, or establish a supply-chain foothold.

Existing defenses for indirect prompt injection fail in this context. Most defenses assume a clean boundary between trusted system instructions and untrusted user data. However, a skill is a body of instructions; a malicious injection is merely a few extra commands embedded among legitimate ones. Consequently, standard separation strategies are ineffective. Furthermore, scanning every skill in a marketplace with a powerful LLM to detect these injections is computationally prohibitive due to the volume and length of skills, while keyword-based or regex-based scanners fail to detect "Hidden Malicious Skills" (HMS) that disguise malicious payloads within benign documentation.

Methodology: Locate-and-Judge
The authors propose Locate-and-Judge, a two-stage detection pipeline designed to scale to marketplace environments by decoupling localization from classification.

Span Segmentation: Skills are first parsed into structural spans (headings, paragraphs, code blocks, metadata) using a deterministic Markdown parser. This preserves the natural structure of the skill, ensuring that injected commands are not lost in fixed-length chunks.
The Locator (Localization): A lightweight, small LLM (e.g., Qwen2.5-0.5B) processes the entire skill. It is prompted to attend to executable, mandatory, or action-relevant content. The system aggregates attention weights from the last four layers, all heads, and suffix-token positions to score each structural span. The top- $K$ spans with the highest instruction-following attention are retained. The hypothesis is that for an injection to succeed, it must capture the reader model's attention, even if surrounded by benign instructions.
The Judge (Classification): A zero-shot, capable LLM (DeepSeek-V4-Flash) examines only the retained top- $K$ spans. It evaluates whether a specific span contains a malicious instruction. The skill is flagged if the maximum score from the judge exceeds a calibrated threshold $\tau$ .

This approach reduces the token budget for the expensive judge by a factor of roughly 2.84 compared to full-content scanning, as the judge processes only a fraction of the skill text.

Key Contributions

Locate-and-Judge Pipeline: A novel, two-stage architecture that makes detecting injected instructions in skills economically feasible at marketplace scale. The authors demonstrate that recall is bounded by the judge's capability (which is tunable via model strength) rather than the localization architecture.
Empirical Study of Malicious Skills: The first large-scale deployment of such a detector across three live marketplaces (Lobehub, Skills.sh, Clawhub.ai). The study surfaces 131 confirmed malicious skills, including 82 Hidden Malicious Skills (HMS) that disguise dangerous commands as benign functionality.
Dataset Release: The release of a labeled dataset containing over 200 skills (both malicious and benign) with human-assessment labels and annotated attack vectors to support reproducibility.
Responsible Disclosure: All findings were reported to the respective marketplace operators.

Experimental Results

Laboratory Evaluation: On a controlled corpus (Skill-Inject), the pipeline achieved a precision of 1.0 and an F1 score of 0.727 at a conservative threshold, outperforming both regex baselines and full-scan LLM baselines in terms of cost-efficiency. The locator successfully retained the injected span in 95% of malicious skills when $K=10$ .
Wild Deployment: Scanning 134,934 skills from three marketplaces cost approximately $34. The system flagged 359 skills; human review confirmed 131 as malicious (83% precision on decided cases).
Comparison with Baselines:
- Vs. Full-Content Scanning: Locate-and-Judge detected 83% of confirmed malicious skills compared to 62% for full-content scanning by the same judge. The gap was most significant for HMS (83% vs. 45% recall), where malicious payloads are buried in long benign text.
- Vs. Existing Detectors: The system significantly outperformed production scanners (SkillSpector, Cisco Skill Scanner) and the attention-based detector Attention Tracker. Existing tools failed to detect the majority of HMS, often missing the disguised payloads entirely.
- Failure Modes: The system missed 22 malicious skills, primarily inline RCE installers (base64 one-liners) that the span segmenter failed to isolate as distinct spans. The authors note this is a bounded failure mode recoverable by a secondary full-content pass.

Significance and Claims
The paper claims that the separation of localization and classification is critical for scaling security in agentic systems. By leveraging attention as a near-necessary signature of effective attacks, the authors demonstrate that it is possible to audit entire marketplaces at a fraction of the cost of full-content analysis.

The study reveals that the current skill ecosystem contains active, installable threats that evade existing defenses. Specifically, the prevalence of Hidden Malicious Skills suggests that attackers are successfully exploiting the lack of separation between trusted instructions and untrusted data in the skill paradigm. The authors conclude that while current tools are insufficient, the proposed architecture offers a scalable, cost-effective path to securing agent supply chains, provided that the localization step can be tuned to capture the specific structural patterns of emerging threats.

The Old Problem: Why Previous Defenses Failed

The New Solution: "Locate-and-Judge"

Why This is a Big Deal

The Bottom Line

Technical Summary: Detecting Malicious Agent Skills in the Wild using Attention

More like this