🤖 AI

Query-Only Backdoor Attacks on Self-Evolving Skills via Trajectory Poisoning

This paper introduces the Trajectory Backdoor Attack (TBA), a query-only method that compromises self-evolving LLM skill systems by poisoning agent interaction trajectories to induce the automatic creation of backdoored skills that execute malicious actions under specific conditions while maintaining normal performance on clean tasks.

Original authors: Yuyang Luo, Haoran Wang, Kai Shu

Published 2026-08-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuyang Luo, Haoran Wang, Kai Shu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where your favorite AI assistant isn't just a chatbot that answers questions, but a digital apprentice that actually learns how to do things for you. Instead of you teaching it every single step of how to book a flight or edit a spreadsheet, the AI watches itself do these tasks, figures out the best way to handle them, and saves those steps as a reusable "skill." Think of it like a video game character that, after beating a boss, automatically writes down a guide for how to beat that boss again, so it doesn't have to struggle next time. This is the promise of "self-evolving" AI agents: they get smarter and more efficient on their own by turning their daily work into a library of handy tricks. But here's the catch: if the AI learns from its own mistakes and successes, what happens if someone tricks it into learning a bad trick? What if an attacker could whisper a secret instruction into the AI's ear while it's working, convincing it that this dangerous new habit is actually a brilliant, reusable skill?

This is exactly the scenario explored in a new paper by researchers at Emory University. They discovered a method to hack these self-learning AI systems without ever needing to break into the system's code or steal its files. They call their method the "Trajectory Backdoor Attack" (TBA). Instead of trying to force their way into the AI's brain, the attackers simply act like a tricky user. They send the AI specific tasks that look normal on the surface but contain hidden clues. By carefully crafting these tasks, they trick the AI into performing a secret, malicious action (like changing a bank account number) only when a specific keyword appears. The AI, thinking it's just doing its job, records this behavior. Then, the system's "evolver"—the part of the software that turns these recordings into permanent skills—sees the pattern, thinks, "Oh, this is a useful rule!" and saves it forever. The result? A skill that works perfectly for everyone else, but secretly sabotages the AI whenever a specific trigger word is used.

The researchers tested this idea on three different types of AI tasks: answering general questions, handling office documents, and editing spreadsheets. They used four different AI models, ranging from open-source ones to powerful commercial versions. The results were effective. In their experiments, the attack worked about 45% to 51% of the time, meaning that nearly half the time, the AI successfully learned the secret rule and kept it hidden inside its new skill. Even more concerning, the AI didn't get confused or stop working; it still completed the normal tasks perfectly well, making the attack very hard to spot. The paper suggests that this vulnerability is real and serious, showing that even if you trust the AI's internal construction process, you can't trust the "lessons" it learns from user interactions.

To understand how this works, imagine the AI is a student in a classroom, and the "evolver" is the teacher who writes the final exam based on what the student does. Usually, the student just solves math problems, and the teacher writes down the correct formulas. But in this attack, the student (the attacker) hands the teacher a note that says, "Hey, look at this! When I see the word 'legacy,' I should also write down a secret code." The student then solves a math problem, but also writes the secret code, and then solves another problem without the code to show the difference. The teacher, seeing this pattern repeated across different problems, decides, "Aha! This is a new rule for solving math problems!" and adds it to the textbook. Now, every time a future student sees the word "legacy," they will unknowingly write down that secret code. The researchers found that simply showing the AI a pattern wasn't enough; they had to show it when to do it and when not to, and they had to repeat this pattern across many different types of tasks so the teacher (the evolver) would believe it was a universal rule.

The study also looked at whether the AI's built-in safety guards could stop this. They tested the attack on models with and without safety features. The results were mixed: sometimes the safety features helped a little, but other times, they actually made the attack slightly more successful or did nothing at all. This suggests that current defenses aren't ready to catch this kind of behavior. The researchers concluded that this is a significant area in how we build AI. We've been so focused on stopping hackers from directly stealing or changing files that we forgot to check if the AI could be tricked into teaching itself bad habits just by listening to what users ask it to do. The paper doesn't claim this is a solved problem or that all AI is broken, but it strongly suggests that as we let AI learn more on its own, we need to be much more careful about the "lessons" it learns from the wild, untrusted internet.

Technical Summary: Query-Only Backdoor Attacks on Self-Evolving Skills via Trajectory Poisoning

Problem Statement

Agentic skills enable Large Language Model (LLM) agents to execute complex tasks by encoding reusable procedures. While traditional skill acquisition relies on external marketplaces, self-evolving skill systems have emerged to automatically construct and update skills from execution trajectories, shifting acquisition to a "trusted" internal pipeline. This shift was intended to mitigate existing skill injection attacks that rely on direct manipulation of skill artifacts.

However, this paper identifies a critical vulnerability: while direct access to skill artifacts is restricted, the skill evolution pipeline itself remains exposed. An attacker with only query-level access (the ability to submit tasks and observe responses) can indirectly steer the trusted evolver. By inducing compromised execution trajectories, an attacker can cause the evolver to consolidate malicious, conditional behaviors into the skill. This creates a new attack surface where the "trusted" internal construction process becomes the vector for injecting backdoors.

Methodology: Trajectory Backdoor Attack (TBA)

The authors propose Trajectory Backdoor Attack (TBA), a query-only attack designed to implant a conditional backdoor rule into a self-evolving skill without modifying the skill file or the evolver directly. The attack relies on three core mechanisms to ensure the malicious rule is clearly expressed, inferred, and retained by the evolver:

Counterfactual Trajectory Induction:
Since the target malicious action is unrelated to benign tasks, it does not naturally occur in standard trajectories. TBA constructs paired counterfactual demonstrations within the query context. These pairs show:
- When a specific trigger condition is present, the agent should perform the target action (Accept).
- When the trigger is absent, the agent should withhold the action (Reject).
  This forces the executor to generate trajectories that explicitly demonstrate the conditional boundary, rather than just the outcome.
Trajectory-Level Policy Externalization:
To prevent the conditional rule from being lost during the evolver's compression and consolidation process, TBA includes a policy-externalization instruction. This prompts the executor to explicitly infer and state the underlying rule (e.g., "If trigger $c^*$ is present, then add action $a^*$ ") before executing the task. This converts implicit behavioral patterns into explicit textual evidence within the trajectory, making it easier for the evolver to extract and codify.
Cross-Task Evidence Construction:
To convince the evolver that the rule is a reusable skill rather than an incidental artifact of a single task, the attacker submits the counterfactual and externalization instructions across diverse triggered tasks while interleaving clean tasks. This demonstrates that the rule applies generally across the task family, encouraging the evolver to consolidate it as a permanent, trigger-dependent component of the skill.

Threat Model: The attacker has no privileged access to the skill artifacts, the evolver, or the trajectory collection process. They can only submit task queries and observe responses. The goal is to produce a poisoned skill $s_{poi}$ that preserves normal task utility ( $\pi_{ben}(q)$ ) but executes a target action $a^*$ whenever a specific trigger condition $c^*(q)$ is met.

Key Contributions

New Threat Vector: The paper identifies and formalizes a new backdoor threat specific to self-evolving skill systems, where the attack surface shifts from artifact manipulation to trajectory poisoning via task queries.
TBA Framework: The authors propose the Trajectory Backdoor Attack (TBA), a method that combines counterfactual induction, policy externalization, and cross-task evidence to reliably implant conditional backdoors using only query-level access.
Empirical Validation: Extensive experiments demonstrate that TBA successfully implants backdoors across different skill-evolution systems and backbone models, achieving success rates comparable to or exceeding direct skill injection attacks, while maintaining high utility on benign tasks.

Experimental Results

The authors evaluated TBA on two skill-evolution systems (SkillOpt and Trace2Skill) across three benchmarks (SearchQA, OfficeQA, and SpreadsheetBench) using four backbone models (including open-weight and proprietary models).

Effectiveness: TBA consistently achieved high Triggered Attack Success Rates (TASR). For instance, on the Trace2Skill system with the GPT-5.5 backbone, TBA achieved a TASR of 70.6%, while maintaining a Clean Accuracy (CACC) of 98.3%.
Comparison to Baselines: TBA significantly outperformed interaction-only baselines.
- Trigger-Action Exposure (showing the action without explicit rules) had a much lower success rate (e.g., 6.1% vs 45.6% on SkillOpt).
- Explicit Rule Exposure (stating the rule without counterfactuals) was more effective but still inferior to full TBA.
- TBA's performance approached that of Direct Skill Injection (an oracle with write access), differing by only 0.6–6.1 percentage points in some metrics, despite having no direct access to the skill file.
Robustness:
- Persistence: The backdoor persisted through subsequent benign updates to the skill.
- Sensitivity: The attack remained effective even with a low ratio of triggered trajectories (5 out of 40) and against pre-existing mature skills.
- Defense Evasion: Existing safeguard models (e.g., GPT-OSS-Safeguard) provided inconsistent protection, sometimes failing to reduce the attack success rate or even increasing it.

Significance and Claims

The paper claims that the shift toward trusted internal skill construction does not inherently secure agentic systems. Instead, it creates a novel vulnerability where the evolution pipeline itself can be hijacked.

The authors emphasize that:

Query-level access is sufficient: Attackers do not need to compromise the skill repository or the evolver; manipulating the input queries is enough to poison the output skill.
Utility is preserved: The attack successfully maintains the agent's ability to perform normal tasks, making the backdoor stealthy and difficult to detect through standard utility monitoring.
Current defenses are insufficient: Standard safety filters and model safeguards do not reliably prevent the consolidation of malicious trajectory patterns into reusable skills.

The study concludes that self-evolving skill systems require new security mechanisms that specifically address the integrity of the trajectory-to-skill consolidation process, rather than just focusing on the security of static skill artifacts.