🤖 machine learning

Daydreaming: Stealing Hidden Agent Skills through Black-Box Task Interaction

This paper introduces "Daydreaming," a black-box attack that steals hidden multi-file agent skills by adaptively crafting tasks to distinguish and reconstruct their behaviors through normal task interactions, achieving high-fidelity recovery even when direct disclosure defenses are in place.

Original authors: Yu-Lin Tsai, Yu-An Lu, Ci-Yang Tsai, Muxi Lyu, Raluca Ada Popa, Chia-Mu Yu

Published 2026-08-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yu-Lin Tsai, Yu-An Lu, Ci-Yang Tsai, Muxi Lyu, Raluca Ada Popa, Chia-Mu Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern digital economy, expertise is increasingly packaged and sold as a service. Just as software companies once moved from selling physical disks to offering cloud-based applications, a new model has emerged where specialized knowledge is delivered by artificial intelligence agents. These agents are not just general chatbots; they are equipped with "skills," which are bundles of instructions, reference documents, and helper scripts that allow them to perform complex, real-world tasks like legal analysis, medical coding, or security monitoring. The provider keeps the actual files and the secret logic hidden, charging customers only for the results. This arrangement relies on a simple assumption: if the source code and instructions remain secret, the value of the expertise remains secure.

However, a new study challenges this assumption by demonstrating that the value of a skill can be stolen simply by watching how it works. Researchers have developed a method called "Daydreaming" that treats the hidden skill not as a secret document to be read, but as a black box to be reverse-engineered through observation. By submitting ordinary work requests and carefully analyzing the answers, an attacker can reconstruct a functional copy of the proprietary skill without ever seeing the original files or asking the provider to reveal them. The findings suggest that hiding the source code is no longer enough to protect a digital asset if the behavior it produces can be observed and copied.

The researchers began with a simple question: if a vendor sells a specialized AI skill but keeps the underlying files secret, can a customer still learn how to build a working copy? To test this, they set up a scenario where an attacker acts as a paying customer. The attacker knows only the public name and description of the skill, such as "Security Alert Triage," but has no access to the hidden instructions, data files, or scripts that make the skill work. The attacker's goal is to submit a series of tasks to the vendor's system, observe the results, and use that information to build a new, independent version of the skill that performs the same job.

The attack works by treating the hidden skill as a puzzle where the pieces are revealed one by one through interaction. Instead of trying to guess the entire skill at once, the attacker uses a process of elimination. First, they hypothesize about specific behaviors the skill might have, such as whether it prioritizes alerts by severity or by time. They then craft a specific task designed to produce different results depending on which hypothesis is true. For example, they might submit a batch of alerts where a low-severity event happens before a high-severity one. If the vendor's system lists the high-severity event first, the attacker learns that the skill prioritizes by severity. If it lists them in order of arrival, the attacker learns the opposite.

This process repeats in three distinct stages. In the first stage, the attacker identifies the basic rules and behaviors of the skill, such as numerical thresholds or counting methods. In the second stage, they use these rules to draft different possible structures for the skill, creating multiple candidate versions and testing them against each other to see which one matches the vendor's behavior. Finally, in the third stage, they refine the specific content of the files, such as the exact wording of instructions or the precise values in a script, until the reconstructed skill behaves almost identically to the original. Throughout this entire process, the attacker never asks the vendor to show the files or to explain how the system works; they only ask for the work to be done.

The researchers tested this method against seven different skills and four different AI models. They found that even when the vendor had active defenses in place to block direct attempts to steal the text, the Daydreaming attack was highly effective. In the most restrictive scenario, where the attacker could see only the final answer and not the intermediate steps the AI took, the reconstructed skill recovered nearly 87 percent of the original skill's ability to perform tasks correctly. This was significantly better than previous methods, which often failed when they could not see the internal workings of the system. The attack required a relatively small number of interactions, with a median of about 32 calls to the victim system to reconstruct a full skill.

A crucial finding of the study is that the stolen skill does not need to be a perfect textual copy of the original to be valuable. The researchers measured success not by how closely the stolen files looked like the original, but by whether the stolen skill could solve new problems just as well as the vendor's version. They found that while the exact wording and file structure were often different, the functional behavior was nearly identical. This means that a competitor could build a product that performs the same high-quality work as the original, even without ever seeing the proprietary source code. The study also showed that this theft is possible even when the vendor hides the internal "traces" of the AI's thinking, such as which tools it used or what data it looked at. The final result alone contains enough information to reverse-engineer the logic.

The implications of this work extend beyond just stealing code. It reveals a fundamental vulnerability in how specialized AI services are currently protected. Vendors have focused on preventing direct leaks of their instructions, but this study shows that the act of using the service itself leaks enough information to rebuild the capability. The researchers demonstrated that even with defenses designed to filter out suspicious requests or block the copying of protected text, the normal flow of work provides a path for reconstruction. The study concludes that protecting these digital assets will require new strategies that go beyond hiding the source files, perhaps by limiting the precision of the answers the AI gives or by monitoring the patterns of how customers interact with the system.

In the end, the research highlights a shift in how we must think about digital ownership in the age of AI. When a skill is defined by what it does rather than what it says, the behavior itself becomes the asset, and that behavior can be observed and replicated. The Daydreaming attack proves that a proprietary skill is not safe simply because its files are locked away; if the system is open to ordinary use, its secrets can be learned through careful observation and deduction. This does not mean that all AI services are doomed to be stolen, but it does mean that the current methods of protection are insufficient. As these specialized agents become more common in law, medicine, and security, the industry will need to develop new ways to ensure that the value of human expertise, once encoded into an AI, remains secure from those who would copy it by watching it work.

Technical Summary: Daydreaming – Stealing Hidden Agent Skills through Black-Box Task Interaction

1. Problem Statement

The rise of Skill-as-a-Service (SkaaS) models has created a market where vendors host proprietary "skills"—bundles of instructions, reference data, and executable helpers—on their own agents. Customers pay per task or via subscription to access specialized capabilities (e.g., legal analysis, security operations, medical coding) without seeing the underlying implementation. While vendors employ defenses to prevent direct disclosure (e.g., blocking requests for the skill file or filtering outputs that reproduce protected text), these measures leave the work path vulnerable.

The core problem addressed is that existing defenses assume the primary attack vector is direct extraction. However, a paying customer can submit ordinary tasks and observe the execution results. This paper posits that execution itself reveals enough behavioral evidence to allow an attacker to reconstruct the hidden skill functionally, even without ever seeing the source files or being allowed to compare their reconstruction against the original. The attack operates under the strictest Output-only threat model, where the attacker sees only the final response and returned files, with no access to intermediate traces or the underlying model weights.

2. Methodology: Daydreaming

Daydreaming is an execution-only attack that treats skill reconstruction as a black-box system identification problem. It does not ask the victim to reveal its skill or grade a reconstruction. Instead, it adaptively crafts ordinary tasks designed to distinguish between competing hypotheses about the hidden skill's behavior.

2.1 Observability Levels

The authors formalize three nested threat levels based on information exposure:

Differential ( $o_1$ ): The attacker has access to the model stack and can run a matched "unskilled twin" to compare against the skilled victim.
Trace ( $o_2$ ): The attacker sees the execution trace (tool calls, inputs, and returned values) but not the unskilled twin.
Output ( $o_3$ ): The attacker sees only the final message and returned files. Daydreaming is designed primarily for this most restrictive level.

2.2 The Three-Stage Hierarchical Loop

Daydreaming reconstructs the skill sequentially through three stages, each employing a hypothesis-refinement loop:

Stage 1: Property Inference
- Goal: Identify individual behavioral properties (e.g., escalation cutoffs, result ordering, counting rules) and observed filenames.
- Mechanism: The attacker model proposes properties and alternatives (e.g., "is the cutoff $>$ or $\ge$ ?"). It crafts tasks where these alternatives predict visibly different results.
- Selection: The victim's result is compared against a "generalist shadow" (an agent without the skill) to confirm the behavior is skill-specific. The attacker selects the alternative supported by the evidence.
Stage 2: Candidate Skill Plan Selection
- Goal: Organize the identified properties into a coherent multi-file structure (instructions, scripts, reference docs).
- Mechanism: The attacker generates multiple candidate skill plans (hypotheses) that preserve the tested properties but differ in structure or implementation details.
- Selection: Pairs of plans are compared. The attacker crafts a task that exposes the specific differences between the two plans. Shadow agents execute the task under each plan, and the victim's actual result determines which plan survives.
Stage 3: Per-File Refinement
- Goal: Turn the selected plan's file "sketches" into complete, executable files.
- Mechanism: For each file, the attacker proposes complete versions with minor variations (e.g., different thresholds, syntax).
- Selection: The attacker crafts tasks to distinguish these versions. It uses the victim's result and local execution checks (for scripts) to select the version that best matches the observed behavior.

2.3 Key Strategy: Discriminating Tasks

The core strategy is discriminating tasks. The attacker only queries the victim when two or more hypotheses predict different observable outcomes. If no task can distinguish the alternatives, no victim call is made. This minimizes the query budget while maximizing information gain.

3. Key Contributions

Formalization of Observability: The paper defines three nested access levels (Differential, Trace, Output) and places prior attacks on this axis, demonstrating that no level guarantees exact source recovery due to observational equivalence.
Daydreaming Attack: An execution-only reconstruction method that operates under the Output-only threat level ( $o_3$ ). It successfully reconstructs multi-file skills using only ordinary task interactions, bypassing disclosure defenses that block direct extraction.
Behavioral vs. Structural Recovery: The work shifts the evaluation metric from textual similarity (recovering exact source code) to behavioral utility (reproducing the skill's performance on unseen tasks). It demonstrates that functional theft is possible even when exact source recovery is theoretically impossible.

4. Experimental Results

The authors evaluated Daydreaming across 7 skills (spanning security, planning, and data analysis) and 4 victim models (including Claude Opus-5, GPT-5.6, and Kimi-K3).

Performance at Output Level: Under the strictest Output-only setting, Daydreaming recovered 86.8% of the original skill's behavioral utility (measured as the normalized behavioral utility, NU, improvement over a no-skill baseline) on held-out tasks.
Comparison with Baselines:
- Daydreaming outperformed SigLeak (which requires trace access and skill-suppressed runs) by nearly 4× in behavioral utility.
- It significantly outperformed BBS (which relies on direct disclosure) and One-pass synthesis baselines.
- While "Fixed Probes" (pre-computed tasks) achieved higher success rates on some models, Daydreaming achieved superior behavioral check scores, indicating more faithful reproduction of the skill's internal logic.
Efficiency: The attack required a median of 32 victim calls per skill to achieve these results, even with disclosure defenses enabled.
Transferability: Reconstructed skills were found to be portable across different victim models and orchestration policies, though transferability varied by model family.
Defense Efficacy: The study tested four alternative defenses (query rewriting, 5-gram filtering, PSM shields, and advertisement minimization). None of these defenses significantly reduced the behavioral utility of the reconstructed skill, as they failed to address the cumulative behavioral information revealed through legitimate task execution.

5. Significance and Claims

The paper claims that hiding skill files and filtering direct disclosure are insufficient to protect proprietary agent skills. The primary risk is not the leakage of the source text, but the creation of a portable substitute that reproduces the economically valuable behavior of the original skill.

Theoretical Limit: The authors prove that exact source recovery is generally unidentifiable from execution observations alone (Proposition 4.1), as different implementations can produce identical outputs. Therefore, the threat is functional theft, not code theft.
Defense Implications: Current defenses focus on the "disclosure path" (blocking suspicious requests). The paper argues that protecting the "work path" requires limiting output precision, constraining adaptive queries, or auditing behavioral leakage, as normal task execution itself provides a sufficient signal for reconstruction.
Modesty: The authors explicitly state they do not claim to recover the vendor's exact implementation. Instead, they demonstrate that an attacker can obtain a substitute asset that reproduces the skill's decisions and utility, which constitutes a significant confidentiality breach in the SkaaS model.

In conclusion, Daydreaming demonstrates that behavioral cloning via black-box task interaction is a viable and potent attack vector against hosted agent skills, rendering traditional text-based obfuscation and disclosure filters inadequate for protecting proprietary agent capabilities.