💻 computer science

Do System Prompts Leave Behavioral Fingerprints? A Large-Scale Empirical Study of Clone Detection via Output Similarity

This paper introduces Black-Box Behavioral Fingerprinting (BBF), a method enabling prompt owners to detect cloned system prompts via output similarity, and validates through a large-scale empirical study that while prompt choice significantly influences model behavior and detection is generally effective, it remains vulnerable to specific stylistic manipulations.

Original authors: Linghan Chen, Yudong Gao, Jiyao Wang, Kaiyan Ji, Honglong Chen

Published 2026-08-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Linghan Chen, Yudong Gao, Jiyao Wang, Kaiyan Ji, Honglong Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, a new kind of intellectual property has emerged: the system prompt. This is not a physical object or a complex machine, but a carefully written set of instructions given to a computer program before it begins a task. Think of it as the hidden script that tells a general-purpose AI how to behave, what tone to use, and which specific skills to apply, effectively turning a broad tool into a specialized product. Just as a chef might have a secret recipe that defines a dish, developers craft these prompts to create unique digital services. However, unlike a physical recipe book that can be locked away, these digital instructions are vulnerable. Because they exist only as text, they can be tricked out of the system by an adversary asking the right questions, copied, and then used by anyone, anywhere, at no cost. Once stolen, the original creator has no way to prove that a competing service is using their stolen script, leaving them without recourse.

This uncertainty led a team of researchers to ask a fundamental question: even if a thief rewrites the stolen instructions to look different, does the original script still leave a trace in the answers the computer gives? To find out, the researchers developed a method called Black-Box Behavioral Fingerprinting. They treated the system prompt not as a text to be read, but as a set of habits that shape how an AI responds. Their approach was simple yet powerful. First, a prompt owner would send a specific list of questions to their own AI and record the answers. They would then analyze the patterns in those answers to create a unique "fingerprint" of that specific prompt's behavior. Later, if they suspected a competitor was using a stolen version, they would ask the same questions to the competitor's AI. By comparing the patterns of the competitor's answers against the original fingerprint and against a random, unrelated set of answers, they could determine if the two systems were behaving in the same way.

The researchers tested this idea on a massive scale, involving four different families of commercial AI models and eight different types of tasks, ranging from medical questions to legal contract analysis. They generated nearly three hundred thousand responses to see if the fingerprint method could reliably spot a clone. The results were striking. They found that the choice of system prompt explains nearly a quarter of the differences in how an AI answers questions. In other words, the hidden instructions leave a strong, measurable mark on the output. When the researchers compared a stolen prompt running on the same type of AI as the original, their method successfully identified the match in almost ninety percent of cases. Even when the stolen prompt was moved to a completely different AI model, the method still worked, though with slightly less certainty, correctly identifying the clone in about seventy-two percent of scenarios on average.

The study also explored how well this method holds up against an adversary trying to hide their tracks. The researchers found that simply rewriting the stolen instructions with different words or changing the sentence structure did not fool the system. The behavioral fingerprint remained strong, allowing the detection method to succeed even when the text of the prompt was heavily altered. However, the researchers discovered a specific weakness. If an attacker added a short, formal instruction at the very beginning of the prompt to force the AI to sound like a serious academic, the method struggled when the task involved short, structured answers. In these specific cases, the detection accuracy dropped significantly, suggesting that while the method is robust against most changes, it can be confused by deliberate attempts to override the AI's natural style on brief tasks.

To make the process even more efficient, the team introduced a rule for selecting the best questions to ask. They found that not all questions are equally useful for spotting a clone; some questions are so straightforward that any AI would answer them the same way, regardless of the prompt. By filtering out these obvious questions and focusing only on those where the prompt makes a real difference, the researchers improved the detection accuracy by a significant margin. They determined that asking just twenty-five carefully chosen questions was enough to confirm a theft with a very high degree of confidence. This means a prompt owner does not need to monitor an entire system or have access to its internal code; they simply need to ask a few targeted questions and compare the results.

The study concludes that while the threat of prompt theft is real and the ability to steal these instructions is well-established, there is now a viable way to detect it. The method works by looking at the behavior of the AI rather than the text of the instructions, making it a practical tool for verification. It is not a perfect shield against every possible attack, particularly those designed to manipulate the style of short answers, but it provides a solid foundation for protecting intellectual property in the age of generative AI. For the first time, prompt owners have a way to verify their suspicions and prove that a specific deployment is running on a stolen script, offering a new layer of security for the digital economy.

Technical Summary: Do System Prompts Leave Behavioral Fingerprints?

Problem Statement

The proliferation of Large Language Model (LLM) application platforms (e.g., GPT Store, Coze) has established the system prompt as a critical form of intellectual property (IP). While model weights are difficult to replicate, system prompts can be extracted verbatim via adversarial queries with success rates exceeding 0.8 and redeployed on any platform at zero cost. Existing defenses focus on preventing extraction, but once a prompt is stolen, the owner lacks a mechanism to verify if a suspected deployment is a clone.

The central research question is whether a system prompt leaves a usable behavioral trace in model outputs that survives extraction, paraphrasing, and cross-model transfer. If such a trace exists, it could enable Black-Box Behavioral Fingerprinting (BBF): a method where a prompt owner registers a behavioral signature from their original deployment and later tests whether a suspect deployment matches that signature more closely than an unrelated baseline, requiring only black-box API access.

Methodology: Black-Box Behavioral Fingerprinting (BBF)

BBF operationalizes clone detection as a two-sample comparison in an embedding space, relying solely on off-the-shelf sentence embeddings and cosine similarity. The process consists of two phases:

Registration: The defender sends a set of diagnostic queries $Q = \{q_1, \dots, q_K\}$ to the original deployment (prompt $P$ ). For each query, $N$ stochastic output samples are collected, embedded via a sentence encoder $\phi$ , and aggregated into a behavioral centroid $\hat{\mu}_k$ . The fingerprint $F$ is the set of these centroids.
Detection: The defender sends the same queries to a suspected deployment (prompt $P^*$ ). Suspect centroids $\hat{\mu}^*_k$ are computed. To control for the fact that different prompts may produce semantically similar answers to the same factual question, the defender also computes centroids for an unrelated prompt ( $P_u$ ) from the same broad domain.
Decision Statistic: The detection signal is the "controlled gap" $\delta_k = s_{clone} - s_{unrel}$ , where $s$ represents cosine similarity between the suspect/clone centroid and the original centroid. A positive gap indicates the suspect is behaviorally closer to the original than to the unrelated baseline.
Aggregation: Detection is performed via majority vote over $T$ queries. Theoretical analysis (Proposition 1) suggests that with a sufficient query budget (e.g., $T=25$ ), the probability of detection converges exponentially.

Diagnostic Query Optimization (DQO):
To maximize detection power, the paper proposes DQO, a zero-cost rule for selecting queries. It posits that queries where the original and unrelated prompts produce highly divergent outputs (low baseline similarity $s_{unrel}$ ) are most sensitive to prompt changes. DQO selects the $T$ queries with the lowest baseline similarity to the unrelated prompt, without requiring access to the suspected clone.

Experimental Setup

The study is a large-scale empirical evaluation involving:

Models: 4 model families (Claude, GPT, DeepSeek, Qwen).
Benchmarks: 8 NLP benchmarks across 7 domains (e.g., MMLU, GSM8K, MNLI, MedQA), totaling 2,000 diagnostic queries.
Scenarios: 128 total scenarios covering all $4 \times 4$ cross-model combinations and 8 benchmarks.
Data: 288,000 generated responses (250 queries $\times$ 3 samples $\times$ 3 prompt types $\times$ 128 scenarios).
Embedding Model: all-MiniLM-L6-v2.

Key Results

1. Existence of Behavioral Fingerprints

The study confirms that system prompt choice explains a significant portion of output variance. A one-way ANOVA yielded a mean effect size of $\eta^2 = 0.244$ (a "large" effect per Cohen's standards), indicating that prompt choice accounts for ~24.4% of the variance in output embeddings. In 118 of 128 scenarios, clone outputs were closer to the original than unrelated outputs.

2. Detection Performance

Same-Model Detection: Highly reliable with a mean AUC of 0.876.
Cross-Model Detection: Performance is bounded by the identity of the detector model.
- Claude as a detector achieved the highest cross-model AUC (0.845).
- Qwen as a detector achieved the lowest (0.665).
- The overall mean cross-model AUC is 0.725.
Query Budget: A budget of 25 queries is sufficient for reliable detection (scenario-level probability > 0.999).
DQO Efficacy: Applying DQO increased cross-model AUC by +0.120 on average.

3. Robustness and Failure Modes

Paraphrasing: BBF is robust to non-adaptive paraphrasing (synonym substitution, structural reorganization, and functional rewrites). Even under aggressive L3 rewrites, AUC remained $\ge 0.889$ .
Imperfect Extraction: Clones derived from imperfect automated extraction were at least as detectable as handcrafted paraphrases, likely because extraction preserves functional framing while handcrafted rewrites introduce orthogonal stylistic noise.
Failure Modes: 11 of 128 scenarios failed (AUC $\le 0.5$ $\leq 0.5$ ). Failures clustered into three patterns:
1. Short Outputs: Tasks with extremely short outputs (e.g., sentiment analysis on SST-2) where style dominates content.
2. Data Artifacts: The CUAD benchmark failed due to a dataset loading issue causing a prompt-query mismatch.
3. Style Mismatch: Specific model pairs (e.g., Qwen $\to$ GPT) exhibited systematic style mismatches.
Adaptive Attacks: While paraphrasing failed to evade detection, a single-sentence formal-tone prefix injected into the prompt successfully collapsed detection on short, structured tasks (MNLI AUC dropped from 0.978 to 0.547). This identifies style-invariant detection as the primary open challenge.

Significance and Claims

The paper positions BBF as a verification tool rather than a discovery tool, intended to complement existing IP enforcement mechanisms. Its primary significance lies in:

Feasibility: Demonstrating that system prompts leave measurable, transferable behavioral fingerprints in black-box outputs.
Accessibility: Providing a detection method that requires no model weights, no platform cooperation, and no modification of the generation process, relying only on API access.
Practicality: Establishing that a fixed, low-cost budget of 25 queries is sufficient for high-confidence detection in most content-rich scenarios.

The authors conclude that while BBF is effective against standard cloning and paraphrasing, the field must address style-invariant detection to defend against adaptive adversaries who manipulate output tone or format. The work does not claim to solve the legal status of prompt cloning but offers a technical protocol for owners to substantiate suspicions of theft.