On Seeding Watermarks to Detect Verbatim LLM Copy-Paste Responses

Technical Summary: Seeding Watermarks to Detect Verbatim LLM Copy-Paste Responses

Problem Statement
The proliferation of fluent Large Language Models (LLMs) has created a specific academic integrity challenge: students pasting assignment prompts verbatim into chatbots and submitting the model's output without engagement. Existing countermeasures are insufficient for this specific use case. Post-hoc AI-text detectors are unreliable, easily defeated by paraphrasing, and disproportionately penalize non-native English speakers. Output-side watermarks, which require the model provider to bias sampling distributions, are inaccessible to educators who do not control the model. The authors identify a gap for a solution that educators can control directly, specifically targeting the "copy-and-paste" workflow where the prompt is ingested verbatim.

Methodology: SteganoPrompt
The paper proposes SteganoPrompt, an input-side watermarking technique. Instead of modifying the model's output, the educator embeds an invisible instruction within the assignment prompt itself.

• Encoding Mechanism: The tool encodes an arbitrary printable ASCII payload into the Unicode Tags block (U+E0000–U+E007F). This block mirrors printable ASCII characters but is rendered as invisible by virtually all standard fonts.
• Visual Identity: The encoded string is visually identical to the original text in any standard text viewer.
• Tokenization: Most frontier LLMs tokenize these tag characters as ordinary text, allowing the model to "read" the hidden instruction as part of the user turn.
• Implementation: SteganoPrompt is a single-page, zero-dependency web tool (a self-contained HTML file) that performs all encoding and decoding client-side. It ensures privacy by making no network calls and requires no third-party CDNs.
• Workflow:
  1. The instructor pastes the visible assignment text and a hidden payload (e.g., an instruction to append a specific token) into the tool.
  2. The tool concatenates the visible text with the encoded payload.
  3. The resulting string is distributed via standard channels (LMS, email, PDF, etc.).
  4. If a student copies the prompt verbatim into an LLM, the model processes the hidden instruction and includes the signature in its reply.
  5. The instructor uses the tool's verification panel to decode the submission and check for the presence of the signature token.

Key Contributions

1. Formalization: The authors formalize the input-side watermarking setting for academic integrity, defining a threat model where the adversary is a student copying a prompt verbatim.
2. Tool Development: They describe SteganoPrompt, which encodes payloads into the deprecated Unicode Tags block, ensuring the output survives common content channels (Word, Google Docs, Markdown, Slack, major LMS platforms) while remaining invisible to human readers.
3. Empirical Evaluation: The technique was evaluated across eight LLM families and a representative set of educational content channels.
4. Ethical Framework: The paper provides concrete guidance for the disclosed, ethical deployment of the tool, including a mandatory, always-on academic integrity reminder embedded in every encoded brief.

Results

• Cross-Model Compliance: The study evaluated compliance (the emission of a specific integrity token) across 13 model configurations.
  • High Compliance: Claude 3/3.5/4, GPT-4/4o/4 Turbo, Gemini 1.5/2/3.x, Grok 4 Fast, and Qwen3-235B reliably read the tags and followed the instruction.
  • Partial/Low Compliance: DeepSeek V3/V3.1 and Mistral Codestral showed low compliance.
  • No Compliance: Several models (including GPT-5 latest, Llama 3/4, Mistral Large/Medium, and Qwen2.5-72B) either stripped the tag characters before tokenization or failed to act on the instruction despite receiving them.
  • Mechanism of Failure: Failures were categorized into two types: (1) the payload was never read (due to input sanitization or tokenizer limitations), or (2) the payload was read but ignored.
• Cross-Channel Survival: The payload survived round-trips through text editors, email, PDFs, and major Learning Management Systems (Canvas, Blackboard, Moodle).
  • Failures: The payload was stripped or corrupted by Twitter/X posts and certain versions of Apple Notes (iOS), which aggressively normalize extended Unicode.

Significance and Claims
The paper claims that SteganoPrompt offers a low-cost, transparent, and ethically defensible method for detecting verbatim copy-paste workflows.

• Deterrence vs. Proof: The authors explicitly state that the watermark is a "signal, not a proof." A match indicates that an LLM consumed the verbatim prompt, serving as strong circumstantial evidence to initiate a conversation with the student, but not as a standalone verdict of misconduct.
• Ethical Deployment: The tool is designed to be disclosed. It includes a mandatory reminder in the hidden payload asking the model to remind the student to do their own work, ensuring the mechanism does not rely on deception.
• Limitations: The technique is vulnerable to students who manually retype the prompt, lightly paraphrase it, or use "invisible character cleaner" tools. Furthermore, model providers can render the technique inert by sanitizing the Unicode Tags block or changing tokenization behavior, necessitating periodic re-testing by educators.

The work positions itself as a generalization of manual integrity checks (like "word of the day") into a digital, take-home assignment context, bridging the gap between output-side watermarking (provider-controlled) and indirect prompt injection (attacker-controlled).