🤖 machine learning

dgMARK: Decoding-Guided Watermarking for Diffusion Language Models

The paper introduces dgMARK, a plug-and-play watermarking method for discrete diffusion language models that leverages their sensitivity to token unmasking order to embed detectable signals via parity-constrained candidate selection, ensuring robustness against various text editing operations.

Original authors: Pyo Min Hong, Albert No

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pyo Min Hong, Albert No

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: "Who Wrote This?"

Imagine Large Language Models (LLMs) as incredibly talented ghostwriters. They can write stories, code, and essays that look and sound exactly like they were written by humans. But this creates a problem: if a bad actor uses these models to write fake news or phishing emails, how do we know it wasn't a real person?

We need a way to tag the text so we can prove, "Hey, a computer wrote this." This is called watermarking.

The Old Way vs. The New Way

Most current watermarking methods work like a biased coin flip.

The Analogy: Imagine a chef (the AI) who usually picks ingredients based on what tastes best. The old watermarking method forces the chef to pick a "green" ingredient (like a specific type of pepper) 60% of the time, even if a "red" ingredient (like a tomato) would have tasted better.
The Flaw: This changes the flavor of the dish. The text might sound slightly robotic or lower quality because the AI is being forced to make unnatural choices just to hide a secret code.

The New Method: dgMARK (The "Traffic Controller")

The paper introduces dgMARK, a new method designed specifically for a newer type of AI called Diffusion Language Models (dLLMs).

What is a Diffusion Model?
Unlike the old "Autoregressive" models that write sentences strictly from left to right (like a typist), Diffusion models are more like a puzzle solver.

They start with a blank page full of question marks (masks).
They fill in the words one by one, but they can fill them in in any order they want. They might fill in the last word first, then the first word, then the middle.

The Secret Insight
The authors realized that while these models should theoretically work the same way regardless of the order they fill in the blanks, in reality, they are sensitive to the order. The order in which they reveal words changes the final result slightly.

The Analogy: The Traffic Controller
Instead of forcing the chef to pick a specific ingredient (changing the flavor), dgMARK acts like a Traffic Controller.

The AI looks at all the empty spots on the page and says, "I think I could put a 'cat' here, or a 'dog' there."
The Traffic Controller (dgMARK) has a secret rule: "If the word you want to put in matches a secret pattern (like a parity check), you get to go first."
The AI still picks the best word for that spot (the flavor doesn't change), but the order in which the words appear on the page is subtly guided by the secret rule.

How the Secret Code Works

The "secret rule" is based on a simple math trick called parity (odd vs. even numbers).

Imagine the AI is filling in a sentence. For every position (1st word, 2nd word, etc.), there is a secret list of "approved" words.
If the AI wants to fill in the 3rd word, and the word it wants is on the "approved" list for the 3rd spot, dgMARK says, "Great! Fill that one in now!"
If the word isn't on the list, the AI might wait a moment and fill in a different spot first.

Over the course of a whole paragraph, this creates a statistical pattern. If you check the text, you'll find that the words appear in positions that match the secret pattern more often than random chance would allow. It's like finding that a deck of cards has been subtly shuffled so that every 4th card is always a Heart.

Why This is Better

No Flavor Change: Because the AI still picks the best word for the job, the text sounds just as natural and high-quality as before. The paper shows that the "perplexity" (a measure of how confusing the text is) barely changes.
Hard to Remove: If someone tries to edit the text (adding, deleting, or swapping words), the "Traffic Controller" logic leaves a fingerprint. The paper uses a sliding window detector (like looking at the text through a moving magnifying glass) to find these patterns even if the text has been edited.
Plug-and-Play: It works with the AI's existing way of thinking. You don't need to retrain the model; you just add this "Traffic Controller" layer on top of the decoding process.

The Results

The authors tested this on several advanced models (like LLaDA and Dream).

Detectability: They could identify the watermarked text with very high accuracy, even when the text was edited or paraphrased.
Quality: The watermarked text was almost indistinguishable from non-watermarked text in terms of quality.
Robustness: Even if someone tried to "paraphrase" the text (rewrite it to hide the watermark), the method remained effective, especially when using a "lookahead" feature (where the AI plans one step ahead to ensure the pattern continues).

Summary

dgMARK is a clever way to watermark AI text by orchestrating the order in which words are revealed, rather than forcing the AI to pick specific words. It's like conducting an orchestra: you don't tell the musicians to play the wrong notes (which ruins the music); you just tell them when to play their best notes so that the rhythm follows a secret, detectable pattern.

Technical Summary: dgMARK – Decoding-Guided Watermarking for Diffusion Language Models

Problem Statement
Large Language Models (LLMs) present significant risks regarding disinformation, plagiarism, and copyright infringement, necessitating reliable provenance tools to distinguish machine-generated text from human-authored content. While watermarking is a practical solution, existing methods are predominantly designed for autoregressive models (ARMs) that generate text strictly left-to-right. These methods typically bias token probabilities (e.g., "green/red" list partitioning) to embed signals. However, Discrete Diffusion Language Models (dLLMs) operate differently: they iteratively denoise masked sequences and can finalize tokens in arbitrary orders. In dLLMs, the "previous token" context required by ARMs is often undefined during decoding, rendering traditional probability-biasing watermarks ineffective or requiring complex adaptations that alter the model's output distribution. Furthermore, while ideal dLLMs should be order-invariant, practical models exhibit sensitivity to the decoding order, creating a gap between theoretical invariance and practical behavior that remains underutilized for watermarking.

Methodology: dgMARK
The authors propose dgMARK (Decoding-Guided Watermarking), a method that embeds watermarks by steering the decoding order rather than reweighting the model's learned token probabilities. The core premise is that dLLMs select high-reward candidate tokens at each step; dgMARK exploits this by prioritizing the unmasking of positions where the candidate token satisfies a specific parity constraint.

Parity Constraint: Given a secret key $\xi$ , a deterministic hash function $f(v, \xi)$ maps each token $v$ to a binary value. For a decoding position $i$ , the "matching set" $G_i$ contains tokens where $f(v, \xi) \equiv i \pmod 2$ .
Decoding Strategy: During the decoding process, the algorithm identifies candidate tokens for all unrevealed positions. It prioritizes positions $j$ where the predicted token $v_j$ belongs to the matching set $G_j$ . Among these valid candidates, the position with the highest reward (based on the underlying decoding strategy, e.g., confidence, entropy, or margin) is selected for unmasking. If no candidate satisfies the parity condition, the system falls back to the standard selection process.
One-Step Lookahead: To prevent greedy choices from depleting future parity-matching opportunities, dgMARK offers a variant using a top- $k$ beam search with one-step lookahead. This variant evaluates how many future positions would remain parity-aligned if a specific token were committed, selecting the path that maximizes future alignment.
Detection: Detection relies on statistical testing of the parity-matching rate. Since the hash function is balanced, non-watermarked text should exhibit a matching rate near 0.5. Watermarked text exhibits a significantly elevated rate. To ensure robustness against post-editing (insertions, deletions, substitutions), the detector employs a sliding-window approach, aggregating $z$ -statistics across overlapping windows to handle index shifts.

Key Contributions

Order-Agnostic Watermarking: dgMARK is the first method to utilize the decoding order itself as the primary channel for watermarking in dLLMs, avoiding direct modification of token probabilities.
Plug-and-Play Compatibility: The method functions as a wrapper around existing decoding strategies (confidence, entropy, margin-based) and does not require retraining the model or altering its internal parameters.
Robustness: The sliding-window detection mechanism ensures the watermark remains detectable even after significant text modifications, including paraphrasing and token-level edits.
Quality Preservation: By not explicitly biasing the probability distribution (logits), dgMARK aims to minimize the degradation of text quality compared to probability-shifting baselines.

Experimental Results
The authors evaluated dgMARK on benchmark datasets (C4, Writing Prompts) using state-of-the-art dLLMs (LLaDA, LLaDA 1.5, LLaDA 2.0, Dream).

Detectability: dgMARK achieves strong detectability. With a 3-beam search lookahead, it reaches True Positive Rates (TPR) of nearly 100% at False Positive Rates (FPR) of 0.01% on the C4 dataset, outperforming or matching baselines like KGW (Kirchenbauer et al., 2023) and PATTERN-MARK.
Text Quality: The method incurs minimal degradation in text quality. Perplexity (PPL) increases are consistently smaller than those observed with KGW and PATTERN-MARK. Downstream benchmark performance (MMLU, GSM8K, HumanEval) shows only minor drops, with dgMARK generally preserving capability better than probability-biasing methods.
Robustness: Under token-level attacks (insertion, deletion, substitution) with modification budgets up to 40%, dgMARK maintains high Area Under the Curve (AUC) scores. It also demonstrates resilience against paraphrasing attacks using DIPPER and Llama 3, particularly when combined with beam search.
Scalability: Detectability improves with sequence length, making the method particularly effective for long-form generation tasks.

Significance and Claims
The paper claims that dgMARK establishes decoding order as a practical and complementary watermarking channel for dLLMs. The authors emphasize that their approach offers a distinct advantage over existing paradigms: it achieves reliable provenance without the direct distortion of the model's learned conditional probabilities. The work highlights that the inherent order-sensitivity of practical dLLMs, often viewed as a limitation or a side effect, can be leveraged as a control knob for security. The authors modestly note that the method's effectiveness relies on the empirical imperfection of current models (i.e., their sensitivity to decoding order), acknowledging that a perfectly trained, order-invariant dLLM would theoretically render this specific watermarking approach ineffective. Nevertheless, for current and near-future dLLMs, dgMARK provides a robust, low-distortion solution for content provenance.