🤖 machine learning

How Transparent is DiffusionGemma?

This paper demonstrates that while DiffusionGemma's continuous latent space initially appears to hinder transparency, mapping its denoising steps through an interpretable token bottleneck significantly improves variable transparency and reveals unique reasoning phenomena, ultimately showing that the model remains as monitorable as its autoregressive counterpart.

Original authors: Joshua Engels, Callum McDougall, Bilal Chughtai, Janos Kramar, Senthoran Rajamanoharan, Cindy Wu, Arthur Conmy, Asic Q Chen, Jean Tarbouriech, Min Ma, Brendan O'Donoghue, João Gabriel Lopes de Oliveir

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Joshua Engels, Callum McDougall, Bilal Chughtai, Janos Kramar, Senthoran Rajamanoharan, Cindy Wu, Arthur Conmy, Asic Q Chen, Jean Tarbouriech, Min Ma, Brendan O'Donoghue, João Gabriel Lopes de Oliveira, Rohin Shah, Neel Nanda

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Question: Can We See Inside the Machine?

Imagine you have two different types of chefs trying to write a story.

Chef A (The Autoregressive Model, like Gemma 4): This chef writes one word at a time, from left to right. They write "The," then "cat," then "sat." You can watch them write every single word in real-time. It's easy to see exactly how they are thinking because their "thought process" is literally the words they are typing.
Chef B (DiffusionGemma): This chef starts with a giant canvas full of random scribbles (like static on an old TV). They don't write word-by-word. Instead, they look at the whole messy picture and slowly "clean it up" over and over again. In the first round, the scribbles might look like gibberish. In the second round, they look a bit more like words. By the 48th round, it's a perfect story.

The Problem: With Chef B, the "middle steps" (the scribbles between rounds) aren't just words; they are mathematical smudges that humans can't read. The paper asks: Is Chef B hiding its thinking in those smudges, making it impossible for us to know what it's doing?

The Investigation: Three Ways to Check Transparency

The researchers broke down "transparency" (how easy it is to understand the model) into three main tests.

1. The "Hidden Steps" Test (Opaque Serial Depth)

Think of a relay race.

Chef A passes the baton (the thought) to the next runner immediately. The distance between runners is short.
Chef B seems to pass the baton through a long, dark tunnel 48 times before the next runner sees it. If that tunnel is dark (uninterpretable), the race is 28.6 times harder to follow than Chef A's race.

The Discovery: The researchers found a way to shine a flashlight into that tunnel. They realized that even though the "smudges" look like math, they actually contain mostly guesses about what the final words will be. If we treat those guesses as readable words, the "dark tunnel" shrinks dramatically. Suddenly, Chef B's race is only 1.1 times harder to follow than Chef A's. It's almost as transparent as the standard model.

2. The "Guessing Game" Test (Variable Transparency)

The researchers asked: "Are those smudges actually just guesses?"
They looked at the "scribbles" between Chef B's cleaning rounds. They found that 85% to 90% of the time, the model was just holding up a sign saying, "I think the next word is 'cat', or maybe 'dog'."

The Good News: These signs are mostly just guesses about the final answer. They aren't secret codes.
The Bad News: Sometimes, the model holds up a sign with a bunch of random words just to fill space. But mostly, the "thinking" is visible if you look closely at the top guesses.

3. The "Safety Monitor" Test (Monitorability)

Imagine a safety inspector trying to catch a chef if they try to write something dangerous.

The researchers tested if an inspector could spot bad behavior in Chef B's story just by reading the final story.
The Result: The inspector was just as good at catching Chef B as they were at catching Chef A. Even though Chef B thinks differently, the final story gives away the same clues about whether the model is being "naughty" or "safe."

The Weird Stuff: How Chef B Thinks Differently

Even though Chef B is transparent, it thinks in ways Chef A never could. The paper found some cool, weird behaviors:

Thinking Backwards (Non-Chronological Reasoning): Chef A must write the beginning before the end. Chef B can write the ending first, then go back and fix the beginning.
- Example: Chef B might guess the answer to a math problem is "9," write it down, then do the math in the next round, realize it's wrong, and change the "9" to an "8" without rewriting the whole page.
The "Blurry" Effect (Token Smearing): Sometimes Chef B knows what it wants to say but not where to put it.
- Example: It might put the word "bracket" in three different spots on the page at the same time, like a blurry photo, and then sharpen it into one spot in the final round.
Holding Two Worlds at Once (Sequence Smearing): Chef B can imagine two different stories happening at the same time.
- Example: It might be 50% sure the answer is "12" and 50% sure it's "9," keeping both versions of the sentence alive in its mind until the very last second when it picks one.
Secret Placeholders (Intermediate Context): Sometimes Chef B uses a temporary word to help it think, but then deletes it before showing you the final answer.
- Example: To solve a math problem, it might write the number "3" to help with a calculation, then swap it for the word "Gold" in the final output. The "3" was necessary for the thinking, but it never appears in the final story.

The Conclusion

The paper concludes that DiffusionGemma is surprisingly transparent.

Even though it uses a different, more complex way of thinking (cleaning up a canvas instead of writing word-by-word), we can still see what it's doing.

Its "hidden steps" are mostly just guesses about the final words.
Safety monitors can catch it just as well as they can catch standard models.
It does some cool, non-linear thinking (fixing the past, holding two options at once), but we can still watch it happen.

The Warning: The authors say this might only be true for this specific model. If future models are trained differently, those "smudges" might become truly secret codes that we can't read. But for now, DiffusionGemma is safe to watch.

Technical Summary: How Transparent is DiffusionGemma?

Problem Statement

The paper addresses the critical challenge of LLM reasoning transparency, specifically concerning the emerging class of text diffusion models like DiffusionGemma. Unlike standard autoregressive models that generate tokens sequentially (left-to-right), diffusion models iteratively refine a "canvas" of tokens through denoising steps. A primary concern is that these models perform significant computation in a continuous latent space (self-conditioning vectors) between denoising steps. This raises the question: Does this architecture render the model's reasoning less transparent, thereby hindering our ability to understand decisions, detect misalignment, and debug surprising behaviors?

The authors decompose transparency into two core components:

Variable Transparency: Can we understand intermediate snapshots of the model's computational state?
Algorithmic Transparency: Can we reconstruct the process by which the model arrived at its outputs using these snapshots?

Methodology

The authors employ a multi-faceted approach to evaluate DiffusionGemma's transparency, comparing it against the autoregressive Gemma 4 model.

1. Opaque Serial Depth Analysis

The authors formalize opaque serial depth as the longest path of serial computation in a model's circuit that does not pass through an interpretable bottleneck (e.g., natural language tokens).

They analyze DiffusionGemma's architecture, which includes a bottleneck between denoising steps consisting of $C$ natural language tokens and $C$ self-conditioning vectors.
They calculate the opaque serial depth under two assumptions: (a) the self-conditioning vectors are uninterpretable, and (b) they are interpretable.

2. Bottleneck Interpretability Experiments

To test if the latent self-conditioning vectors ( $S_t$ ) can be made interpretable without performance loss:

Logit Lens Ablation: They restrict the information flow in the bottleneck to only the top- $k$ tokens or tokens with probability $>p$ using the Logit Lens technique.
Performance Evaluation: They measure capability on benchmarks (Natural2Code, LiveCodeBench, AMC/AIME/IMO, GPQA) to ensure that restricting the bottleneck does not degrade model performance.
Token Identity Analysis: They analyze the top intermediate tokens to determine if they represent "guesses" for final tokens (interpretable) or opaque reasoning.

3. Monitorability Evaluation

The authors adapt the monitorability evaluations introduced by Guan et al. (2025). These tests measure whether a monitor can extract information about a model's reasoning (e.g., detecting reward hacking or prompt injections) given access to the model's chain of thought (CoT). They compare DiffusionGemma and Gemma 4 across three categories:

Intervention: Does the monitor detect changes in reasoning corresponding to input interventions?
Process: Can the monitor find evidence of specific solution paths?
Outcome-property: Can the monitor detect if the model leverages specific output properties (e.g., subverting tests)?

4. Algorithmic Transparency Case Studies

The authors conduct qualitative case studies to identify unique reasoning phenomena in diffusion models that differ from autoregressive constraints. They utilize a custom visualization library (Summary, Line Graph, and Sampling Table views) to track token probabilities across denoising steps.

Key Results

1. Opaque Serial Depth

Naive View: If the self-conditioning vectors are uninterpretable, DiffusionGemma's opaque serial depth is 28.6X higher than Gemma 4 (608,016 vs. 21,235 at 256k context).
Interpretable View: If the intermediate states are treated as interpretable, the opaque serial depth drops to 1.1X that of Gemma 4.
Conclusion: The "opacity" is contingent on whether the intermediate bottleneck can be mapped to natural language.

2. Variable Transparency (Bottleneck Interpretability)

Performance Preservation: Restricting the bottleneck to just the top 8 tokens ( $k=8$ ) or tokens with probability $>0.03$ results in no significant performance degradation on capability benchmarks.
Token Nature: Analysis of 800 WildChat prompts reveals that the top intermediate tokens are predominantly guesses for the final token or tokens at adjacent positions.
- Under a strict $p > 0.03$ threshold, >85% of tokens are categorized as "Final," "Adjacent," or "Semantically Similar."
- This suggests the load-bearing intermediate states are largely interpretable, reducing the effective opaque serial depth.

3. Monitorability

DiffusionGemma is found to be similarly monitorable to Gemma 4 across intervention, process, and outcome-property evaluations.
Note: DiffusionGemma generates shorter chains of thought (fewer characters) than Gemma 4, yet maintains comparable monitorability scores.

4. Algorithmic Transparency & Novel Phenomena

The study uncovers several non-autoregressive behaviors unique to DiffusionGemma:

Non-chronological Reasoning:
- Early Length Prediction: The model predicts the total sequence length (via padding token probabilities) before determining the specific content tokens.
- Retroactive Self-Correction: The model can generate an incorrect answer first, then generate reasoning, and finally correct the initial answer based on the reasoning (impossible in autoregressive models where early tokens are fixed).
- Skeleton-First Code Generation: The model often commits to the algorithmic structure (loops, logic) before filling in variable names or comments that appear earlier in the text.
Token and Sequence Smearing:
- Token Smearing: Probability mass for a single token (e.g., "bracket") is distributed across multiple adjacent positions before converging.
- Sequence Smearing: The model maintains superpositions of distinct candidate sequences (e.g., a single-digit vs. double-digit answer) simultaneously before collapsing to one.
Intermediate Context Reasoning: The model sometimes uses tokens as intermediate placeholders (e.g., generating a "3" to compute a Fibonacci sequence) which are then replaced in the final output. These tokens are causally necessary for the reasoning but do not appear in the final answer.

Significance and Claims

The paper claims that DiffusionGemma retains most of the transparency benefits of autoregressive CoT, provided that the intermediate bottlenecks are treated as interpretable.

Reassuring Findings: The authors find that DiffusionGemma is similarly monitorable to Gemma 4 and that its opaque serial depth can be reduced to near-parity with autoregressive models by mapping intermediate vectors to tokens.
Caveats and Limitations:
- The results rely heavily on DiffusionGemma's specific architectural choices (e.g., projection layers biasing vectors toward embeddings) and training paradigms.
- The authors caution that future latent reasoning models trained with different objectives (e.g., outcome-based RL) or architectures might have significantly more opaque latent spaces.
- Current monitorability evaluations span multiple canvases (acting more autoregressively); single-canvas monitorability (where complex reasoning occurs entirely within one diffusion loop) remains an open question and might be lower.
Future Direction: The paper advocates for using these transparency audits as a template for future latent reasoning models. It emphasizes the need for techniques like Natural Language Autoencoders (NLAs) and Activation Oracles to translate latent reasoning into human-understandable text if future models become less transparent.

In summary, while DiffusionGemma introduces new algorithmic complexities (non-chronological reasoning, smearing), the authors demonstrate that its reasoning remains largely accessible to human interpretation and monitoring, challenging the assumption that latent reasoning architectures are inherently opaque.