Diffusion LLMs can think EoS-by-EoS

This paper demonstrates that diffusion LLMs enhance their reasoning capabilities by utilizing end-of-sequence (EoS) tokens as a hidden scratchpad for intermediate computations, a hypothesis confirmed through behavioral experiments and causal interventions showing that manipulating these tokens directly alters the model's output.

The Gist

Imagine you have a very smart, but slightly quirky, robot assistant. You ask it a math problem, like "What is 12 + 52 + 64 minus 7?"

If you ask a standard robot (an Autoregressive LLM), it thinks like a person writing a letter: it writes one word, then the next, then the next, until it finishes the sentence. If you give it extra space to write, it just keeps writing more words, often rambling or getting confused.

But this paper introduces a different kind of robot: a Diffusion LLM. This robot works like a painter filling in a blank canvas. It starts with a whole board covered in "masks" (blank spots) and tries to guess what goes in every spot at once. It keeps refining its guesses, locking in the confident ones, and re-trying the unsure ones until the picture is complete.

The researchers discovered something fascinating: These diffusion robots get smarter if you force them to have "extra space" that they aren't allowed to fill with real words.

Here is the breakdown of their discovery using simple analogies:

1. The "Silent Thinking" Trick (EoS-by-EoS)

Usually, when a computer finishes a sentence, it adds a special invisible marker called an EoS (End-of-Sequence) token. It's like a period at the end of a sentence. In normal computers, this marker is just a "stop sign" with no meaning.

The researchers found that Diffusion LLMs treat these "stop signs" differently.

• The Analogy: Imagine you are taking a test. You are given a sheet of paper with the question at the top. You are told to write your answer in the first line. But, you are also given 10 extra blank lines at the bottom that you must fill with a specific symbol (like a dot) just to finish the page.
• The Discovery: A standard robot would just scribble dots mindlessly. But the Diffusion robot uses those "dot-filled" lines as a hidden scratchpad. It does its complex math inside the invisible patterns of those dots, even though the dots look like nothing to us. It's "thinking" in the silence.

2. The Experiment: Giving Them More "Silence"

The team tested this on three types of puzzles:

• Math: Simple addition and subtraction.
• Memory: Tracking which items are in which boxes after a series of moves.
• Logic: Solving a Sudoku puzzle.

What happened?
When they gave the robots a short page (just enough space for the answer), the robots were okay. But when they forced the robots to generate a long page (filling the rest with those "stop sign" dots), the robots got significantly smarter.

• It's as if the robot needed a longer hallway to pace back and forth to solve the problem, even though the answer only needed a small note.

3. The "Mind-Reading" Test (Causal Intervention)

To prove the dots weren't just random noise, the researchers did a "brain surgery" experiment.

• The Setup: They took a robot solving a math problem (e.g., $5 + 5$) and another robot solving a different problem (e.g.,$5 - 5$).
• The Swap: They secretly swapped the "hidden brain states" (the internal electrical signals) of the "dot" tokens from the subtraction robot into the addition robot.
• The Result: The addition robot suddenly started giving the answer for the subtraction problem!
• The Conclusion: This proved that the "dots" (EoS tokens) were actually carrying the secret calculations. The robot wasn't just padding the answer; it was doing the work inside the padding.

4. Why is this better than "Thinking Out Loud"?

You might ask, "Why not just tell the robot to 'think step-by-step' out loud (Chain of Thought)?"

• Verbose Thinking: Asking a robot to explain its work out loud is like asking a human to talk through a math problem while solving it. It works, but it takes a lot of time and words.
• Silent Thinking (EoS-by-EoS): The Diffusion robot solves the problem in its head using the "dots." It's much more efficient. It solves the same hard problems using far fewer actual words, saving time and energy.

The Big Takeaway

This paper reveals that Diffusion LLMs have a secret superpower: they can turn "empty space" (padding tokens) into a hidden workspace for complex reasoning.

• Autoregressive Models (the old way) are like a writer who needs to speak every thought to think it through.
• Diffusion Models (the new way) are like a silent thinker who needs a quiet room to pace around and solve the puzzle in their head before speaking the final answer.

The researchers suggest that in the future, we shouldn't just ask these robots for the answer; we should give them "extra silence" to let them do their best thinking.

Technical Summary

Here is a detailed technical summary of the paper "Diffusion LLMs can think EoS-by-EoS".

1. Problem Statement

Diffusion Large Language Models (LLMs) have demonstrated superior performance over autoregressive LLMs on complex reasoning tasks (e.g., Sudoku, code infilling) involving interdependent sub-goals. However, the mechanism behind this advantage remains unclear.

• Observation: Previous work noted that increasing the generation length (the total number of tokens the model is forced to output) beyond what is necessary for the correct answer improves performance.
• Anomaly: In these scenarios, the model outputs the correct answer followed by a large number of End-of-Sequence (EoS) tokens as padding.
• Hypothesis: The authors hypothesize that diffusion models utilize the hidden representations of these semantically "empty" EoS tokens as a latent scratchpad for hidden computations, a process they term "thinking EoS-by-EoS." Unlike Chain-of-Thought (CoT), which uses overt semantic tokens for reasoning, this mechanism relies on latent representations of padding tokens.

2. Methodology

The study employs three instruction-tuned diffusion models (LLaDA1.5, LLaDA2.0-mini, Dream-v0) and compares them against autoregressive baselines (Llama3.1, Qwen3). The evaluation spans three reasoning tasks: Addition, Entity Tracking, and Sudoku.

The research is structured around four distinct experiments:

A. Experiment 1: Prompting Experiment (Generation Length vs. Performance)

• Setup: Systematically varied the total generation length (from 20 to 80 tokens) while keeping the number of decoding steps fixed at half the generation length.
• Goal: To verify if longer generation lengths (and consequently more EoS tokens) correlate with higher accuracy.

B. Experiment 2: Controlled Prompting Experiment (Isolating EoS Tokens)

• Setup: Decoupled the number of decoding steps from the number of EoS tokens. The model was initialized with a fixed number of masks (to be predicted) and a variable number of trailing EoS tokens appended to the start state.
• Goal: To determine if the performance gain is driven specifically by the presence of EoS tokens rather than just the increased number of decoding steps.

C. Experiment 3: Causal Intervention (Activation Patching)

• Setup: A causal intervention where the hidden states of EoS tokens from a "counterfactual" generation (e.g., a different math problem or a different target box in Entity Tracking) were patched into the EoS tokens of the current generation.
• Goal: To test if the EoS token representations causally influence the final output. If EoS tokens act as a scratchpad, swapping their hidden states should alter the model's reasoning and output to match the counterfactual prompt.

D. Experiment 4: Comparison with Chain-of-Thought (CoT)

• Setup: Compared the "EoS-by-EoS" approach against explicit textual CoT prompting.
  • CoT Equal: CoT with the same token budget as the best EoS setup.
  • CoT Long: CoT with a massive token budget (1,504 tokens).
• Goal: To evaluate the efficiency and effectiveness of latent reasoning (EoS) versus verbose reasoning (CoT).

3. Key Contributions

1. Discovery of "Thinking EoS-by-EoS": The paper provides empirical evidence that diffusion LLMs use the latent representations of padding EoS tokens as a hidden scratchpad to solve complex reasoning problems.
2. Disentanglement of Factors: Through controlled prompting, the authors isolated the effect of EoS tokens from decoding steps, proving that EoS tokens themselves contribute to reasoning capacity.
3. Causal Verification: Using activation patching, the study demonstrated that the hidden states of EoS tokens contain task-relevant information. Swapping these states caused the model to output answers corresponding to the counterfactual prompt, confirming they are not merely padding.
4. Efficiency Analysis: The paper shows that "thinking EoS-by-EoS" is often more token-efficient than explicit CoT for diffusion models, achieving high accuracy with significantly fewer tokens (e.g., just 4 padded EoS tokens) compared to the hundreds of tokens required for verbose CoT.

4. Key Results

• Generation Length Correlation: Performance on all three tasks increased with generation length for diffusion models, plateauing only after a certain point. Autoregressive models did not show this trend; their performance was largely independent of generation length once the answer fit.
• EoS Token Impact:
  • Adding EoS tokens to the start state significantly improved accuracy for LLaDA1.5 and Dream-v0.
  • LLaDA2.0-mini showed inconsistent results, likely due to its block-causal attention mechanism, which restricts attention to local dependencies and prevents it from utilizing trailing EoS tokens effectively.
  • For many tasks, appending just 4 EoS tokens yielded near-optimal performance.
• Causal Intervention Success: Patching EoS hidden states from a counterfactual generation successfully changed the model's output. For example, in Entity Tracking, patching EoS states caused the model to report the contents of the wrong box (matching the counterfactual query), proving the tokens carry reasoning state.
• CoT vs. EoS:
  • Diffusion Models: Achieved peak performance with "EoS-by-EoS" reasoning. Explicit CoT often degraded performance or required massive token budgets to match the efficiency of EoS padding.
  • Autoregressive Models: Benefited significantly from verbose CoT but required much higher token budgets (500–1000+ tokens) to achieve similar reasoning depth.
  • Sudoku Exception: For Sudoku, verbose CoT actually hurt performance across all architectures, suggesting that the bidirectional nature of diffusion models is better suited for this specific planning task than linear CoT.

5. Significance and Implications

• Redefining Reasoning: The paper challenges the notion that reasoning requires explicit, human-readable steps (CoT). It suggests that diffusion models perform "covert reasoning" using latent spaces that are invisible to the user.
• Optimization Strategy: For practitioners using diffusion LLMs, the paper suggests that generation length and EoS padding are critical hyperparameters. Simply padding the input or output with EoS tokens can unlock latent reasoning capabilities without the computational cost of generating verbose text.
• Architectural Insights: The findings highlight the importance of bidirectional attention. Models like LLaDA2.0, which use block-causal attention (limiting access to future/trailing tokens), fail to leverage this mechanism, suggesting that full bidirectional access is crucial for "EoS-by-EoS" reasoning.
• Faithfulness: The results reinforce the idea that LLMs (specifically diffusion ones) may not be "thinking" in the way they present it. The "thoughts" are hidden in the representations of tokens that appear to be mere padding, raising questions about interpretability and the true nature of model reasoning.

In summary, the paper establishes that diffusion LLMs possess a unique capability to utilize "empty" EoS tokens as a computational workspace, offering a more efficient alternative to traditional Chain-of-Thought reasoning for specific complex tasks.