💻 computer science

When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning

This paper introduces Side-by-Side (SxS) Interleaved Reasoning, a framework that enables large language models to dynamically control the timing of content disclosure during generation to balance the trade-off between deliberation time and premature commitment, thereby improving accuracy-latency performance across various benchmarks.

Original authors: Jiaqi Wei, Xuehang Guo, Pengfei Yu, Xiang Zhang, Wanli Ouyang, Siqi Sun, Qingyun Wang, Chenyu You

Published 2026-05-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiaqi Wei, Xuehang Guo, Pengfei Yu, Xiang Zhang, Wanli Ouyang, Siqi Sun, Qingyun Wang, Chenyu You

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are asking a brilliant but very cautious friend for help with a complex puzzle.

The Old Way: The "Silence Tax"
In the current way Large Language Models (LLMs) work, your friend has to think out loud. But here's the catch: as soon as they say a word, that word is public. They can't take it back.

If they stay silent to think deeply, you sit there waiting in awkward silence (the "Silence Tax").
If they start talking immediately to be polite, they might say something wrong or half-baked. Because they already said it, they are now "committed" to that idea, which might make it harder for them to correct their course later. They are forced to keep building on a shaky foundation just because they spoke too soon.

The New Idea: Side-by-Side (SxS) Reasoning
This paper introduces a new way for the AI to talk, called Side-by-Side (SxS) Interleaved Reasoning.

Think of it like a live cooking show with a secret ingredient.

The "Think" Mode (Private): The chef (the AI) is in the kitchen, chopping, tasting, and mixing ingredients. You can't see this part. It's their private workspace.
The "Speak" Mode (Public): The chef steps out to the counter and tells you, "I'm adding salt now."
The Magic Rule: The chef is only allowed to step out and speak if they have already tasted the dish and are 100% sure that the salt is the right move. They don't just shout out guesses to fill the silence.

How It Works (The "Entailment" Trick)
The paper teaches the AI a specific rule: "Don't speak until your thinking supports what you're about to say."

Training the Chef: The researchers took thousands of examples where an AI solved a problem. They used a "supervisor" (another AI) to check: "Does this specific thought step actually prove this specific answer step?"
The Interleaved Dance: They created a new training format where the AI practices switching back and forth between "Thinking" (private) and "Speaking" (public).
- Think: "I need to calculate the area." (Private)
- Think: "The formula is length times width." (Private)
- Speak: "The area is 50." (Public, because the thinking just proved it).
- Think: "Wait, let me double-check the units." (Private)
- Speak: "So, 50 square meters." (Public).

The Results: Best of Both Worlds
The paper tested this on math problems (AIME25) and science questions (GPQA-Diamond) using two different sizes of AI models.

Faster Updates: Instead of waiting 20,000 tokens (a very long time) for the final answer, the AI now gives you small, verified chunks of the answer much earlier. It's like getting a progress bar that actually moves, rather than a spinning wheel.
No "Filler": Because the AI is trained to only speak when it has proof, it doesn't just babble nonsense to fill the silence.
Accuracy Remains High: Sometimes, forcing an AI to talk early makes it dumber. But because this method uses a two-step training process (first learning how to switch modes, then using reinforcement learning to make sure the answers are still correct), the AI stays smart.
The "Pareto" Win: The paper claims this creates a better balance (a "Pareto trade-off"). You get faster, more frequent updates without sacrificing the quality of the final answer.

In Summary
This paper solves the "Silence Tax" by teaching the AI to be a confident, verified speaker. It allows the model to keep its "thinking hat" on while it works, and only take it off to share a piece of the solution when it knows that piece is solid. This makes the interaction feel faster and more responsive, without forcing the AI to guess or lie to fill the silence.

Technical Summary: When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning

Problem Formulation

Current single-stream autoregressive Large Language Models (LLMs) face a structural limitation: the coupling of internal state updates with public disclosure. In standard interfaces, every generated token simultaneously updates the model's hidden state and becomes an irreversible public commitment. This creates a "silence tax" where users must wait for the entire reasoning process to complete before seeing task-relevant content, or conversely, risk "premature commitments" if the model streams early, potentially biasing subsequent generations with unsupported claims.

While Chain-of-Thought (CoT) prompting improves accuracy by eliciting intermediate reasoning, it exacerbates this tension by manifesting deliberation as long, user-visible preambles. Existing solutions either rely on fixed templates/heuristics for disclosure or incentivize earlier output in ways that encourage low-information filler. The core problem addressed is how to make disclosure timing a learnable decision variable within standard autoregressive decoding, ensuring that early visible text is strictly supported by the reasoning produced up to that point.

Methodology: Side-by-Side (SxS) Interleaved Reasoning

The authors propose Side-by-Side (SxS) Interleaved Reasoning, a framework that treats visibility as a controllable attribute within a single token stream. The model chooses between two actions at each step:

Think (R): Private deliberation (non-disclosed).
Speak (A): User-facing disclosure.

Both actions occur within the same autoregressive context; "Speak" tokens form a prefix of the final response that the model chooses to reveal earlier, while "Think" tokens allow for continued private refinement without contradicting earlier commitments.

Training Pipeline

The approach employs a two-stage training process to learn pacing without incentivizing filler:

Entailment-Aligned Supervised Fine-Tuning (SFT):
- Data Construction: Standard $(x, r, a)$ triples (input, reasoning, answer) are transformed into interleaved trajectories.
- Alignment: An entailment checker (implemented via a large LLM) determines the largest answer prefix supported by a given reasoning prefix.
- Interleaving: The model is trained to emit answer blocks only when the entailment boundary advances. This ensures that "earlier" disclosure is synonymous with "supported" disclosure.
- Trailing Reasoning: If the full answer is supported before the reasoning sequence ends, the remaining reasoning is appended as a trailing block to preserve post-solution behaviors (e.g., self-checks).
Reinforcement Learning (RL):
- Objective: SFT teaches the dual-action semantics, but the interleaved format induces a distribution shift that can degrade final-task accuracy. RL is used to recover reasoning performance under the new format.
- Algorithm: Group Relative Policy Optimization (GRPO) is used with an outcome-only reward (correctness of the final answer).
- Granularity Control: The authors explore an optional reward shaping mechanism using a quadratic program to encourage shorter reasoning blocks (higher granularity) while strictly preserving the correctness signal (separating correct from incorrect rollouts).

Key Contributions

Disclosure as Control: The paper formalizes disclosure timing as a sequential decision problem (think vs. speak) within standard autoregressive decoding, turning visibility into a controllable attribute without requiring architectural changes or separate models.
Justified Early Disclosure: By constructing entailment-aligned interleaved trajectories, the method ensures that early disclosures are safe to show, avoiding premature commitments and filler.
Learning Pareto Trade-offs: The combination of SFT and RL demonstrates improved accuracy–latency trade-offs across different model architectures (MoE vs. dense), scales, and domains.

Experimental Results

The authors evaluated SxS on two Qwen3 architectures (MoE Qwen3-30B-A3B and dense Qwen3-4B) across in-domain (AIME25) and out-of-domain (GPQA-Diamond, LiveCodeBench, KOR-Bench) benchmarks.

Breaking the Silence Tax: SxS significantly reduces the Average Inter-Response Wait (AIRW), a token-level proxy for user waiting time. For example, on Qwen3-4B, AIRW dropped from ~21,316 tokens (Standard CoT) to ~8,519 tokens (SxS), indicating more frequent partial disclosures.
Accuracy Recovery: SFT initially causes a drop in accuracy (e.g., Qwen3-30B-A3B dropped from 76.9% to 50.8% post-SFT) due to the distribution shift. However, the subsequent RL stage successfully recovers accuracy (restoring it to 79.2%) while maintaining the interleaved structure.
Out-of-Domain Robustness: SxS mitigates catastrophic forgetting on out-of-domain tasks. On GPQA-Diamond, Standard CoT after math-focused RL saw a significant drop (e.g., Qwen3-4B from 55.9% to 19.0%), whereas SxS retained substantially higher accuracy (49.3%).
Granularity Stability: The interleaved behavior remains robust even under outcome-only RL. While RL tends to slightly reduce interleaving frequency, it does not collapse back to a single monolithic reasoning block. Granularity can be actively controlled via reward shaping if desired.

Significance and Claims

The paper positions response pacing as a practical and underexplored control dimension for reasoning models. The authors claim that SxS improves the accuracy–content-latency Pareto trade-off without architectural modifications. By making disclosure a learned commitment policy tied to a concrete support condition (entailment), the method allows models to reveal partial progress earlier while maintaining reliability.

The findings suggest that the perceived trade-off between deliberation and responsiveness is largely shaped by the single-stream disclosure convention. SxS demonstrates that pacing can be learned as a stable behavioral policy rather than imposed by heuristics, enabling "anytime interaction" where the model discloses justified partial progress early and continues deliberation privately. The authors note that while absolute accuracy gains are sometimes modest, the consistent reduction in delay between meaningful visible updates represents a significant improvement in user-perceived responsiveness.