💬 NLP

STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Models

STITCH is a novel generation method for Spoken Language Models that reduces latency by interleaving unspoken reasoning chunks with spoken response chunks, allowing the model to "think" while simultaneously playing audio to the user.

Original authors: Cheng-Han Chiang, Xiaofei Wang, Linjie Li, Chung-Ching Lin, Kevin Lin, Shujie Liu, Zhendong Wang, Zhengyuan Yang, Hung-yi Lee, Lijuan Wang

Published 2026-02-10

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Cheng-Han Chiang, Xiaofei Wang, Linjie Li, Chung-Ching Lin, Kevin Lin, Shujie Liu, Zhendong Wang, Zhengyuan Yang, Hung-yi Lee, Lijuan Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are having a conversation with a very smart friend. Usually, when you ask them a difficult math question, they either:

The "Mutterer": They start talking immediately, but they stumble, correct themselves, and eventually get the answer. (This is how most current AI voice models work).
The "Silent Thinker": They stare at you in total silence for 30 seconds while they do the math in their head, and then they finally start speaking. (This is how "Chain-of-Thought" AI works—it's smart, but the silence is awkward and feels like the machine is broken).

The researchers at Microsoft and National Taiwan University have created a third way, which they call STITCH.

The Analogy: The "Chef and the Sous-Chef"

Think of the AI as a professional kitchen team.

In the old way (The Silent Thinker), the Head Chef (the reasoning part) has to finish the entire recipe, cook the whole meal, and plate it perfectly before the Waiter (the speaking part) is even allowed to walk into the dining room. The customer sits there staring at an empty table, getting hungrier and more impatient.

STITCH works like a high-speed, synchronized dance between a Chef and a Sous-Chef:

The Sous-Chef (The Voice): They immediately bring out a small appetizer (the first few words of the response) to let the customer know, "Don't worry, the food is coming!" This eliminates that awkward "dead air" silence.
The Head Chef (The Thinking): While the Sous-Chef is busy serving that appetizer, the Head Chef is in the back, frantically chopping vegetables and cooking the main course (the complex reasoning).

Because the "serving" (speaking) takes much longer than the "chopping" (thinking), the Chef can actually finish the next part of the meal while the customer is still eating the first bite. They are stitching the thinking and the talking together so seamlessly that the customer never feels a gap in service.

How does it actually work? (The "Chunking" Secret)

Instead of thinking of a giant, long paragraph of logic, STITCH breaks everything into small "chunks."

Chunk 1: The AI does a tiny bit of "silent thinking" (just enough to get started).
Chunk 2: It immediately says a few words out loud.
The Magic Trick: While those words are being played through your speakers, the AI uses that "free time" to do a big chunk of silent thinking for the next part of the sentence.

Why is this a big deal?

It’s Smart: Because the AI is still "thinking" while it's talking, it performs much better on hard tasks like math. It doesn't just guess; it calculates.
It’s Fast: It doesn't have that long, awkward pause at the beginning. It feels like a real, responsive human conversation.
It’s Natural: It achieves the intelligence of a "silent thinker" with the speed of a "mutterer."

In short: STITCH allows AI to "think on its feet"—literally using the time it spends talking to do the heavy lifting of thinking.

Technical Summary: STITCH: Simultaneous Thinking and Talking with Chunked Reasoning

Published at: ICLR 2026

1. Problem Statement

Current Spoken Language Models (SLMs) are designed to receive speech input and generate spoken responses. However, they lack an internal, unspoken reasoning process. While humans use mental reasoning to formulate clear and concise spoken ideas, current SLMs typically generate speech tokens directly.

While "Chain-of-Thought" (CoT) reasoning can be integrated into text-based LLMs to improve accuracy, applying it to SLMs presents a major challenge: latency. A naive approach—generating a complete text-based CoT before starting to speak—results in significant, uncontrollable delays, making real-time interaction impossible. The goal is to enable SLMs to "think" (reason) and "talk" (speak) simultaneously without increasing the perceived latency for the user.

2. Methodology

The authors propose STITCH, a novel generation pipeline that alternates between generating unspoken reasoning chunks and spoken response chunks. The core insight is that the time required to play an audio chunk to a user is significantly longer than the time required for the model to generate the next set of tokens.

The paper introduces two primary variants of the STITCH framework:

STITCH-R (Reasoning First): The model generates a partial reasoning chunk ( $N_{reason}$ tokens), followed by a chunk of text tokens ( $N_{text}$ ) and speech tokens ( $N_{speech}$ ). While the speech decoder is synthesizing and playing the first audio chunk to the user, the model uses that "free time" to generate the next partial reasoning chunk. This creates a continuous cycle of thinking while the previous thought is being verbalized.
STITCH-S (Speaking First): To eliminate the initial latency caused by waiting for the first reasoning chunk, STITCH-S reverses the order. It starts by generating the first spoken response chunk (text + speech) immediately, and then generates the reasoning chunk during the playback of that audio. This ensures the model has the same "first packet latency" as standard SLMs that do not reason.

Training Approach:
The models are fine-tuned from the GLM-4-Voice-9B backbone. The training data is constructed by taking existing reasoning traces and "stitching" them into interleaved sequences of reasoning tokens, text tokens, and speech tokens.

3. Key Contributions

First Unspoken Reasoning for SLMs: The authors are the first to introduce an internal, silent thinking process to spoken language models.
Latency-Neutral Reasoning: They demonstrate that reasoning can be added to SLMs without increasing the initial response latency (specifically via the STITCH-S variant).
Chunked Generation Paradigm: They introduce a method to utilize the "idle time" during audio playback to perform background computation (reasoning).

4. Results

The performance was evaluated on math reasoning and non-reasoning (knowledge-intensive) datasets.

Reasoning Performance: STITCH models significantly outperform baselines. On five math QA datasets, STITCH-R and STITCH-S outperformed non-reasoning baselines by over 15%. Even the "Speaking First" variant (STITCH-S), which has zero added latency, showed massive improvements over models without reasoning.
Latency: STITCH-S achieves the same first-packet latency as standard interleaved SLMs, effectively providing "free" reasoning capability in terms of time.
Non-Reasoning Tasks: The models performed comparably to standard SLMs on factual QA and dialogue tasks, proving that adding reasoning does not degrade general conversational abilities or speech quality (measured via UTMOSv2 and GPT-4o fluency scores).
Human Evaluation: Human testers rated STITCH-S as the most responsive model, followed by STITCH-R, both of which were perceived as more responsive than the "Think Before Speaking" (TBS) baseline.

5. Significance

STITCH represents a significant step toward making AI assistants more "human-like." By decoupling the internal cognitive process from the external verbal output, the researchers have solved the fundamental trade-off between intelligence (reasoning depth) and interactivity (low latency). This framework allows for the deployment of highly capable, reasoning-heavy models in real-time voice applications, such as sophisticated virtual assistants, tutors, and real-time translators, without the awkward silences typically associated with complex computation.