⚡ electrical engineering

Closing the Modality Reasoning Gap for Speech Large Language Models

The paper introduces TARS, a reinforcement learning framework that utilizes asymmetric reward design with representation and behavior alignment signals to effectively close the modality reasoning gap between speech and text inputs in Speech Large Language Models, achieving state-of-the-art performance on challenging benchmarks.

Original authors: Chaoren Wang, Heng Lu, Xueyao Zhang, Shujie Liu, Yan Lu, Jinyu Li, Zhizheng Wu

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chaoren Wang, Heng Lu, Xueyao Zhang, Shujie Liu, Yan Lu, Jinyu Li, Zhizheng Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Ear vs. Brain" Disconnect

Imagine you have a brilliant student (the AI) who is a genius at reading books. They can solve complex math problems, debate history, and write essays perfectly when reading text.

However, when you speak to this student out loud, they suddenly become confused. They hear your voice, but their "thinking brain" gets lost. They might stumble over simple logic or give wrong answers, even though they know the answer if they could just read the question.

This is the Modality Reasoning Gap. The AI is great at reading (Text) but terrible at listening and thinking at the same time (Speech).

Why does this happen?
The paper suggests that when the AI listens, the "signal" gets distorted as it travels through the computer's layers (like a game of "Telephone"). By the time the message reaches the thinking part of the brain, it has drifted away from the clear logic it uses when reading. It's like trying to solve a puzzle while wearing foggy glasses; the pieces are there, but you can't see how they fit together.

The Solution: TARS (The "Twin-Track" Trainer)

The researchers created a new training method called TARS (Trajectory Alignment for Reasoning in Speech). Think of TARS as a strict but helpful coach who forces the student to think the same way whether they are reading or listening.

Here is how TARS works, using two main tools:

1. The "Internal GPS" (Representation Alignment)

The Analogy: Imagine the AI's brain is a multi-story building. When the student reads a question, they walk up the stairs in a straight, logical line. When they listen, they tend to wander off into the wrong rooms or take the elevator to the wrong floor.
What TARS does: It installs a GPS tracker. It checks the student's "internal location" at every single floor of the building. If the student is listening and their thoughts start to drift away from the straight path they take when reading, the GPS gives a gentle "correction signal."
The Goal: To make sure the process of thinking is identical, regardless of whether the input is sound or words.

2. The "Final Verdict" Check (Behavior Alignment)

The Analogy: Sometimes, even if the student takes a slightly different route up the stairs, they might still arrive at the right answer. But if they arrive at the wrong answer, we need to know.
What TARS does: This is the final exam. It compares the student's spoken answer to the perfect written answer. It asks, "Do these two answers mean the same thing?" If the spoken answer is semantically different (even if the words are different), it gets a penalty.
The Goal: To ensure the final result makes sense, not just that the internal steps were correct.

The Secret Sauce: The "Asymmetric Reward"

Most training methods are like a teacher who only gives a grade at the very end of the test. If the student gets the final answer wrong, they get a zero, and the teacher doesn't know where they went wrong. This is frustrating and inefficient.

TARS uses a Reinforcement Learning approach with a special twist:

The "Moving Target": Instead of comparing the student to a static textbook, TARS compares the student's listening performance to their own reading performance in real-time.
The "Safety Net": Even if the student gets the final answer wrong (which happens often with speech), TARS still gives them points for keeping their internal thoughts aligned with their reading logic. This prevents the student from giving up or getting confused when they make a mistake.

Think of it like learning to ride a bike.

Old Method: You fall down, and the coach says, "Bad job, try again." You don't know if you fell because you pedaled too hard or turned too sharply.
TARS Method: The coach has a harness. If you start to wobble (drift in reasoning), the harness gently pulls you back to the straight path before you fall. Even if you still wobble, the coach praises you for staying close to the straight line.

The Results: A Genius Who Can Finally Listen

The researchers tested this on two major benchmarks (MMSU and OBQA), which are like the "SATs" for AI reasoning.

Before TARS: The AI was like a smart person with a hearing aid that made everything sound muffled. Their reasoning score on speech was much lower than on text.
After TARS: The AI's listening skills caught up to its reading skills. In fact, for one of the models, the listening score actually became better than the original reading score!

Why is this a big deal?

No New Hardware: They didn't need to build a new, bigger brain. They just taught the existing brain how to listen better.
Real-World Use: This means future voice assistants won't just be good at dictating notes; they will be able to hold complex conversations, solve problems, and reason through your spoken questions just as well as if you typed them.

Summary

The paper solves the problem of AI being "smart when reading, but dumb when listening" by teaching the AI to keep its internal thought process perfectly aligned, whether the input is sound or text. They did this by acting as a real-time coach, checking both the student's internal steps and their final answers, ensuring the AI doesn't lose its way just because it's listening.

1. Problem Statement: The Modality Reasoning Gap

Despite the rapid advancement of Speech Large Language Models (Speech LLMs), a significant performance disparity exists between text and speech inputs. While these models can process speech, their reasoning capabilities on audio inputs are markedly weaker than on text.

The Gap: Empirical evidence shows that when a Speech LLM processes a reasoning task via audio, its accuracy drops significantly compared to the same task processed via text.
Root Causes:
1. Representational Drift: As speech data passes through Transformer layers, the hidden states diverge from their text-conditioned counterparts, leading to a loss of logical coherence.
2. Behavioral Deviation: Existing methods often rely on off-policy supervision (e.g., forcing the model to generate specific tokens). This causes exposure bias, where a single token error shifts the model into an unsupervised state, causing the reasoning trajectory to collapse.
3. Limitations of Prior Work:
  - Input-side fusion (freezing the LLM backbone) only achieves surface-level alignment and cannot adapt to speech-specific dynamics.
  - Output-side supervision (distillation/prompt-switching) enforces strict token-level matching, which is often unreachable for speech distributions and fails to teach the model how to self-correct its reasoning path.

2. Methodology: TARS Framework

The authors propose TARS (Trajectory Alignment for Reasoning in Speech), a reinforcement learning (RL) framework designed to align speech-conditioned reasoning trajectories with text-conditioned ones without modifying the model architecture.

Core Components

A. Asymmetric Reward Design
TARS employs a composite reward function for speech-conditioned completions, while text completions are optimized using a standard base reward.
$R_{total} = R_{base} + \alpha \cdot R_{rep} + \beta \cdot R_{beh}$

Base Reward ( $R_{base}$ ): Optimizes task accuracy (using an answer extractor like xFinder) and output format compliance.
Representation Alignment ( $R_{rep}$ ):
- Goal: Mitigate representational drift.
- Mechanism: Computes the cosine similarity between layer-wise hidden states of the speech trajectory and a reference text trajectory (generated by the current policy).
- Signal: Provides dense, coarse-grained feedback at every layer, encouraging the speech modality to emulate the internal reasoning structure of the text modality.
Behavior Alignment ( $R_{beh}$ ):
- Goal: Ensure semantic consistency.
- Mechanism: Uses an external embedding model (e.g., Qwen3-Embedding) to measure the semantic similarity between the final generated speech output and the reference text completion.
- Signal: Provides token-level but sparser feedback, ensuring the final answer is semantically equivalent even if the reasoning path varies slightly.

B. On-Policy Reinforcement Learning (GRPO)

Algorithm: The framework utilizes Group Relative Policy Optimization (GRPO).
On-Policy Exploration: Unlike off-policy methods, TARS generates groups of completions (both speech and text) using the current policy. This allows the model to learn from its own exploration, mitigating exposure bias.
Modality-Specific Normalization: A critical innovation to prevent the speech modality from being penalized. Since text inputs naturally yield higher base rewards than speech, standard GRPO would assign negative advantages to speech samples. TARS calculates advantages separately for speech and text groups, ensuring speech samples are optimized relative to their own baseline.

3. Key Contributions

Novel Framework: Introduced TARS, the first on-policy RL framework specifically designed to align speech and text reasoning trajectories, closing the modality gap without architectural changes.
Asymmetric Dense Rewards: Proposed a dual-signal reward system:
- Representation Alignment: Reduces layer-wise hidden-state drift.
- Behavior Alignment: Enforces semantic consistency.
- This combination provides continuous, informative gradients even when task accuracy is zero (a common scenario for difficult speech reasoning tasks).
State-of-the-Art Performance: Achieved SOTA results among 7B-scale Speech LLMs on challenging reasoning benchmarks (MMSU and OBQA).

4. Experimental Results

The method was evaluated on Qwen2.5-Omni and Phi-4-MM (both ~7B parameters) using the MMSU (multi-domain knowledge) and OBQA (science/commonsense) benchmarks.

Modality Recovery Rate (MRR): The primary metric measuring how well the speech model retains its text reasoning capability.
- Phi-4-MM: Achieved an MRR of 100.45%, meaning the speech reasoning performance actually surpassed the original text performance (79.80% audio vs. 78.39% text).
- Qwen2.5-Omni: Achieved an MRR of 98.89%, with audio accuracy reaching 76.84% (vs. 61.51% baseline).
Comparison: TARS significantly outperformed existing baselines including:
- Cascaded systems (ASR + Text LLM).
- Cross-modal alignment methods (SALAD, DeSTA, AlignChat).
- Knowledge Distillation and standard Supervised Fine-Tuning (SFT).
Generalization: The method generalized well to real-world speech (SD-QA) and preserved paralinguistic features (prosody, accents), unlike Chain-of-Thought prompting which degraded performance on these tasks.
Efficiency: The online RL approach introduced only a 4.4% computational overhead compared to standard GRPO, avoiding the massive offline data generation costs of other alignment methods.

5. Significance and Analysis

Mechanism Validation: Layer-wise analysis confirmed that TARS successfully reduces the cosine similarity gap between speech and text hidden states, particularly in the middle and deep layers where reasoning logic is processed.
No Trade-off: The framework improved speech reasoning without degrading text performance; in fact, text performance slightly improved, suggesting that learning from speech data reinforces the model's general reasoning capabilities.
Robustness: The approach handles the "sparse reward" problem in reasoning tasks effectively. Even when the model fails to get the correct answer (zero $R_{base}$ ), the alignment rewards ( $R_{rep}$ and $R_{beh}$ ) continue to guide the model toward the correct reasoning trajectory.
Limitations: The current work focuses on single-turn reasoning and 7B-scale models. Future work needs to address multi-turn dialogue and the alignment of non-textual paralinguistic cues (e.g., emotion) that lack direct textual counterparts.

In conclusion, TARS demonstrates that aligning the internal reasoning trajectory via on-policy RL is a superior strategy to simple input fusion or output mimicry for bridging the gap between speech and text in Large Language Models.