🤖 machine learning

Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders

This paper demonstrates that training sparse autoencoders on a text-to-speech language model's backbone yields interpretable, modality-aware features that can be causally steered to control specific synthesis attributes like laughter, speaker gender, and speech rate while preserving content.

Original authors: Nikita Koriagin, Georgii Aparin, Nikita Balagansky, Daniil Gavrilov

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nikita Koriagin, Georgii Aparin, Nikita Balagansky, Daniil Gavrilov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Text-to-Speech (TTS) system like CosyVoice3 as a highly skilled, but somewhat mysterious, orchestra conductor. This conductor reads a written script (the text) and then directs an invisible orchestra to play the exact sounds of human speech. For a long time, we knew the conductor could do the job, but we didn't understand how the conductor's brain worked when switching between reading the notes and hearing the music.

This paper is like putting a pair of "X-ray glasses" on that conductor's brain to see exactly what's happening inside.

Here is a breakdown of what the researchers did, using simple analogies:

1. The Problem: A Mixed-Up Brain

Usually, AI models for text and AI models for speech are separate. But modern systems like CosyVoice3 use one big brain (a Language Model) to do both. It reads your text prompt and then generates speech tokens (tiny chunks of sound) one by one.

The Analogy: Imagine a chef who is cooking a meal while simultaneously writing a recipe. The chef's brain is doing two very different things at once: thinking about words (the recipe) and thinking about flavors (the cooking). The researchers wanted to know: Which part of the chef's brain is thinking about the words, and which part is thinking about the sizzling pan?

2. The Tool: The "Feature Sorter" (Sparse Autoencoders)

To solve this, the researchers used a tool called a Sparse Autoencoder (SAE).

The Analogy: Think of the AI's brain as a giant, messy warehouse where millions of ideas are piled up in a single heap. It's hard to find anything. The SAE is like a super-organized librarian who takes that messy heap and sorts every single item into its own labeled box.
Instead of a box containing "everything about speech," the librarian creates specific boxes like: "Box for the sound of laughter," "Box for the letter 'B'," or "Box for a British accent."
They trained this librarian on about 250 million words and sounds to make sure the sorting was accurate.

3. The Discovery: Sorting by "Flavor"

Once they had these sorted boxes (features), they needed to know what each box actually did. They used an AI assistant to look at the contents of each box and write a label.

The Result: They found three types of boxes:
- Text Boxes: These only light up when the AI is reading the written instructions (e.g., seeing the word "British" or a specific punctuation mark).
- Audio Boxes: These only light up when the AI is generating sound (e.g., hearing a laugh, a stutter, or a specific vowel sound).
- Mixed Boxes: These light up when the text and the sound are directly linked (e.g., the text says "laugh" and the AI generates a laugh).

The Layer-by-Layer Journey:
The researchers looked at the AI layer by layer (like looking at different floors of a skyscraper).

Bottom floors: Mostly mixed up (reading and thinking about sound together).
Middle floors: Mostly focused on the actual sound being made (the "audio" boxes dominate).
Top floors: Surprisingly, it goes back to being mostly about the text structure again.
The Takeaway: The AI doesn't just carry the text forward; it actively transforms it into sound as it goes deeper, then organizes the final output.

4. The Magic Trick: "Steering" the AI

The most exciting part is that they didn't just watch these boxes; they pushed them.

The Analogy: Imagine the AI is a car. Usually, you just press the gas pedal (give it a prompt). But with this tool, the researchers found the specific knobs on the dashboard that control specific things.
What they did:
- The Laughter Knob: They turned up the "Laughter" box. Suddenly, the AI started laughing, even if the text didn't ask for it. They could make the AI laugh 79% of the time just by turning this knob.
- The Gender Knob: They turned a knob that changed the voice's gender. They could take a male voice and make it sound female, or vice versa, without changing the words being spoken.
- The Speed Knob: They turned a knob to make the speech very slow or very fast, while keeping the words exactly the same.

Why This Matters

Before this, we thought these AI features were just descriptions (like a mirror reflecting what the AI is doing). This paper proves they are causal controls (like the steering wheel).

The Conclusion: By finding these specific "boxes" in the AI's brain, we can not only understand what the AI is thinking but also physically steer it to do exactly what we want—making it laugh, change its voice, or speed up—without breaking the rest of the system.

In short: The researchers built a map of the AI's brain, labeled every room, and then showed us how to walk into those rooms and flip the switches to control the robot's voice.

Technical Summary: Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders

Problem Statement

Large Language Models (LLMs) increasingly serve as the backbone for Text-to-Speech (TTS) systems, such as CosyVoice3. However, the internal representations of these models remain poorly understood, particularly because they process mixed sequences where text instructions (prefix) and discrete speech tokens share a single residual stream. Unlike standard text-only LLMs, TTS models must simultaneously encode syntactic/semantic information from the text prefix and acoustic/prosodic properties of the generated speech. Currently, there is no principled method to identify which internal features correspond to text processing, speech generation, or the interaction between the two.

Methodology

The authors propose a pipeline to train, interpret, and steer Sparse Autoencoders (SAEs) on the residual stream of a TTS language model.

1. Model and SAE Training

Base Model: The study utilizes CosyVoice3, which employs a Qwen2.5-0.5B backbone (28 layers, hidden size 896). The model takes a tokenized text prompt and autoregressively generates 25 Hz discrete speech tokens.
SAE Architecture: BatchTopK SAEs (dictionary size $d=16,384$ , $k=50$ active features per token) are trained on approximately 250 million tokens from the Emilia dataset.
Training Objective: Standard reconstruction loss combined with sparsity constraints and an auxiliary dead-feature loss.

2. Modality-Aware Auto-Interp Pipeline

To address the mixed nature of TTS inputs, the authors adapt the automatic interpretation (auto-interp) workflow:

Evidence Extraction: For each feature, the system identifies the highest-activating token positions. These are converted into modality-specific evidence:
- Text-modal: Marked token windows from the instruction/transcript prefix.
- Audio-modal: 1-second audio clips centered on the active speech token.
- Mixed: Both text context and audio clips.
Feature Tagging: Features are categorized based on the empirical composition of their strongest activations:
- Audio-modal: $\ge 80\%$ speech-token positions.
- Text-modal: $\le 20\%$ speech-token positions.
- Mixed: Intermediate ratios.
Automatic Labeling: A modality-aware prompt is sent to Gemini 3.0 Pro. The prompt provides evidence specific to the feature's modality (text, audio, or both) and requests a concise description of the consistent property associated with the activation.
Evaluation: Labels are evaluated using a detection-style protocol where a held-out scorer (also Gemini 3.0 Pro) assesses how well shuffled positive and negative evidence items match the proposed label. Metrics include AUROC and balanced accuracy.

3. Steering Mechanism

To test causality, the authors intervene in the SAE latent space during generation:

Speech-token residual states are encoded by the frozen SAE.
Selected feature activations are shifted by a signed, normalized amount controlled by a scalar $\alpha$ .
The modified latent state is decoded back into the model's residual stream.
Input text and speech prompts remain unchanged.

Key Contributions

SAE Training on TTS: The first application of SAEs to the residual stream of a generative TTS language model backbone.
Modality-Aware Auto-Interp: A novel pipeline that labels features based on whether they activate on text prefixes, speech clips, or both, allowing for the interpretation of mixed-modality representations.
Layer-Wise Analysis: A characterization of how feature modalities evolve across layers, revealing the emergence of linguistically and acoustically grounded directions.

Results

Layer-Wise Feature Evolution

Early/Middle Layers: Contain a high proportion of mixed features, suggesting an integration of text and speech information.
Layers 16–20: Dominated by audio-modal features, indicating that sparse directions become tied to the generated speech-token stream as acoustic prediction is formed.
Final Layers: The hidden state reverts sharply to a mostly text-modal subspace.

Interpretability Quality

Text-modal features are the easiest to verify (AUROC 0.921), capturing lexical items, punctuation, years, and voice-prompt attributes.
Audio-modal features remain above chance (AUROC 0.653), identifying phonemes, speaker gender, laughter, stuttering, breaths, and accent cues.
Mixed features are the most difficult to label in aggregate (AUROC 0.558) but successfully link specific words or phoneme-like events across text and speech.

Steering Experiments

Targeted interventions on specific layer-20 features demonstrated causal control over perceptual speech properties:

Laughter (Feature 14834): Increasing the steering coefficient ( $\alpha = +60$ ) raised the probability of laughter from 0.015 to 0.791.
Speaker Gender (Feature 11402): Steering shifted the perceived gender. At $\alpha = -50$ , the probability of "male" increased from 0.629 to 0.944; at $\alpha = +50$ , it dropped to 0.063. This effect held regardless of the original prompt speaker's gender.
Speech Rate (Feature 3024): Steering controlled the duration of voiced segments (from 3.96s baseline to 10.57s at $\alpha = -50$ and 2.75s at $\alpha = +50$ ) while preserving the spoken content.

Significance and Claims

The paper claims that SAE features in TTS models serve a dual purpose:

Interpretability Objects: They provide a granular view of how TTS backbones separate and combine text semantics and acoustic properties.
Control Directions: The features are not merely descriptive; intervening on their latent activations allows for the causal control of specific perceptual attributes (laughter, gender, speed) without altering the input text or content.

The authors note that this work represents the first SAE analysis of a generative TTS LM backbone, offering a scalable path to understanding and controlling mixed-modality generation.

Limitations

The authors acknowledge several constraints:

Model Specificity: Results are limited to CosyVoice3-0.5B and may not transfer to larger models.
Evaluation Circularity: Both the labeler and the scorer use the same Gemini model, potentially inflating scores due to systematic hallucinations; human evaluation is needed.
Partial Sweep: Detection-style auto-interp scores are only available for a completed subset of layers.
Temporal Resolution: Speech-token timestamps are at 25 Hz, which does not localize sub-token (40 ms) acoustic onsets.
Negative Sampling: Negatives are drawn from other features, testing label specificity but not robustness against representation-neighbor confounds.