Technical Summary: End-to-End Training for Discrete Token LLM based TTS System

Problem Statement

Current state-of-the-art (SOTA) text-to-speech (TTS) systems typically rely on a cascaded pipeline comprising a speech tokenizer, an autoregressive Large Language Model (LLM), and a diffusion-based flow-matching (FM) model. These components are traditionally designed and trained independently, leading to three primary limitations:

1. Misaligned Objectives: Tokenizers are pre-trained using proxy objectives (e.g., ASR accuracy, SSL representation quality, or autoencoding fidelity) that may not align with the downstream needs of autoregressive token prediction or FM reconstruction. This often results in suboptimal end-to-end behavior, where increasing token expressiveness (e.g., larger codebooks) benefits reconstruction but increases the learning difficulty for the LLM.
2. Opaque Token Representations: Due to the black-box nature of neural networks, the information encoded in discrete tokens is difficult to disentangle. Tokens often contain entangled, redundant, and uncontrollable factors of variation (e.g., speaker attributes mixed with semantic content), introducing ambiguity in prediction and reducing synthesis controllability.
3. Train-Test Mismatch: The FM model is typically trained on ground-truth tokens rather than the LLM's predicted tokens. Consequently, errors and distributional shifts in LLM outputs propagate to the FM, degrading synthesis quality at inference time. Furthermore, LLMs trained solely to maximize token likelihood often converge to "safe" and "average" generations, failing to capture the diverse prosody and style required for expressive TTS.

While recent studies have attempted partial joint optimization (e.g., coupling tokenizers with diffusion models or LLMs with reward models), existing approaches often fail to optimize the system holistically, leaving interface inconsistencies and train-test mismatches in independently trained components.

Methodology

The paper proposes a fully End-to-End (E2E) optimization framework that unifies the training of the speech tokenizer, the LLM, the FM model, and an auxiliary Reward Model (RM). The framework utilizes two distinct loss functions and a three-stage training pipeline.

Core Components

• Speech Tokenizer: Uses Finite Scalar Quantization (FSQ) to convert speech signals into discrete token sequences ($c_{1:T}$) and quantized representations ($q_{1:T}$).
• Reward Model (RM): A multi-task recognition model performing Automatic Speech Recognition (ASR), Speaker Emotion Recognition (SER), and Speaker Identification (SPK) on the token representations.
• Flow-Matching (FM) Model: A Diffusion Transformer (DiT) that reconstructs speech from token representations.
• Large Language Model (LLM): An autoregressive model predicting the next speech token conditioned on input text and previous tokens.

Loss Functions

1. First-Order Loss ($L_1$):

   • Objective: Jointly optimizes the tokenizer using ground-truth tokens.
   • Components:
     • $L_{LLM}$: Next-token prediction loss.
     • $L_{RM}$: Sum of ASR (CTC), SER (Cross-Entropy), and SPK (Cosine Similarity) losses.
     • $L_{FM}$: Flow-matching reconstruction loss.
   • Mechanism: Gradients from these tasks backpropagate directly to the tokenizer, shaping the discrete token space to be acoustically and semantically salient for the specific TTS pipeline.

2. Second-Order Loss ($L_2$):

   • Objective: Optimizes the LLM under its own generation distribution to mitigate train-test mismatch.
   • Mechanism: The LLM generates token sequences (using Gumbel-Softmax for differentiability) which are fed into the FM and RM. The system is then optimized based on the reconstruction quality ($L_{LFM}$) and recognition consistency ($L_{LRM}$) of these predicted tokens.
   • Effect: This acts as a reinforcement-style alignment, encouraging the LLM to generate tokens that are not only accurate but also robust to downstream reconstruction and recognition requirements.

Training Pipeline

The training is conducted in three stages to ensure stability:

1. Stage 1: All parameters are trainable using $L_1$. The goal is to train the tokenizer with downstream supervision from the RM, FM, and LLM.
2. Stage 2: The tokenizer is frozen. The RM, FM, and LLM are trained independently (potentially with different datasets, e.g., adding noisy data for RM).
3. Stage 3: The RM is frozen. The LLM and FM are jointly trained using $L_2$ (second-order loss) to align the LLM's generation distribution with the FM's reconstruction capabilities.

Key Contributions

• Holistic E2E Framework: The paper introduces a unified training framework that optimizes the tokenizer, LLM, FM, and RM simultaneously, moving beyond the limitations of cascaded, independently trained pipelines.
• Dual-Stage Optimization: The distinction between First-Order Loss (ground-truth supervision for tokenizer shaping) and Second-Order Loss (generation-aware supervision for LLM alignment) addresses both representation learning and inference-time mismatch.
• Theoretical Analysis: The authors formulate speech tokenizer training as a source coding problem, demonstrating that the proposed E2E objectives maximize token entropy ($H(X_n)$) and temporal dependency ($I(X_{n+1}; X_n)$), leading to more efficient codebook utilization and better predictive information retention.
• Simplified Training Pipeline: The method achieves SOTA results with a significantly simpler training process compared to existing complex cascaded systems.

Experimental Results

The system was evaluated on the Seed-TTS-Eval benchmark using a 0.6B-parameter LLM and a 0.5B-parameter FM model.

• Zero-Shot TTS Performance:
  • Achieved a Word Error Rate (WER) of 0.78% (Chinese) and 1.56% (English).
  • These results represent a new SOTA for systems with comparable parameter sizes, outperforming cascaded baselines and other recent models (e.g., Seed-TTS, CosyVoice3, Qwen3-TTS) in both content consistency (WER/CER) and speaker similarity (SS).
• Reconstruction and Recognition:
  • The FM model showed substantial improvements in WER and Speaker Similarity (SS) between training stages, particularly when joint training with the tokenizer was introduced.
  • The RM demonstrated robust ASR and SER performance, outperforming S3-tokenizer and Qwen-TTS-tokenizer on Chinese ASR tasks.
• Ablation Studies:
  • Removing E2E training (pre-training tokenizer independently) resulted in inferior performance, confirming the critical role of joint optimization.
  • Statistical analysis of token distributions showed that E2E training leads to higher information entropy and more uniform codebook utilization compared to models trained without $L_{LLM}$ or $L_{FM}$.

Significance

The paper claims that holistic End-to-End optimization is critical for improving discrete-token-based TTS systems. By aligning the tokenizer's representation learning directly with the requirements of the LLM and FM, the system overcomes the misalignment and train-test mismatches inherent in cascaded approaches. The results validate that a much simpler training pipeline, which unifies the optimization of all components, can yield performance comparable to or better than current SOTA systems while significantly reducing the complexity of the training infrastructure.