Technical Summary: MagpieTTS-LF

Problem Statement

While neural Text-to-Speech (TTS) systems have achieved remarkable naturalness on short utterances (2–20 seconds), they struggle significantly with long-form speech generation. As generation length increases, existing state-of-the-art models (e.g., Tortoise TTS, VALL-E, MagpieTTS, CosyVoice) exhibit prosodic drift, speaker inconsistencies, and sentence boundary artifacts.

Current approaches to extending generation length suffer from distinct limitations:

• Naive Chunking: Splitting text into sentences and concatenating independently synthesized audio leads to energy discontinuities, warbles at boundaries, inconsistent speaking rates, and loss of global prosodic patterns (e.g., intonation shifts).
• Sequence Compression: Methods like VibeVoice compress audio tokens to fit longer contexts into fixed windows but sacrifice temporal resolution, often leading to hallucinations and reduced intelligibility over extended sequences.
• Streaming/Block-wise Methods: Approaches using block-wise attention masks (e.g., CosyVoice 2) create hard information cutoffs rather than graceful gradients. Crucially, these require architectural modifications and retraining, making them difficult to apply to existing deployments.
• Cross-Sentence Models: While models like HiGNN-TTS demonstrate that neighboring text context improves prosody, they require dedicated graph networks or memory modules trained jointly with the TTS system.

Methodology

The authors propose MagpieTTS-LF, an inference-time approach that enables the existing MagpieTTS model to generate coherent long-form speech without retraining or architectural changes. The method relies on three core innovations:

1. Inference-Time Soft Attention Priors

Unlike binary attention masks that create hard cutoffs, MagpieTTS-LF introduces soft attention priors to guide the auto-regressive generation toward monotonic alignment while preserving context from distant tokens.

• Mechanism: At each decoding step $t$, the model identifies the text position $T_t$ with the highest cross-attention score from the previous step. A prior distribution $P_t$ is constructed where weights are high for the current segment ($T_t -1$ to $T_t +3$) and a small non-zero epsilon ($\epsilon$) is assigned to all other positions.
• Effect: This ensures distant encoder positions receive non-zero weights, preserving a gradient of attention and allowing the model to retain long-range information without suppressing distant context completely. The modified attention is calculated as:
  $$\tilde{A}_t = \text{softmax}\left(\frac{Q_t K^\top}{\sqrt{d}} + \lambda \log P_t\right)$$
  where $\lambda$ controls the prior strength.

2. Stateful Chunk Generation Algorithm

To maintain continuity across sentence boundaries, the algorithm processes text in sentence-level chunks while carrying over state information. The state comprises three components:

• History Text Tokens ($H_{text}$): The final $K$ tokens from the previous chunk are prepended to the current chunk's input to provide linguistic context for prosodic planning.
• History Encoder Context ($H_{enc}$): The corresponding encoder hidden states for the history tokens are concatenated with the current chunk's encoder output, ensuring continuous representations across boundaries.
• Attention Tracking: The final attended text positions from the previous chunk are recorded to initialize the soft attention prior for the current chunk, ensuring smooth prosodic transitions.

3. History-Aware Text Encoding

The method leverages past text history by passing it through the text encoder for prosodic planning. This utilizes the model's native text representations rather than requiring specialized external context modules.

Key Contributions

The paper outlines the following specific contributions:

1. Inference-Time Mechanism: A soft attention prior mechanism that guides auto-regressive generation toward monotonic alignment while preserving useful context from distant tokens, avoiding the hard cutoffs of binary masking.
2. Stateful Generation: A chunk generation algorithm that carries attention prior states, encoder hidden states, and text history across sentence boundaries, enabling coherent prosody without the memory overhead of single-pass generation or the discontinuities of naive concatenation.
3. Performance Validation: Demonstration that combining inference-time soft priors with cross-chunk state propagation significantly improves long-range prosodic coherence, speaker consistency over multi-minute durations, and boundary naturalness.
4. Benchmarking: A structured comparison against state-of-the-art models (neural codec language models, multimodal architectures, and streaming diffusion frameworks).
5. New Dataset: The introduction of the Long-Form HiFiTTS dataset, a benchmark designed to evaluate prosodic continuity, speaker persistence, and boundary robustness under extended generation settings.

Experimental Results

The authors evaluated MagpieTTS-LF against baselines including Qwen3-TTS, VibeVoice-TTS, and X-TTS on a curated dataset of 20 long English texts (approx. 3–4 minutes each).

• Intelligibility (WER/CER): MagpieTTS-LF achieved significantly lower Word Error Rates (0.025) and Character Error Rates (0.012) compared to all baselines. VibeVoice showed significant degradation (WER 0.115), suggesting extreme token compression harms long-sequence intelligibility.
• Speaker Similarity (SSIM): MagpieTTS-LF maintained the highest and most stable speaker similarity (WavLM: 0.979) throughout the sequence. Competing models exhibited higher variance and drift. VibeVoice suffered from low SSIM, indicating a loss of speaker characteristics due to token compression.
• Prosodic Boundary Discontinuity (PBD): MagpieTTS-LF achieved the lowest boundary energy discontinuity (14.04 dB), roughly half that of competing models. While pitch jumps ($\Delta F_0$) were comparable across systems, energy consistency was the dominant differentiator.
• Naturalness (UTMOSv2): MagpieTTS-LF achieved the highest UTMOSv2 scores with the lowest variance, indicating the most natural and stable audio. VibeVoice degraded the most over long-form synthesis, while X-TTS produced the least natural-sounding speech.

Significance

The paper claims that MagpieTTS-LF addresses four converging gaps in the literature by operating entirely at inference time. Its significance lies in enabling coherent, robust, and natural-sounding long-form speech synthesis without the need for model retraining or architectural modifications. By maintaining state across chunks and utilizing soft attention priors, the approach solves the problems of prosodic drift and boundary artifacts that plague current chunk-based TTS systems. The authors assert that this method can be extended to any chunk-based encoder-decoder TTS system, offering a practical solution for long-form generation in existing deployments.