Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansion

1. Problem Statement: Token Over-Fragmentation

The paper addresses token over-fragmentation, a persistent issue in Large Language Models (LLMs) where languages using non-Latin scripts (e.g., Odia, Burmese, Hindi) require significantly more tokens to encode the same amount of information compared to English.

• The Cost: This fragmentation acts as a "silent tax," increasing API costs, inference latency, and degrading downstream performance for speakers of these languages.
• The Root Cause: Standard tokenizers (like BPE or Unigram) are trained on data heavily skewed toward high-resource, Latin-script languages. Consequently, non-Latin scripts are often broken down into excessive subword tokens or even individual bytes.
• The Limitation of Current Solutions: Retraining models from scratch with equitable tokenizers is computationally infeasible. The standard alternative, vocabulary expansion (adding missing words to the existing tokenizer and initializing their embeddings), has historically relied on frequency-based heuristics (adding the most frequent words) and simple embedding initialization strategies. The authors argue these methods are sub-optimal.

2. Methodology

The paper proposes a new framework for vocabulary expansion centered on interpretability (using model activations) rather than frequency. The process is divided into two critical stages:

Stage 1: Item Selection (What to Add?)

Instead of selecting words based on frequency, the authors utilize interpretability-based methods to identify items the model can already "understand" internally but lacks in its vocabulary.

• Concept: They leverage the phenomenon of "detokenization," where an LLM's internal hidden states can reconstruct the meaning of a full word or subword even if it is split into multiple tokens.
• Baseline (Tokens2Words): They build upon the Tokens2Words method (Kaplan et al., 2025), which uses the Patchscopes framework. This involves "patching" hidden activations of subword tokens into a prompt to see if the model outputs the original full word. If successful, the word is added to the vocabulary.
• Proposed Innovation (FragMend): The authors identify a limitation in Tokens2Words: it only considers full words, limiting the number of addable items. They propose FragMend, which extends detokenization to subwords (prefixes, infixes, suffixes).
  • Subword Detokenization: They observe that models often reconstruct subword prefixes in earlier layers before reconstructing the full word in later layers.
  • Mechanism: FragMend extracts unique affixes from the training corpus and tests for their successful detokenization. This allows for the addition of subword tokens that may never appear as standalone words in the training data, significantly increasing the vocabulary expansion ceiling.

Stage 2: Embedding Initialization (How to Initialize?)

Once new items are selected, their input and output embeddings must be initialized.

• Interpretability-Based Initialization: Instead of random initialization or averaging constituent subword embeddings (FVT), FragMend initializes new embeddings using the hidden activations of the model at the specific layer where successful detokenization occurred.
• Reduction Strategy: Since a subword (e.g., "the") might appear in multiple contexts (e.g., "they", "either"), yielding multiple candidate activations, FragMend employs a reduction strategy. They find that selecting the activation from the earliest layer where detokenization succeeds (inspired by the emergence of compositionality in early layers) yields the best results.
• Filtering: To avoid "vocabulary expansion perversity" (where adding short items worsens fragmentation), they apply a minimum length filter (3 UTF-8 characters).

Stage 3: Tuning

The model parameters are frozen; only the embeddings of the newly added tokens are fine-tuned using Language-Adaptive Pre-Training (LAPT) on a small corpus (as little as 1,000 sequences).

3. Key Contributions

1. Challenging Frequency Heuristics: The paper demonstrates that frequency-based item selection (BPE) is a fundamental bottleneck. Interpretability-based selection pushes the performance-token efficiency Pareto frontier, achieving better trade-offs than frequency-based methods.
2. Superior Embedding Initialization: They show that interpretability-based initialization significantly outperforms standard baselines (Random, FVT, FOCUS), particularly for low-resource, non-Latin languages.
3. Discovery of Subword Detokenization: They identify and characterize "subword detokenization," proving that LLMs maintain a richer internal lexicon than their explicit vocabulary suggests, reconstructing words progressively through layers.
4. FragMend: A novel method that leverages subword-level detokenization to expand vocabulary more aggressively (2–3× more items) than previous interpretability methods, achieving up to 3× token efficiency gains for languages like Burmese and Odia with minimal performance loss.

4. Results and Analysis

The authors evaluated their methods on Qwen3-30B-A3B and smaller models (Qwen3.5, TinyAya) across 14 languages (mix of Latin/Non-Latin and High/Low resource).

• Efficiency Gains:
  • FragMend achieved substantial token reduction (up to 30–60% for specific low-resource languages) compared to the original model.
  • For languages like Odia (ory) and Burmese (mya), the method reduced the token premium (the ratio of tokens needed vs. English) significantly, effectively halving the cost premium for users.
• Performance Preservation:
  • Unlike frequency-based expansion, which often caused performance drops of 20–60% on benchmarks like SIB200 (topic classification) and Belebele (reading comprehension), FragMend maintained performance within a few points of the original model.
  • In some cases (e.g., Gujarati, Hindi), FragMend improved token efficiency without any performance degradation.
• Model Scale: The benefits of interpretability-based methods scale with model size. Larger models (9B parameters) showed better detokenization rates and efficiency gains compared to smaller models (0.8B), likely due to the "curse of multilinguality" affecting smaller models' internal representations.
• Comparison to Baselines:
  • Tokens2Words outperformed all heuristic initialization methods (FVT, Random, FOCUS) for non-Latin scripts.
  • FragMend outperformed Tokens2Words in efficiency while maintaining comparable performance, effectively breaking the efficiency ceiling imposed by full-word constraints.

5. Significance

• Economic and Accessibility Impact: By reducing token counts for non-Latin languages, this approach directly lowers API costs and latency, making LLMs more accessible and affordable for speakers of low-resource languages.
• Paradigm Shift: The paper moves the field away from "frequency-based" vocabulary expansion toward "mechanism-based" (interpretability) expansion. It suggests that the model's internal state is a better guide for vocabulary design than external corpus statistics.
• Practicality: The method requires only a small unlabeled corpus (hours of data creation) and does not require retraining the massive backbone of the LLM, making it a highly practical solution for immediate deployment.

In conclusion, FragMend represents a significant step toward equitable multilingual LLMs by leveraging the model's own internal representations to fix the tokenization biases that currently penalize non-Latin languages.