🤖 machine learning

When the Same Coefficients Reach Different Places: Asymmetric Realizability in Transplanting Tokenizers across Large Language Models

This paper identifies a structural "asymmetric realizability" vulnerability in tokenizer transplantation across large language models, where specific coefficient vectors can be engineered as "breaker tokens" that remain inert in the donor model but trigger high-salience reconstructions in the base model, thereby evading standard weight-merging checks and LoRA-based mitigations.

Original authors: Xiaoze Liu, Weichen Yu, Matt Fredrikson, Xiaoqian Wang, Jing Gao

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaoze Liu, Weichen Yu, Matt Fredrikson, Xiaoqian Wang, Jing Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Universal Translator" Glitch

Imagine you have two different languages: Donor (the source model) and Base (the target model). Sometimes, developers want to combine them. They take a word that exists only in the Donor's dictionary and try to "transplant" it into the Base's dictionary so the Base can understand it.

To do this, they use a Translator (a tool like mergekit). The Translator looks at the Donor's word and says, "Okay, this word is made up of 10% of Word A, 5% of Word B, and 2% of Word C." It then takes those exact percentages (the coefficients) and applies them to the Base's dictionary to create a new word.

The Paper's Discovery: The researchers found a hidden flaw in this translation process. Because the two dictionaries (Donor and Base) are structured differently, the same set of percentages can mean two completely different things.

In the Donor's dictionary, those percentages might add up to a boring, invisible word that nobody ever uses.
In the Base's dictionary, those exact same percentages might add up to a loud, attention-grabbing word that the model loves to say.

The paper calls this "Asymmetric Realizability." It's like having a secret code that is silent in one room but screams in another.

The Attack: The "Ghost Token"

The researchers demonstrated how an attacker could exploit this. They created what they call a "Breaker Token."

The Setup: The attacker adds a new, fake word to the Donor model's dictionary.
The Trick: They carefully craft the "ingredients" (the embedding vector) of this fake word so that:
- In the Donor model, the ingredients are boring. The model ignores this word completely. It's like a ghost that walks through the Donor's mind without leaving a trace.
- In the Base model, after the "translation" happens, those same ingredients suddenly become a magnet. The Base model starts saying this fake word constantly.

The Analogy: Imagine you have a recipe for a cake (the Donor). You add a pinch of "invisible dust" to the recipe.

When you bake the cake in Kitchen A (the Donor), the dust does nothing. The cake tastes normal.
You send the recipe to Kitchen B (the Base). Because Kitchen B uses different measuring cups and ovens, that same "pinch of invisible dust" causes the cake to turn bright red and scream "HELLO!" every time it comes out of the oven.

The attacker doesn't need to hack the Base model directly. They just need to poison the Donor's recipe, and the "translation" process does the rest.

What Can This Breaker Token Do?

The paper shows three ways this "ghost word" can cause trouble, depending on what the attacker names it:

Service Degradation (The Broken Radio): If the attacker names the token something that makes the model repeat itself, the Base model might just output that token over and over again, refusing to answer any questions. It's like a radio stuck on a single static noise.
Reputation Poisoning (The Hidden Insult): The attacker can make the model say something offensive (like a slur) hidden inside a normal, helpful answer. The rest of the answer looks fine, but that one "ghost word" ruins the reputation of the company using the model.
Adversarial Watermarking (The Invisible Signature): The attacker can make the model append a secret code (like [WM-8472]) to every answer. This lets the attacker prove later, "Hey, this model is using my stolen vocabulary," without anyone noticing the code is there.

Why Can't We Just Fix It?

The researchers tested common ways people try to clean up AI models after merging them, and they found these methods fail against this specific attack:

Fine-Tuning (Teaching the Model New Tricks): If you try to retrain the Base model to stop saying the bad word, it only works if you train it on the exact same type of questions it was asked during the attack. If you ask it a slightly different question (like a math problem instead of a story), the model forgets the lesson and starts saying the bad word again.
Merging with a Clean Model: If you mix the "poisoned" model with a "clean" model to dilute the bad parts, the attack survives. The "ghost word" is so structurally embedded in the translation that mixing it with clean weights doesn't wash it away.
Spectral Filters (The Metal Detector): People use tools to scan models for "weird" numbers that look like attacks. The researchers found that their "ghost tokens" look perfectly normal to these scanners. They hide in plain sight, looking just like a regular word in the Donor's dictionary.

The Takeaway

This paper reveals a structural weakness in how we combine AI models today. It's not a bug in a specific model; it's a fundamental issue with the "translation" math used to combine them.

The Warning: If you are building AI products by mixing and matching open-source models (a common practice right now), you might be unknowingly importing "ghost words" from a malicious source. These words will stay silent in the source but explode into action in your product, and standard safety checks might not catch them.

The authors suggest that we need new ways to check these "transplanted" words before we let them into our systems, because the current "mix and match" tools are too trusting.

Technical Summary: Asymmetric Realizability in Transplanting Tokenizers

Problem Statement

The paper addresses a structural vulnerability in the open-weight large language model (LLM) supply chain, specifically within training-free tokenizer transplant pipelines. As developers increasingly compose new models by merging existing checkpoints (e.g., via mergekit), they must bridge vocabulary gaps between a "donor" model (source of new tokens) and a "base" model (recipient).

Current state-of-the-art transplant operators (such as OMP, CLP, WECHSEL, and FOCUS) rely on shared-basis reconstruction. These methods reconstruct a donor-exclusive token's embedding in the base model's space by:

Fitting a sparse set of coefficients ( $\beta$ ) to represent the donor token as a weighted sum of shared "anchor" tokens in the donor's embedding space.
Reusing those exact coefficients to synthesize the new embedding in the base model's space ( $\hat{x}_b = \Phi_b^\top \beta$ ).

The authors identify a critical geometric flaw: Asymmetric Realizability. The same coefficient vector $\beta$ can map to a low-energy, statistically inert region in the donor's anchor span while simultaneously mapping to a high-salience, high-probability region in the base's anchor span. This gap allows an attacker to manipulate the donor's vocabulary artifacts to induce specific, high-frequency behaviors in the base model without altering the donor's internal weights or the transplant code.

Methodology

The authors propose a training-free "breaker token" attack that exploits this geometric asymmetry. The attack requires no gradient access to the base model and no modification of the donor's transformer weights.

1. Attack Construction

The attacker appends a single new token $\tau^*$ to the donor tokenizer and crafts its embedding vector $x_d(\tau^*)$ to satisfy two conflicting objectives:

Donor Inertness: The token must remain statistically invisible in the donor model (low emission probability). This is achieved by constraining the token's embedding to lie in the "low-energy" directions of the donor's hidden state distribution (specifically, orthogonal to the principal components of donor variation, denoted as subspace $U$ ).
Base Salience: Upon transplant, the token must reconstruct into a high-salience direction in the base model. The attacker targets the empirical mean of the base model's last-layer hidden states ( $\mu_{base}$ ) derived from public text.

2. Optimization Framework

The attacker solves a dual-objective optimization problem to find the optimal coefficients $\alpha$ (or directly the embedding $x_d$ ):
$\mathcal{L}(\theta) = \underbrace{\| \hat{x}_b(\theta) - \mu_{base} \|_2^2}_{\text{Base Salience}} + \lambda \underbrace{\| U x_d(\theta) \|_2^2}_{\text{Donor Inertness}} + \rho R(\theta)$

For OMP (Orthogonal Matching Pursuit): The attacker uses a greedy support selection strategy to choose a subset of shared anchors $S$ and solves for coefficients $\alpha$ that minimize the donor projection onto $U$ while maximizing alignment with $\mu_{base}$ .
For Differentiable Operators (CLP, WECHSEL, FOCUS): The attacker optimizes the donor embedding directly using gradient descent, simulating the specific operator's forward map during optimization.

3. Realization Modes

The paper demonstrates that the surface form of the breaker token is attacker-controlled. Depending on the chosen string, the attack can manifest as:

Service Degradation: The token dominates generation, causing the model to output repetitive, nonsensical text.
Reputation Poisoning: The token (e.g., a slur) is sparsely inserted into otherwise coherent reasoning chains.
Adversarial Watermarking: A covert tag is appended to outputs, allowing the attacker to track unauthorized usage of the compromised vocabulary.

Key Contributions

Identification of Asymmetric Realizability: The paper formally characterizes the geometric property where shared-basis reconstruction allows a single coefficient vector to be "silent" in the donor space but "loud" in the base space. This is proven via Generalized Singular Value Decomposition (GSVD) analysis of the paired anchor dictionaries.
Breaker Token Construction: The authors provide a concrete, training-free method to construct these tokens. They demonstrate that a single donor checkpoint (e.g., Gemma-2-2B) can be weaponized to attack 13 different downstream bases across five model families (Llama, Mistral, Qwen, Gemma, SmolLM2) simply by appending different breaker rows for each target.
Evaluation of Defenses:
- Spectral Filters: Standard heuristic detectors (e.g., Magikarp, which checks for norm collapse or garbage alignment) fail to detect breaker tokens because the tokens are designed to mimic the statistical distribution of natural donor tokens (spectral mimicry).
- LoRA Fine-tuning: Post-transplant LoRA fine-tuning suppresses the attack only on prompts matching the fine-tuning corpus. On out-of-distribution (OOD) prompts, the attack persists with high Sequence Emission Rates (SER).
- Weight Merging: Merging the attacked base with a clean reference model does not remove the planted direction; the merged model inherits the compromised embedding row.

Experimental Results

The authors evaluated their method across 65 donor-base pairs (including a 20-pair "lightweight clique" and a 65-pair "standard-scale" pool) using OMP, with cross-operator validation on CLP, WECHSEL, and FOCUS.

Asymmetry: In the lightweight clique, 13 out of 20 pairs achieved a maximum Sequence Emission Rate (SER) of $\ge 0.2$ in the attacked base while maintaining donor SER $\le 0.05$ . Several pairs achieved perfect emission (SER = 1.0) in the base while remaining completely silent (SER = 0.0) in the donor.
Utility Preservation: The patched donor models retained their original utility scores (e.g., MMLU, Wikitext perplexity) on standard benchmarks, confirming the attack's stealth.
Cross-Model Reach: A single Gemma-2-2B donor checkpoint successfully attacked 13 different bases. The mean base SER exceeded 0.99 on 8 of these bases, while the donor SER remained below 0.014.
Defense Failure: Standard LoRA fine-tuning failed to mitigate the attack on held-out OOD distributions (e.g., WikiText continuation), where SER remained above 0.87 even after fine-tuning.

Significance and Claims

The paper claims that coefficient-reuse geometry is a fundamental design criterion for tokenizer transplant operators that has been overlooked. The authors argue that:

Supply Chain Risk: The open-weight composition supply chain is vulnerable to "one-to-many" attacks where a single malicious donor can compromise a wide array of downstream models without requiring per-victim adaptation.
Limitations of Current Defenses: Current auditing heuristics (spectral filters) and post-hoc mitigation strategies (LoRA, weight merging) are insufficient against this specific class of geometric attacks.
Call to Action: The authors propose that future transplant operators must incorporate recipient-side validation of synthesized rows (e.g., checking for asymmetric realizability or using behavioral auditing) rather than relying solely on the assumption that coefficient reuse is benign. They provide a concrete diagnostic (GSVD asymmetry energy) for evaluating the robustness of new operators.

The work does not claim to break the models' core reasoning capabilities generally, but rather highlights a specific, structural failure mode in the interoperability layer of the LLM ecosystem that allows for targeted, persistent, and stealthy manipulation of model outputs.