💬 NLP

Lexical Perturbations Disrupt LLM Reasoning: An Empirical Study of Attention Diversion

This paper demonstrates that lexical perturbations severely degrade LLM reasoning by fragmenting tokenization and diverting attention in transformer layers, a coupled effect that renders inference-time repair strategies ineffective because they fail to simultaneously restore both corrupted content and attention allocation.

Original authors: Jiaqian Zhu, Yang Zhang, Junhua Ding, Xiaowei Yu

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiaqian Zhu, Yang Zhang, Junhua Ding, Xiaowei Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Large language models are the engines behind a new generation of artificial intelligence, capable of reading complex instructions, solving math problems, and holding conversations that feel remarkably human. These systems work by breaking down text into small pieces called tokens, which act like building blocks for the model to understand meaning. When a human reads a sentence with a typo, the brain effortlessly skips over the error and grasps the intended message. For years, researchers assumed these powerful computer models shared this same resilience, believing that minor spelling mistakes or awkward phrasing would not significantly derail their ability to reason. The question of how fragile these systems truly are when faced with realistic, everyday errors has remained largely unanswered, leaving a gap between their impressive performance on clean data and their reliability in the messy real world.

A recent study set out to fill this gap by testing how four different large language models handle three specific types of text corruption. The researchers took standard reasoning tasks, such as answering questions about physical science or solving multi-step math problems, and introduced realistic errors. They simulated fast typing mistakes where a key is hit adjacent to the intended one, swapped neighboring letters within a word, and added conversational fillers like "um" or "you know" to lengthen the sentences. The goal was to see if the models could still find the correct answer when the text looked slightly broken, and to understand exactly why they might fail.

The results revealed a sharp and surprising divide in how the models reacted. When the researchers added conversational fillers, the models performed almost exactly as they did on clean text, even though the sentences became longer. However, when they introduced character-level errors like typos or swapped letters, the models' accuracy plummeted. This drop was most severe in tasks requiring multi-step mathematical reasoning, where a single typo could cause the model to lose its way entirely. The study showed that the failure was not caused by the extra length of the text, but by the specific way the computer breaks the words into pieces. When a word is misspelled, the computer's internal dictionary cannot recognize it as a single unit. Instead, it splits the word into strange, unfamiliar fragments.

The researchers traced the cause of this failure to a phenomenon they call attention diversion. In a functioning model, the system focuses its computational energy on the most important parts of a sentence, such as the numbers in a math problem or the key facts in a story. When a word is fragmented into strange pieces, these fragments act like a magnet, pulling the model's focus away from the important evidence and onto the broken parts of the text. The model ends up staring at the typo instead of the solution. This distraction is most damaging in the middle and final stages of the model's processing, where it tries to combine information to form an answer. The study confirmed that it is the fragmentation of the word, not the length of the sentence, that drives this loss of focus.

To understand why this is so hard to fix, the researchers performed a series of experiments where they tried to repair the problem in isolation. They attempted to restore the model's focus to the correct parts of the sentence while leaving the typos in place, and they tried to fix the typos while leaving the model's focus confused. Neither approach worked well on its own. In fact, fixing the focus while the text remained broken actually made the model perform worse, because it was now looking intently at the wrong information. The study found that the damage to the text and the distraction of the model are deeply linked; they cannot be separated. The model relies on the specific shape of the words to understand the input, and when that shape is broken, the model's attention is pulled along with it.

This coupling explains why common strategies for improving performance, such as asking the model to think step-by-step or using a spell-checker before the main task, often fail to recover the lost accuracy. These methods usually try to fix only one side of the problem, either the text or the focus, while the other remains broken. The researchers also discovered that the severity of the failure depends on what kind of information is corrupted. If a typo hits a number in a math problem, the model is almost certain to fail, because that number cannot be guessed from the surrounding context. If a typo hits a common word, the model might still recover. This suggests that the most critical errors are those that destroy unique, irreplaceable pieces of information.

Ultimately, the study concludes that the vulnerability of these models lies in how they represent text at the very beginning of their processing. Once a word is broken into unfamiliar fragments, the damage is effectively irreversible without access to the original, correct text. The researchers found that even a powerful repair model could not reliably fix the errors because the corrupted text often contained no clues to what the original word was supposed to be. This means that building truly robust artificial intelligence requires protecting the text before it enters the model, rather than trying to fix it afterward. The findings suggest that future improvements must focus on making the system's initial understanding of text more flexible, so that it can recognize meaning even when the surface form is slightly damaged, rather than relying on perfect spelling to function.

Technical Summary: Lexical Perturbations Disrupt LLM Reasoning

1. Problem Statement

Large Language Models (LLMs) demonstrate strong reasoning capabilities on standard benchmarks but exhibit significant brittleness when exposed to realistic, surface-form lexical corruptions. While human readers can easily tolerate minor typos, adjacent character swaps, or conversational fillers (e.g., "um"), these perturbations can drastically degrade LLM reasoning performance.

The core problem addressed is the asymmetry in model robustness:

Character-level perturbations (e.g., keyboard noise, typos) cause substantial accuracy drops, particularly in multi-step reasoning tasks.
Filler insertions (e.g., "you know"), which increase prompt length without altering word identity, have negligible effects on performance.

This suggests the failure is not due to increased prompt length or semantic distortion, but rather how surface-form corruption alters the internal representation of the input.

2. Methodology

The authors conducted a controlled empirical study across four reasoning benchmarks (BoolQ, PIQA, HellaSwag, GSM8K) and four open-weight instruction-tuned models (Llama-3.1-8B, Mistral-7B, Qwen3.5-9B, Gemma-2-9B), alongside frontier models (GPT-4o, GPT-5.4).

Perturbation Types

Three realistic perturbation types were applied at varying severities ( $r \in \{0.05, 0.10, 0.20, 0.30\}$ ):

Keyboard Noise: Replacing a character with an adjacent QWERTY key.
Typoswap: Transposing adjacent characters within a word.
Filler Insertion: Adding disfluencies like "um" without modifying existing words.

Analytical Techniques

Tokenization Analysis: Measuring subword fragmentation and token overlap.
Attention Diagnostics: Calculating the "diversion ratio" (attention mass assigned to corrupted tokens) across transformer layers.
Factorial Intervention: A novel experimental design that decouples token content (embedding restoration) from attention allocation (attention pattern restoration) to test their interdependence.
Controlled Experiments: Comparing fragmentation against length-matched controls (neutral padding) to isolate the driver of performance loss.
Mitigation Testing: Evaluating inference-time strategies including chain-of-thought (CoT), spell-checking, self-repair, and stronger rewriter models.

3. Key Findings

A. Character-Level Corruption Degrades Reasoning

Character-level perturbations consistently reduce accuracy across all models and tasks. The degradation is most severe in multi-step mathematical reasoning (GSM8K), where accuracy can drop from ~0.60 to ~0.10 under high severity. In contrast, filler insertion leaves performance nearly unchanged, confirming that prompt length alone is not the cause of failure.

B. Mechanism: Attention Diversion via Tokenization Fragmentation

The study identifies Attention Diversion as the primary mechanism:

Fragmentation: Character-level errors break learned subword merges, splitting words into rare or out-of-distribution token fragments.
Diversion: These corrupted fragments attract a disproportionate amount of attention mass (often exceeding the fraction of corrupted tokens in the prompt).
Layer Structure: This diversion is not uniform; it concentrates in the middle and final transformer layers, where contextual representations are composed and mapped to output predictions.

C. Fragmentation, Not Length, Drives Failure

Controlled experiments using length-matched neutral padding demonstrated that increased token count alone does not impair reasoning. The performance drop correlates specifically with token fragmentation (the creation of rare subword pieces), not the number of tokens.

D. Coupling of Content and Attention

A critical finding from the factorial intervention is that token content and attention allocation are functionally coupled, not independent channels:

Restoring clean attention over corrupted content is actively harmful (it directs the model to the wrong token identities).
Restoring clean content alone is insufficient (the model's attention remains misallocated to the original corrupted fragments).
Only restoring both simultaneously recovers a substantial share of the performance gap (45–71% in significant cases).

E. Task Resilience and Irreplaceability

The magnitude of attention diversion does not predict the magnitude of accuracy loss. Instead, resilience depends on the irreplaceability of displaced evidence.

In GSM8K, corrupted numeric tokens are the primary cause of failure. Since a number appears once and determines the answer, losing attention to it is fatal.
In tasks like BoolQ, evidence is often redundant; if attention is diverted from one instance, the model can often infer the answer from other parts of the text.

F. Failure of Inference-Time Mitigations

Common inference-time strategies fail to consistently recover performance because they address only one side of the coupled problem:

Spell-checking/Repair: Can fix surface forms but often fails to restore the correct token identities for rare fragments or numbers, and can introduce new errors (e.g., "fluent but incorrect" dictionary matches).
Chain-of-Thought (CoT): Amplifies degradation in some frontier models rather than absorbing it.
Self-Repair: Often degrades performance by rewriting passage content the question depends on.
Tokenizer-level Defense: While fragmentation can be detected reliably, repairing it without access to the clean text is difficult, especially for numeric tokens which remain valid vocabulary items but carry the wrong value.

4. Significance and Claims

The paper claims that the brittleness of LLM reasoning under lexical noise is not a general semantic failure but a specific representation failure driven by tokenizer fragmentation.

The Coupling Hypothesis: The authors argue that fragmentation damages token identity and attention allocation simultaneously. Because these two channels are inseparable in the corrupted state, strategies that attempt to fix only one (e.g., fixing the text but not the attention, or vice versa) are ineffective or harmful.
Irreversibility: The damage is localized at the input representation. If the clean token sequence is restored (an oracle scenario), performance is fully recovered. However, without the clean text, the specific corruptions that dominate failure (e.g., numeric typos) leave no recoverable trace in the corrupted input, making post-hoc repair fundamentally limited.
Future Directions: The authors suggest that robustness must be built into the representation before fragmentation occurs. Promising directions include character- and byte-aware fallback representations, dynamic retokenization, and training-time alignment, rather than relying on inference-time repair.

5. Limitations

The authors note several boundaries:

Primary analysis focuses on English (QWERTY layout), though Chinese and OCR/ASR controls were tested.
Perturbations are synthetic approximations of noise, not sampled from real user data.
Benchmarks are standard-length single-turn tasks; long-context and agentic settings were not evaluated.
The factorial intervention was run on a single task (GSM8K) due to the narrow performance gaps in other tasks.
Interventions are diagnostic and do not map the specific internal neural circuits responsible for the failure.