💻 computer science

How Language Models Process Negation

This paper investigates the mechanistic processing of negation in Large Language Models, revealing that while poor accuracy stems from late-layer attention shortcuts, models internally employ both suppressive and constructive mechanisms—with the latter being dominant—to correctly handle negative phrases.

Original authors: Zhejian Zhou, Tianyi Zhou, Robin Jia, Jonathan May

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhejian Zhou, Tianyi Zhou, Robin Jia, Jonathan May

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) as a highly skilled, but slightly distracted, librarian trying to answer a question. The paper you provided, "How Language Models Process Negation," investigates what happens inside this librarian's brain when you ask a tricky question involving the word "not."

Here is the story of their findings, broken down into simple concepts.

1. The Problem: The Librarian Gets Confused by "Not"

The researchers started with a simple observation: If you ask a modern AI, "What is an animal that is not an amphibian?" it often gives a wrong answer, like "frog" (which is an amphibian). It seems like the AI is ignoring the word "not" entirely.

However, the paper reveals a surprising twist: The AI actually understands "not" perfectly well. It's just that its understanding gets buried under a layer of bad habits.

2. The Two Competing Mechanisms: The "Construction" vs. The "Shortcut"

To understand how the AI thinks, the researchers looked at two different ways it could handle a negative sentence. They compared these to two different ways a person might solve a riddle:

Hypothesis 1: The "Suppression" Strategy (The Eraser)
Imagine you have a list of animals. To answer "not an amphibian," you take the list, find "amphibians," and use an eraser to cross them out. You are left with everything else.
- The Paper's Finding: The AI does this a little bit, but it's not the main way it works.
Hypothesis 2: The "Construction" Strategy (The Architect)
Imagine instead of erasing, the AI builds a brand new mental picture. When it hears "not gas," it doesn't just think "gas" and cross it out. It actively constructs a new concept in its mind: "Solid." It builds a mental model of what the opposite looks like.
- The Paper's Finding: This is the winner. The AI is mostly an architect. It actively builds a representation of the negative concept (e.g., turning "not gas" into "solid") and uses that to find the answer.

3. The Villain: The "Shortcut" Habit

So, if the AI is so good at building these "negative concepts," why does it still get the answers wrong?

The paper identifies a group of "bad actors" inside the AI's brain, specifically in the later layers (the final steps before the AI speaks). The researchers call these "Shortcut Attention Heads."

The Analogy: Imagine the AI is a student taking a test. The student knows the correct answer (because they built the "not gas" = "solid" concept). But, right before they write the answer, a lazy part of their brain whispers, "Hey, I've seen the word 'amphibian' and 'frog' together a million times in my training. Just write 'frog' and save time!"
The Result: This "shortcut" overrides the smart "construction" work. The AI ignores the logic and just guesses based on what words usually appear together.

4. The Fix: Turning Off the Bad Habits

The researchers tested a clever trick to prove this. They used a method they call "Attention Sinking."

The Analogy: Imagine you are listening to a choir. The "Shortcut" is a loud, off-key singer in the back row drowning out the soloist. The researchers didn't try to teach the choir to sing better; they simply muted the loud, off-key singer.
The Result: When they "muted" (ablated) these specific shortcut modules in the later layers, the AI's accuracy on negative questions skyrocketed. It suddenly started giving the correct answers, proving that the ability to understand negation was there all along, just hidden by the shortcut.

5. The Journey of the Thought

The paper maps out exactly how the thought process travels through the AI:

Early Layers (The Movers): The AI takes the word "not" and physically moves its meaning to sit right next to the word it is negating (e.g., moving "not" next to "gas").
Middle Layers (The Builders): This is where the magic happens. The AI constructs the new concept ("Solid") and pushes it forward. At the same time, it weakly tries to suppress the old concept ("Gas").
Late Layers (The Saboteurs): Just as the AI is about to speak, the "Shortcut" heads kick in. They see the word "gas" and the word "amphibian" and try to force the answer back to the most common association, ignoring the "Solid" concept the AI just built.

Summary

The paper concludes that Large Language Models are not bad at understanding "not." They are actually quite good at it, using a sophisticated "construction" method to build the meaning of negative phrases.

However, they are overwhelmed by bad habits (shortcuts) learned during their training. These shortcuts are so strong that they often override the correct logic, leading to errors. By identifying and temporarily turning off these shortcuts, the researchers showed that the AI's true, logical capability is much stronger than its final output suggests.

In short: The AI knows the answer, but it has a bad habit of guessing the easy way out. If you stop it from taking the shortcut, it gets it right.

Technical Summary: How Language Models Process Negation

Problem Statement
Large Language Models (LLMs) frequently fail on queries involving negation (e.g., "An animal that is not an amphibian is a __"), often outputting the positive concept ("frog") instead of the correct negated concept ("dog"). While prior Mechanistic Interpretability (MI) work has established that models process factual recall via additive aggregation of evidence, negation does not fit this paradigm. The paper investigates the internal mechanisms of LLMs to determine whether they possess the capability to process negation and, if so, why they fail to manifest this capability in final outputs. The authors address two competing hypotheses regarding how models handle negation: the Suppression Hypothesis (the model promotes relevant concepts and suppresses the negated token) and the Construction Hypothesis (the model explicitly constructs a new representation for the negated phrase, e.g., representing "not gas" as a vector for "solid").

Methodology
The study employs a controlled dataset of 648 prompts in the form "X that is not Y is Z," utilizing models including Llama-3.1-8B, Mistral-7B, and various OLMo2 checkpoints. The authors apply a suite of observational and causal interpretability techniques:

Sensitivity Metrics: The authors define a sensitivity metric based on logit differences to determine if models internally distinguish between positive and negative prompts, even when accuracy is low.
Attention Sinking: A novel ablation method inspired by attention sink phenomena. By restricting attention heads to attend only to themselves and the first token, the authors effectively nullify specific attention modules. They apply Cumulative Attention Sinking (ablating all layers from a specific point to the end) to identify layers where "shortcut" mechanisms reside.
Path Patching and LogitLens: These techniques are used to trace causal pathways. Path patching replaces attention outputs from negative prompts with those from positive prompts to observe prediction flips. LogitLens projects intermediate hidden states onto the vocabulary to interpret the semantic content of internal representations.
Sparse Autoencoders (SAEs): Used to decompose MLP outputs into sparse latent features, allowing for the identification of specific neurons that promote negated concepts.
LLM-Based Annotation: An automated pipeline using an external LLM (gpt-oss-120b) to label whether tokens promoted by internal attention heads are semantically related to "not Y" or the original concept "Y."

Key Findings

Coexistence of Mechanisms: The paper demonstrates that LLMs possess internal mechanisms capable of correctly processing negation, but these are often overshadowed by "shortcut" mechanisms in later layers.
Shortcut Attention Heads: The authors identify specific attention heads in the middle-to-late layers (typically >50% through the model depth) that promote positive answers on negative prompts. These heads rely on spurious correlations (e.g., co-occurrence of "amphibian" and "frog") rather than logical negation. Ablating these heads via Attention Sinking significantly improves negative accuracy (e.g., from ~50% to ~68% on Llama-3.1-8B).
Construction-Dominant Processing: The primary mechanism for correct negation is Construction, not suppression.
- Early/Mid-Layers: Attention heads move the representation of the negation token ("not") to the position of the concept being negated ("Y").
- Mid-Layer Construction: Causally important attention modules (around layer 14 in Llama-3.1-8B) explicitly construct a negated representation ( $\bar{Y}$ ). LogitLens analysis shows these layers promote tokens semantically related to the negated concept (e.g., "solid" for "not gas") in over 80% of cases.
- Suppression: While the model does suppress the original concept "Y," this occurs to a lesser extent than the construction of the negated concept.
MLP Role: Late-layer MLPs amplify the constructed negated representation, promoting the correct final answer. SAE analysis reveals specific latents that directly instantiate these negated concepts.
Pre-training Origin: The shortcut mechanisms responsible for failures are traced back to early stages of pre-training, where models develop a bias toward positive answers on negative prompts.

Key Contributions

Diagnosis of Negation Failures: The authors show that LLM failures on negation are not due to a lack of internal capability but rather the dominance of shortcut mechanisms in later layers.
Intervention Method: The introduction of Attention Sinking as a plug-and-play intervention that mitigates shortcut behaviors and recovers negation accuracy without additional training.
Mechanistic Account: A detailed, causal map of negation processing, establishing that models implement both construction and suppression, with construction being the dominant mechanism. The paper systematically identifies how attention moves "not" to "Y," constructs $\bar{Y}$ , and how MLPs promote the final answer.

Significance
The paper argues that low black-box accuracy can mask the existence of capable internal mechanisms. By revealing that LLMs are ensembles of competing mechanisms (correct negation processing vs. shortcut heuristics), the work highlights the necessity of thoroughly inspecting model internals to fully audit model capabilities. The findings deepen the understanding of how LLMs perform compositional reasoning and suggest that "construction" is a more central computational strategy for negation than previously assumed. The authors conclude that their work provides a foundation for building more robust and reliable LLMs by understanding and mitigating these competing internal mechanisms.