💬 NLP

CoRect: Context-Aware Logit Contrast for Hidden State Rectification to Resolve Knowledge Conflicts

CoRect addresses knowledge conflicts in Retrieval-Augmented Generation by identifying and rectifying layers where parametric priors suppress retrieved context through a label-free, logit-contrast-based hidden state rectification method.

Original authors: Xuhua Ma, Richong Zhang, Zhijie Nie

Published 2026-02-10

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Xuhua Ma, Richong Zhang, Zhijie Nie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a student taking a history exam. You have a textbook open in front of you (this is the "Retrieved Evidence"), but you also have a very strong memory of what you learned in a different class years ago (this is the "Parametric Knowledge").

Suddenly, the textbook says the war ended in 1980, but your brain is screaming, "No! It was 1979!" Even though you see the correct answer right in front of you, your brain "overwrites" the truth with your old memory. You end up writing 1979 on the test. This is exactly what happens to Large Language Models (like ChatGPT) in a process called RAG (Retrieval-Augmented Generation). They get the right info, but their "internal bias" wins the argument.

The researchers created a tool called CoRect to fix this. Here is how it works using three simple analogies:

1. The "Internal Saboteur" (The Problem)

The researchers discovered that the mistake doesn't happen at the very end when the model is "speaking." Instead, it happens deep inside the model's "brain" (the hidden layers).

Think of the model like a factory assembly line. In the early stages, the workers are correctly picking up the right parts (the correct facts). But as the product moves toward the end, a "saboteur" worker in the middle of the line (a specific layer in the neural network) swaps the correct part for a wrong one because they are so used to doing it the old way. This is what they call Parametric Suppression.

2. The "Smart Detective" (Stage 1: Token Selection)

Before fixing the problem, the model needs to know what the correct answer should be, without being told.

Imagine a detective walking into a crime scene. Instead of just guessing, the detective looks at two things:

The Clues: What does the evidence (the context) suggest?
The Witnesses: Is the model's own "attention" actually looking at the evidence?

CoRect uses a "Detective" phase to find a Target Token. It compares the model's thoughts with the evidence versus its thoughts without the evidence. If a specific word (like "1980") gets a huge boost when the evidence is present, the detective marks it as the "True Target."

3. The "Volume Knob" (Stage 2: Hidden State Rectification)

Now that the detective has found the truth, how do we stop the "saboteur" in the factory?

Most old methods tried to "shout" the right answer at the very end of the assembly line. This is like a manager yelling at the finished product to change its color—it's messy and often ruins the product's quality.

CoRect does something much more elegant. It goes back to the specific "saboteur" worker in the middle of the line and simply turns down their volume.

If the saboteur is trying to push "1979" into the stream, CoRect applies a mathematical "counter-force" to cancel out that specific wrong signal. It doesn't force the model to say "1980"; it simply removes the interference that was preventing "1980" from coming through. It’s like noise-canceling headphones for the model's brain—it silences the "internal noise" so the "truthful signal" can be heard clearly.

The Result

By "cleaning" the internal thoughts rather than just forcing the final words, the model stays smart and fluent. It doesn't start talking like a robot; it just stops lying to itself. It becomes a student who actually listens to the textbook instead of just relying on a faulty memory.

Technical Summary: CoRect (Context-Aware Logit Contrast for Hidden State Rectification)

CoRect is a novel, training-free framework designed to resolve knowledge conflicts in Retrieval-Augmented Generation (RAG). Knowledge conflict occurs when a Large Language Model's (LLM) internal parametric knowledge (memorized during pre-training) contradicts the external evidence provided in the retrieval context, leading to hallucinations and unfaithful outputs.

1. The Problem: Parametric Suppression

The authors identify a specific mechanistic failure mode they term "Parametric Suppression." Through layer-wise analysis using the Logit Lens technique, they observe that while early layers of an LLM may successfully encode the correct information from the retrieved context, deeper layers—specifically certain Feed-Forward Network (FFN) modules—act as "key-value memories" that overwrite these context-sensitive representations with incorrect, memorized priors.

Limitations of existing solutions:

Decoding-time interventions (e.g., CAD, AdaCAD): These treat the model as a "black box," intervening only at the final output layer. They fail to stop the internal propagation of incorrect information and can cause semantic instability.
Model Editing (e.g., ROME, MEMIT): These modify weights to fix facts but require a "gold" target answer (oracle) to perform the update, making them unsuitable for open-ended, autonomous RAG at inference time.

2. Methodology: The CoRect Framework

CoRect shifts the intervention from the output layer to the internal residual stream. It operates in two distinct stages:

Stage 1: Trustworthy Token Selection (Target-Agnostic)
Since the model does not know the "correct" answer in advance, CoRect must infer a reliable target token ( $\tilde{t}^*$ ) from the prompt itself without using ground-truth labels. It uses two metrics:

Contextual Mutual Information: It performs two forward passes (one with full context, one with a "null" context). It calculates the boost in likelihood for tokens provided by the context to identify informative candidates.
Attention-based Evidence Filter: It verifies these candidates by checking if the model’s internal attention heads are actually focusing on the positions in the context where the candidate token appears.

Stage 2: Hidden State Rectification (Disinhibition)
Once a target token is selected and the "suppressive" layers are localized (layers where the FFN output negatively impacts the target token's logit), CoRect applies a rectification patch ( $\phi_l$ ).

Instead of "injecting" new knowledge (which can be aggressive and cause errors), CoRect performs "disinhibition."
It mathematically constructs a patch to cancel out the component of the FFN output that is actively suppressing the target token. This restores the model's ability to follow the evidence while preserving its natural linguistic fluency.

3. Key Contributions

Mechanistic Insight: Unveiling the "Parametric Suppression" phenomenon, proving that deep FFN layers are the primary culprits in overriding context.
Target-Agnostic Localization: A method to identify conflict-inducing layers during inference without needing the correct answer.
Training-Free Intervention: A lightweight, inference-time mechanism that rectifies hidden states rather than just manipulating final logits.
High Recall: The method achieves over 70% recall in identifying the critical layers that weight-editing methods (like ROME) would target.

4. Experimental Results

The authors evaluated CoRect across multiple backbones (Qwen2.5, LLaMA3, Mistral, LLaMA2) and diverse tasks:

Question Answering (QA): On high-conflict benchmarks like NQ-Swap (where context explicitly contradicts priors), CoRect significantly outperformed baselines (CAD, AdaCAD, COIECD) in Exact Match (EM) scores.
Summarization: On datasets like XSum and CNN/DailyMail, CoRect improved factual alignment (AlignScore) and ROUGE-L scores, demonstrating that it can resolve conflicts without degrading the quality of the summary.
Robustness: Ablation studies confirmed that the "disinhibition" approach (setting the scaling factor $\alpha=1$ ) is superior to "direct decoding" (forcing a token), as it maintains the model's internal semantic coherence and prevents "typos" or linguistic artifacts.

5. Significance

CoRect represents a shift in RAG research from surface-level output manipulation to mechanistic internal rectification. By addressing the root cause of hallucinations—the internal overwriting of context by parametric memory—it provides a more robust, interpretable, and mathematically grounded way to ensure LLMs remain faithful to provided evidence. This is particularly critical for high-stakes RAG applications where factual accuracy is paramount.