💬 NLP

Do Hallucination Neurons Generalize? Evidence from Cross-Domain Transfer in LLMs

This paper demonstrates that "hallucination neurons" identified in large language models do not generalize across different knowledge domains, revealing that hallucination mechanisms are domain-specific rather than universal and necessitating domain-calibrated detectors.

Original authors: Snehit Vaddi, Pujith Vaddi

Published 2026-04-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Snehit Vaddi, Pujith Vaddi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Lie Detector" Myth

Imagine you have a high-tech lie detector built inside a robot's brain. This device is designed to spot when the robot is making things up (hallucinating).

Scientists recently discovered that this lie detector works by watching a tiny, specific group of "neurons" (the robot's brain cells). They found that when these specific cells light up, the robot is almost certainly lying. This was a huge breakthrough because it meant we could build a universal "lie detector" that works for any topic the robot talks about.

But this new paper asks a simple question:
If we train this lie detector on the robot talking about history, will it still work if the robot starts talking about law or cooking?

The answer, unfortunately, is no.

The Experiment: Testing the Lie Detector

The researchers treated the robot's brain like a massive city with millions of tiny workers (neurons). They wanted to see if the "liar workers" were the same people regardless of what the robot was doing.

They set up a test with 6 different "neighborhoods" (domains):

General Knowledge (Trivia)
Law (Legal cases)
Finance (Money and stocks)
Science (Facts about the world)
Morality (Right vs. Wrong)
Coding (Computer bugs)

They trained a "lie detector" in each neighborhood. Then, they tried to take the detector from the Law neighborhood and use it in the Science neighborhood.

The Results: A Total Mismatch

The results were shocking. The lie detectors were highly specialized.

Within the same neighborhood: The detector worked great. It caught lies about 78% of the time.
Across neighborhoods: The detector failed miserably. When they took the "Law Lie Detector" and used it on "Science," it barely did better than flipping a coin (56%).

The Analogy:
Imagine you hire a security guard who is an expert at spotting fake paintings. You train him for months on art galleries. He becomes a master at spotting forgeries in oil paintings.

Then, you ask him to guard a bank and spot fake money.

Does he know what a fake $20 bill looks like? No.
Does he know that a fake painting looks suspicious? Yes.
If you try to use his "fake painting" rules to check the money, he might flag a real $20 bill as a fake because it doesn't look like a canvas.

That is exactly what happened here. The "neurons" that light up when the robot lies about law are completely different from the neurons that light up when it lies about science.

The Weird Exceptions

The researchers found a few interesting quirks:

The "Law and Science" Bridge: Interestingly, the lie detectors for Law and Science could sort of understand each other. Why? Because both involve strict logic and rules. If the robot gets the logic wrong in a court case, it uses the same "broken logic" brain cells as when it gets a science fact wrong.
The "Code" Outlier: The Coding neighborhood was totally isolated. The neurons that lie about computer code are so different that they didn't work at all for any other topic. It's like the robot has a completely separate "brain" for coding.
The "Anti-Lie" Effect: In some cases, the detector got it backwards. A neuron that signaled a lie in the Law domain actually signaled the truth in the Moral domain. It's as if the security guard started shouting "THIEF!" whenever someone was actually being honest.

Why Does This Matter? (The Real-World Impact)

This finding changes how we need to build AI safety tools.

The Old Way: "Let's train one AI detector on general trivia, and then we can use it to check medical advice, legal contracts, and financial reports."
The New Reality: "We need six different detectors. You cannot use a 'Medical Lie Detector' to check a 'Legal Lie Detector'."

If you try to use a detector trained on general facts to check a legal contract, you might miss the lies entirely, or worse, you might flag honest answers as lies.

The "Chain of Thought" Twist

The researchers also tested what happens when they tell the robot to "think step-by-step" (Chain of Thought) before answering.

Result: This changed which neurons were lying. It didn't just make the robot think harder; it literally recruited a different set of brain cells to do the thinking. This means even the "thinking process" changes the internal mechanics of the lie.

The Bottom Line

Hallucination isn't a single "bug" in the robot's brain. It's more like a family of different bugs.

Lying about history uses one set of tools.
Lying about law uses a different set.
Lying about code uses a third set.

To catch a robot lying, you can't just have one universal alarm. You need a specialized alarm for every specific room in the house. If you want to deploy AI safely in high-stakes fields like law or medicine, you must build and calibrate your safety detectors specifically for that field, not just hope a general one will work.

1. Problem Statement

Recent mechanistic interpretability research (specifically Gao et al., 2025) identified a sparse set of "Hallucination Neurons" (H-neurons), comprising less than 0.1% of Feed-Forward Network (FFN) neurons, that reliably predict when Large Language Models (LLMs) will hallucinate. These neurons were identified using the Contribution Estimation Through Token-level Traits (CETT) metric on general-knowledge question answering (QA) tasks.

The critical gap this paper addresses is the domain generalizability of these neurons. While previous work showed H-neurons generalize to new instances within the same domain, it remains unknown whether a classifier trained on H-neurons from one knowledge domain (e.g., general QA) can detect hallucinations in fundamentally different domains (e.g., legal, financial, or code). The authors investigate whether hallucination is a universal neural mechanism or a collection of domain-specific pathways.

2. Methodology

Experimental Setup

Models: The study evaluated 5 open-weight instruction-tuned models ranging from 3B to 8B parameters:
- Qwen2.5-3B, Nemotron-Mini-4B, Phi-3.5-mini, Mistral-7B, and Llama-3.1-8B.
Domains: Six distinct knowledge domains were selected to represent diverse reasoning types:
1. General QA: TriviaQA (factual recall).
2. Legal: CaseHOLD (legal holding identification).
3. Financial: FinancialPhraseBank (sentiment analysis).
4. Science: ARC-Challenge (grade-school science reasoning).
5. Moral Reasoning: ETHICS (commonsense moral judgment).
6. Code: Devign (vulnerability detection).
Data: 1,000 questions sampled per domain. Two prompting strategies were tested: Direct prompting and Chain-of-Thought (CoT).

H-Neuron Identification & Classification

Feature Extraction: For each input, the authors extracted CETT scores for all FFN neurons across all layers. The CETT score for a neuron $j$ in layer $\ell$ is calculated as:
$CETT_j^{(\ell)} = a_j^{(\ell)} \cdot \|w_j^{(\ell)}\|^2$
where $a$ is the post-activation value and $w$ is the down-projection weight vector.
Labeling: Inputs were labeled as "hallucinating" or "non-hallucinating" based on exact match or normalized string matching against ground truth.
Classifier Training: An L1-regularized logistic regression classifier was trained on the neuron activation features of a source domain to predict hallucination labels. L1 regularization ensures sparsity, selecting only the most predictive neurons (the H-neurons).
Cross-Domain Transfer Protocol: A $6 \times 6$ $6 \times 6$ transfer matrix was constructed for each model. Each classifier trained on domain $i$ $i$ was evaluated on every domain $j$ $j$ (including $i$ $i$ itself).
- Metric: Area Under the Receiver Operating Characteristic Curve (AUROC).
- Transfer Gap ( $\Delta$ ): Defined as the difference between mean within-domain AUROC and mean cross-domain AUROC.

Robustness Checks

Bootstrap: 1,000 resamples for 95% confidence intervals.
Permutation Testing: 10,000 iterations to test the null hypothesis that the transfer gap is zero.
Stability: Multi-seed analysis confirmed moderate agreement (Jaccard similarity $\approx 0.47$ ) in selected neuron sets.

3. Key Contributions

First Systematic Cross-Domain Analysis: The paper provides the first comprehensive evaluation of H-neuron transferability across 6 distinct domains and 5 models, generating 180 train-test domain pairs.
Evidence of Domain Specificity: It establishes that H-neurons do not generalize across domains, challenging the assumption of a universal "hallucination signature."
Mechanistic Insights: The study identifies specific patterns of partial transfer (e.g., between legal and science) and anti-correlated transfer (below-chance performance), suggesting hallucination arises from domain-conditioned neural pathways rather than a single mechanism.

4. Key Results

Primary Finding: The Transfer Gap

Within-Domain Performance: Classifiers trained and tested on the same domain achieved a mean AUROC of 0.783.
Cross-Domain Performance: When transferred to a different domain, performance dropped significantly to a mean AUROC of 0.563.
Transfer Gap ( $\Delta$ ): The degradation was 0.220 ( $p < 0.001$ ), a statistically and practically significant drop (approx. 28% loss in discriminative power).
Consistency: This gap was consistent across all 5 models tested, regardless of parameter count (3B–8B).

Domain-Specific Patterns

Legal-Science Bridge: A notable exception was the transfer between Legal and Science domains. Classifiers trained on one performed relatively well on the other (AUROC $\approx 0.70$ ), suggesting shared neural substrates for structured logical reasoning.
Code as an Outlier: Code vulnerability detection showed the most distinct neural pathways. Classifiers trained on code performed near random chance (AUROC $\approx 0.51$ ) on all other domains, and vice versa.
Below-Chance Transfers: Several transfers resulted in AUROC < 0.5 (e.g., Code $\to$ Moral reasoning on Nemotron-4B had AUROC 0.210). This indicates anti-correlation: neurons predicting hallucination in one domain actively predicted correct answers in another, implying conflicting neural circuits.

Chain-of-Thought (CoT) Effects

CoT prompting altered the set of identified H-neurons significantly (changing neuron counts by up to +100 or -76).
While CoT sometimes improved within-domain detection, it did not eliminate the cross-domain transfer gap. The mechanism of hallucination under CoT involves reorganizing signals across different neuron populations rather than simply suppressing them.

Model Scale Analysis

Domain specificity did not monotonically increase or decrease with model size (3B to 8B). The transfer gap varied non-linearly across models, suggesting that architecture and training data composition play a larger role than raw parameter count in this range.

Causal Intervention (Activation Scaling)

The authors attempted to modulate hallucination by scaling the activations of identified H-neurons.
Result: No significant causal effect was found. Scaling H-neurons did not reliably increase or decrease hallucination rates, suggesting these neurons are diagnostic markers rather than sufficient causal levers at this level of sparsity (<0.1%).

5. Significance and Implications

Refutation of Universal Detectors: The study disproves the implicit assumption that a hallucination detector trained on general benchmarks (like TriviaQA) can be deployed universally.
Deployment Requirements: Practitioners must build domain-specific detection infrastructure. A single detector cannot cover legal, medical, financial, and code applications simultaneously.
Intervention Strategy: Attempts to suppress hallucinations by targeting H-neurons must be domain-targeted. Suppressing neurons identified in general QA could inadvertently increase hallucinations in other domains (due to the anti-correlated signals observed).
Theoretical Shift: Hallucination should be viewed not as a single "bug" with a universal neural signature, but as a family of domain-conditioned processes arising from the failure of specific reasoning circuits or knowledge retrieval pathways.

Conclusion

The paper concludes that hallucination in LLMs is mechanistically heterogeneous. The "hallucination signal" is not a universal feature but is deeply entangled with the specific knowledge domain and reasoning type being processed. Future work must focus on domain-specific calibration and investigating whether larger models (70B+) develop more universal representations or if this domain specificity persists at scale.