How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study

Technical Summary: How Humans Process AI-generated Hallucination Contents: a Neuroimaging Study

Problem Statement
While AI-generated hallucinations (plausible but factually incorrect content) pose significant risks in critical domains like healthcare and finance, the underlying cognitive mechanisms by which humans recognize or are misled by these hallucinations remain unclear. Existing research has largely focused on the model side (detecting and mitigating hallucinations) or on explicit user behaviors. There is a lack of understanding regarding the temporal dynamics of human brain activity when processing hallucinated versus non-hallucinated content generated by Multi-modal Large Language Models (MLLMs). Specifically, it is unknown at which processing stage detection succeeds or fails, and whether neural signals can predict the presence of hallucinations.

Methodology
The study employed a neuroimaging approach using Electroencephalography (EEG) to investigate the neural dynamics of 27 participants (aged 19–30) performing a verification task.

• Stimuli: Participants viewed images and corresponding text descriptions generated by the MLLM Qwen2.5-VL-3B-Instruct. The dataset consisted of 60 image-text pairs, carefully curated to include hallucinated content (categorized as entity, attribute, or relation hallucinations based on the AMBER benchmark) and non-hallucinated (correct) content.
• Task: In a multi-modal QA task, participants viewed an image followed by a word-by-word presentation of a sentence description. They were required to judge whether the sentence matched the image content (Yes/No).
• Data Analysis: The researchers recorded continuous EEG signals and performed Event-Related Potential (ERP) analysis. They segmented the data into three behavioral categories:
  1. HalluCorrect: Hallucinated words correctly identified as hallucinations.
  2. NoHallu: Non-hallucinated words correctly identified as correct.
  3. HalluWrong: Hallucinated words incorrectly judged as correct (misjudged).
  • Statistical analysis involved repeated-measures ANOVA across specific time windows (50–750 ms) and brain regions (ROI) to identify significant ERP component differences (N100, P200, N400, P600).
• Prediction Experiments: The study further conducted word-level and sentence-level classification experiments using Support Vector Machines (SVM), Gradient Boosting Decision Trees (GBDT), and attention-based models. Features included Frequency-Band-based Features (FBFs) and Event-Related Potential-based Features (ERPFs) extracted from four brain regions.

Key Results

1. ERP Signatures of Recognition: Significant differences in brain signals were observed when participants correctly recognized hallucinations compared to when they processed non-hallucinated content.
   • Early Processing (N100/P200, 50–280 ms): Correctly identified hallucinations elicited enhanced N100 and P200 amplitudes in temporal, parietal, occipital, and frontal regions. This suggests that successful detection recruits early attentional resources and imposes higher cognitive load on the perceptual system.
   • Semantic Integration (N400, 280–550 ms): HalluCorrect words evoked larger N400 amplitudes than NoHallu words, indicating increased effort in semantic integration and conflict resolution when the text diverges from visual context.
   • Reanalysis (P600, 550–750 ms): Enhanced P600 responses were observed for HalluCorrect words, reflecting controlled reanalysis and monitoring processes required to resolve semantic conflicts.
2. The "Misjudgment" Gap: Crucially, when participants failed to detect hallucinations (HalluWrong), their neural responses did not significantly differ from those processing non-hallucinated content (NoHallu). The typical neural markers of anomaly detection (enhanced N100, P200, N400, P600) were absent. This indicates that deceptive, fluent hallucinations fail to trigger the standard neurocognitive fact-verification pathway.
3. Prediction Capabilities: Machine learning models could successfully predict whether content contained hallucinations based on EEG signals at both word and sentence levels, but only when participants correctly recognized the hallucination (AUC up to 0.98 for sentence-level). When participants misjudged the content, prediction performance dropped to random chance levels, confirming that EEG signals only carry discriminative information when the cognitive conflict is consciously registered.

Key Contributions

1. Dataset Creation: The authors collected and will release an EEG dataset from 27 participants involving MLLM-generated text, covering both hallucinated and non-hallucinated conditions.
2. Neural Mechanism Characterization: The study identifies that human recognition of MLLM hallucinations involves a cascade of cognitive processes: early attentional engagement, semantic-thematic integration, inferential reasoning, and memory retrieval. It establishes that these neural signatures are contingent upon the participant's conscious recognition of the error.
3. Implicit Feedback Validation: The research demonstrates the feasibility of using EEG as an implicit feedback signal to detect hallucinations, provided the user is aware of the discrepancy. It highlights that the absence of neural conflict markers correlates with the failure to detect deceptive content.

Significance and Claims
The paper claims to advance the understanding of human-AI interaction by revealing that the cognitive risk of hallucinations depends heavily on whether humans can successfully recognize them. The authors argue that:

• For AI Design: Understanding these distinct neural stages (attention, semantic matching, memory retrieval) could guide the development of MLLMs with modules that are "aware" of deviations, rather than relying solely on passive prediction.
• For Human-AI Interaction: The findings suggest that user interfaces should incorporate active interventions to help users notice incongruities, as conscious awareness is a prerequisite for triggering the neural mechanisms that detect errors.
• Limitations: The authors modestly note that their study is constrained to a laboratory setting with a limited number of hallucination words per participant. They acknowledge that factors like ambient noise and natural reading behaviors were not fully captured, and that the generalizability to experts in specific domains (e.g., medicine, law) requires further investigation.

The study concludes that while EEG can predict hallucinations, this capability is intrinsically linked to the user's successful detection of the error, implying that highly fluent hallucinations may bypass human neurocognitive fact-checking mechanisms entirely.