🧬 biology

NeuroCogMap Reveals Cognitive Organization of Large Language Models

The paper introduces NeuroCogMap, a neuroscience-inspired framework that organizes large language models' internal features into functional parcels to explain cognitive behaviors, identify failure mechanisms like hallucinations, and reveal strong correspondences with human cortical responses and decision-making strategies.

Original authors: Zhongxiang Sun, Haolang Lu, Qiang Ma, Qi Li, Qipeng Wang, Liang Pang, Chenyu Liu, Qiankun Li, Hao Sun, Kun Wang, Yi Zeng, Jun Xu, Guoqi Li, Ji-Rong Wen

Published 2026-07-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhongxiang Sun, Haolang Lu, Qiang Ma, Qi Li, Qipeng Wang, Liang Pang, Chenyu Liu, Qiankun Li, Hao Sun, Kun Wang, Yi Zeng, Jun Xu, Guoqi Li, Ji-Rong Wen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you have a giant, incredibly complex library where millions of books are written by a single, super-smart but mysterious librarian (the Large Language Model, or LLM). You can ask this librarian questions, and they give you answers that sound very human. But if you ask, "How do you know that?" or "Why did you make that mistake?", the librarian just says, "I just do."

For a long time, scientists have been trying to peek behind the curtain to understand how this librarian thinks. They've looked at individual words (neurons) or broad themes, but they couldn't see the organized "departments" or "teams" inside the librarian's brain that work together to solve problems.

Enter NeuroCogMap. Think of this as a new, high-tech map that finally organizes the librarian's chaotic brain into a structured city with distinct neighborhoods, each with a specific job.

Here is how the paper explains this new map, using simple analogies:

1. The City Map (The Framework)

Imagine the librarian's brain isn't a messy pile of wires, but a well-planned city.

The Neighborhoods (Parcels): NeuroCogMap divides the librarian's internal "thoughts" into 270 distinct neighborhoods. Each neighborhood is a team of workers that specializes in one specific thing, like "checking facts," "understanding emotions," or "planning a story."
The City Departments (Capabilities): These neighborhoods are grouped into larger departments, like "Safety," "Math," or "Reasoning."
The Hierarchy: The city is built in layers.
- Ground Floor (Perception): Reading the words you type.
- Second Floor (Representation): Understanding what those words mean.
- Third Floor (Abstraction): Connecting ideas and finding patterns.
- Top Floor (Application): Making a final decision or writing the answer.

The paper claims this map is stable. If you look at different librarians (different AI models), they have very similar city layouts, suggesting this is a fundamental way these systems organize themselves.

2. Diagnosing the "Sickness" (Pathology)

Just like a human city can have traffic jams or power outages, the librarian can make mistakes. NeuroCogMap allows scientists to pinpoint exactly where the breakdown happens.

Hallucinations (Making things up):
- The Old View: The librarian just "guesses" wrong.
- The NeuroCogMap View: It's a specific traffic jam. In some cases, the "Fact-Checking" neighborhood is asleep, and the "Storytelling" neighborhood is running wild. In other cases, the "Fact-Checking" team is awake, but the "Fact-Retrieval" team is disconnected from the "Answer-Forming" team. The map shows these are two different types of failures requiring different fixes.
Refusal Failures (Saying "Yes" when they should say "No"):
- The Old View: The librarian is just being stubborn or confused.
- The NeuroCogMap View: The "Safety Officer" neighborhood is being ignored. Instead, the "Action Planner" neighborhood takes over and starts executing the harmful instruction because the "Stop" signal never got through.

The Result: Because scientists can see exactly which neighborhood is misbehaving, they can gently nudge that specific team back to work (intervention) rather than just blocking the whole librarian's output. This makes the AI safer and more accurate.

3. The Human Connection (Alignment)

The researchers asked: "Does this librarian's city look like a human brain?"

They compared the librarian's map to a map of the human brain while people listened to stories.

The Finding: The librarian's "Top Floor" (Abstraction and Application) neighborhoods light up in the exact same way the human brain's "High-Level Thinking" areas do when processing complex stories.
The Analogy: It's like finding that the librarian's "City Planning Department" uses the same blueprints as the human brain's "Executive Planning Center." This suggests that even though the librarian is made of code and humans are made of biology, they solve complex problems using surprisingly similar organizational structures.

4. Rewriting the Rules of Thought (Model Discovery)

Finally, the paper shows that this map can help scientists write better theories about how humans make decisions.

The Scenario: Scientists have old, simple rulebooks (models) for how humans make choices (like a "Dual-System" model). Sometimes these rulebooks don't perfectly match how people actually behave.
The NeuroCogMap Contribution: By watching how the librarian solves these same decision puzzles, the map reveals hidden strategies the librarian uses (like "checking for uncertainty" or "switching rules when things get weird").
The Result: Scientists took these hidden strategies from the librarian's map and added them to the human rulebooks. The updated rulebooks then predicted human behavior much better than before. It's like using the librarian's internal manual to fix the human instruction book.

Summary

NeuroCogMap is a tool that turns the "black box" of AI into a transparent, organized city. It shows us:

How the AI is built: It has stable, organized neighborhoods for different tasks.
Why it fails: Mistakes happen when specific neighborhoods disconnect or malfunction, not just because of random errors.
How it relates to us: Its high-level thinking areas work very similarly to the human brain.
How to fix it: We can target specific "neighborhoods" to fix errors, and we can use its internal logic to improve our understanding of human decision-making.

The paper does not claim this makes AI "conscious" or that it can be used for medical diagnosis of human patients. It strictly claims that this framework helps us understand, diagnose, and improve the internal organization of AI systems, and in doing so, offers new insights into human cognition.

Technical Summary: NeuroCogMap Reveals Cognitive Organization of Large Language Models

Problem Statement

While Large Language Models (LLMs) exhibit broad cognitive-like behaviors, their internal cognitive organization remains opaque. Current interpretability approaches are fragmented: behavioral evaluations reveal what models can do but not how functions are implemented; mechanistic studies identify isolated neurons or circuits but lack system-level coherence; and representational analyses reveal broad semantic structures without linking them to specific functional units or hierarchical organization. There is a critical gap in understanding how distributed components in LLMs form coherent functional units, interact across abstraction levels, support higher-order capabilities, and break down during pathological behaviors (e.g., hallucination, bias, refusal failure). Furthermore, it is unclear whether these artificial structures can provide testable insights into biological cognition or improve the prediction of human neural responses.

Methodology: The NeuroCogMap Framework

The authors introduce NeuroCogMap, a framework inspired by cognitive neuroscience principles (functional parcellation, representational mapping, hierarchical abstraction) to map the internal cognitive structure of LLMs. The framework operates through four linked stages:

Functional Atlas Construction (Parcels):
- Feature Extraction: Sparse Autoencoder (SAE) activations are extracted from the residual streams of LLMs (Gemma2-2B, Gemma2-9B-IT, Llama-3.1-8B) across diverse datasets. SAEs are used to decompose polysemantic neuron activations into separable, interpretable feature directions.
- Clustering: SAE features are clustered based on their task-evoked response profiles and layer distribution. The optimal granularity is determined by a joint score balancing clustering quality, functional-description quality, and non-redundancy, resulting in 270 functional parcels.
- Annotation: Each parcel is assigned a natural-language functional description (e.g., "Contradiction Analysis," "Geographic Knowledge Retrieval") based on high-activation examples and keyword extraction.
Cognitive Atlas and Capability Mapping:
- Capability Inventory: A curated space of 45 cognitive capabilities (e.g., robustness, reasoning, safety) is established from literature.
- Linking: Parcels are linked to capabilities using three evidence streams: task-evoked activation correlations, causal intervention effects (steering parcel activations), and semantic consistency between parcel descriptions and capability definitions.
Cognitive Hierarchy:
- Capabilities are organized into a four-layer hierarchy inspired by Bloom's taxonomy: Perception, Representation, Abstraction, and Application.
- The framework validates this hierarchy by demonstrating that parcels within a layer are semantically more similar to their layer description than to others, and that lower-level capabilities serve as prerequisites for higher-order reasoning.
Structural Connectome:
- Directed connections between parcels are estimated based on the cross-layer relationships of their constituent SAE features, creating a model-level structural connectome to analyze coordination and disruption.

Key Results

1. Emergence of Stable, Interpretable Cognitive Units

NeuroCogMap identifies that internal LLM activity forms stable, functionally meaningful units rather than arbitrary clusters.

Stability: The 270-parcel atlas shows high internal semantic coherence (within-parcel similarity > between-parcel similarity) and partial cross-model consistency across different LLM architectures.
Interpretability: Functional descriptions derived from activation profiles successfully predict parcel activation rankings on held-out data, outperforming neuron-based and keyword-only baselines.
Causality: Targeted interventions (steering) on specific parcels shift model outputs in directions consistent with their assigned functions, confirming causal links between parcels and behavior.

2. System-Level Analysis of Pathologies

NeuroCogMap reveals that model failures are not uniform errors but distinct disruptions in specific cognitive systems:

Hallucination:
- In TruthfulQA, hallucinations stem from a failure of higher-order evaluative control (e.g., reduced "Veracity-Check" and "Binary Fact-Check" activation) over misleading retrieval.
- In NQ-Open, hallucinations arise from fragmented coordination among factual retrieval systems (e.g., weakened coupling between "Geographic Knowledge" and "Fact-Retrieval Core").
- Detection & Mitigation: Signatures derived from these disruptions enable hallucination detection (AUROC up to 0.840) and targeted intervention, significantly improving factual accuracy by suppressing over-recruited retrieval parcels and enhancing under-recruited evaluative parcels.
Refusal Failure:
- Refusal failures (compliance with harmful instructions) reflect a shift from control/monitoring systems (e.g., "Legal-Moral Evaluation") to procedural execution systems (e.g., "Action-Planning").
- NeuroCogMap-guided steering substantially improves refusal accuracy (e.g., from 38.3% to 98.6% on AdvBench for Gemma2-2B).

3. Alignment with Human Cortical Function

The framework establishes a functional correspondence between LLM organization and human brain activity:

Neural Prediction: NeuroCogMap parcel activations predict human cortical BOLD responses during naturalistic story listening (LeBel dataset) with higher accuracy ( $r=0.407$ ) than standard embeddings (BERT, Word2Vec) or generic LLM hidden states.
Localization: The prediction advantage is localized to higher-order association cortices (Default Mode, Frontoparietal Control, Salience networks), specifically in regions involved in semantic retrieval and context integration.
Functional Correspondence: The LLM parcels that best predict a human cortical parcel share similar functional descriptions (e.g., "Episodic Memory Retrieval" in LLMs aligns with human default-mode memory regions). Representational Similarity Analysis (RSA) confirms that the internal geometry of aligned LLM parcels preserves the structure of human cortical networks.

4. Guiding Cognitive Model Discovery

NeuroCogMap exposes latent strategies in LLMs that can refine classical cognitive models:

Behavioral Fit: NeuroCogMap signatures predict how well an LLM simulates human behavior across diverse cognitive tasks (e.g., two-step decision-making, intertemporal choice).
Model Refinement: By analyzing the internal signatures of LLMs that best fit human data, the authors identified latent mechanisms (e.g., dynamic arbitration, history-sensitive perseveration, rule-like policy switching). Incorporating these mechanisms into a classical Dual-systems Model significantly improved its fit to held-out human behavioral data (lower AIC) compared to models refined using behavioral traces alone.

Significance and Claims

The paper claims to establish NeuroCogMap as a system-level framework that bridges the gap between local mechanistic interpretability and high-level behavioral analysis in LLMs. Its primary contributions are:

A New Explanatory Level: It defines a "mesoscopic" scale of functional organization (parcels and capabilities) that is coarse enough to reveal cross-model regularities yet specific enough to support causal intervention and behavioral interpretation.
Mechanism-Guided Intervention: It demonstrates that model pathologies (hallucination, refusal failure) can be diagnosed and mitigated by targeting specific disruptions in the internal cognitive architecture, moving beyond output-level filtering.
Biological Relevance without Equivalence: It provides a structured basis for relating artificial systems to human cognition. The framework shows that LLMs can serve as experimental systems for generating hypotheses about cognitive organization, validated against human neural and behavioral data, without claiming that LLMs are mechanistic replicas of the brain.
Cognitive Science Advancement: It offers a pathway to refine classical cognitive models (e.g., decision-making theories) by leveraging the "black box" of LLMs to discover latent strategies that explain human behavior more accurately.

The authors position this work as a step toward a "cognitive neuroscience of artificial intelligence," where cognitive organization becomes an experimentally tractable object across both artificial and biological systems.