Technical Summary: Modular Cognitive Architecture Emerges in Large Language Models

Problem Statement

The human brain exhibits a striking degree of functional specialization, where distinct neural networks support specific cognitive domains such as language, formal reasoning, social reasoning (Theory of Mind), and physical reasoning. A central question in cognitive science is whether this modular organization is a fundamental principle required for any intelligent system to function effectively, or merely an evolutionary accident specific to biological tissue. While biological brains face metabolic constraints that may drive sparsity and specialization, artificial systems like Large Language Models (LLMs) are optimized via gradient descent on next-token prediction without such biological pressures. This paper investigates whether LLMs, despite their different optimization processes, spontaneously develop a modular architecture that mirrors the functional segregation observed in the human brain.

Methodology

The authors conducted a large-scale circuit analysis across N=46 tasks spanning four cognitive domains:

1. Language: Morphology, syntax, and lexical semantics.
2. Formal Reasoning: Arithmetic, logical reasoning, and code comprehension (aligned with the Multiple Demand network).
3. Social Reasoning: Emotion inference, belief attribution, and social norms (aligned with the Theory of Mind network).
4. Physical Reasoning: Intuitive physics regarding objects, materials, and forces.

Experimental Pipeline:

1. Minimal Pairs: For each task, the authors constructed pairs of inputs (original vs. alternative) that differed minimally in a task-relevant dimension (e.g., changing a subject from singular to plural, or shifting a number in an equation) but elicited different correct continuations.
2. Attribution Patching: Using a gradient-based method, the authors identified the causal importance of individual neurons. They calculated an attribution score for each neuron by multiplying the activation difference between the original and alternative inputs by the gradient of the logit difference. This estimates the neuron's causal contribution to the task.
3. Overlap Analysis: The authors identified the top 0.1% most important neurons for each task and computed the pairwise Jaccard overlap of these neuron sets across all task pairs.
4. Causal Ablation: To validate causality, the authors performed targeted ablation experiments. They replaced the activations of the top 0.1% neurons identified for a "source" task with their values from an alternative input while the model performed a "target" task. They measured the resulting drop in accuracy to determine if neurons critical for one domain interfered with tasks in the same or different domains.
5. Models: The analysis was conducted on six state-of-the-art instruction-tuned LLMs ranging from 24B to 123B parameters (Mistral, Qwen, OLMo, Llama families). A low-performing baseline (GPT-2) was also tested to rule out dataset artifacts.

Key Results

1. Emergence of Modular Architecture

The study found that LLMs develop a modular functional organization that closely mirrors the human brain:

• Within-Domain Overlap: Tasks within the same cognitive domain (e.g., different arithmetic problems) recruited substantially overlapping sets of neurons.
• Cross-Domain Segregation: Tasks from different domains (e.g., arithmetic vs. physical reasoning) recruited largely distinct sets of neurons.
• Quantitative Metrics: On average, within-domain neuron overlap exceeded cross-domain overlap by more than fourfold (12.9% vs. 3.0%, p < 0.0001). This pattern held consistently across all six models and was recovered via unsupervised hierarchical clustering (Adjusted Rand Index = 0.78).

2. Causal Specificity

Ablation experiments confirmed that this modularity is causal:

• Within-Domain Ablation: Disrupting neurons critical for a specific domain caused a significant drop in performance for tasks within that same domain (mean accuracy drop: 25.9%).
• Cross-Domain Ablation: Disrupting neurons from one domain had a negligible effect on tasks in other domains (mean accuracy drop: 2.5%).
• Double Dissociation: The effect was highly asymmetric. Language neurons were the most selective, disrupting other language tasks by 20.4% while affecting other domains by less than 0.5%. Conversely, lesioning reasoning neurons impaired reasoning conclusions while preserving linguistic well-formedness.

3. Contingency on Performance

The modular structure was contingent on the model's ability to solve the tasks:

• High-Performing Models: All six large models exhibited strong modularity.
• Low-Performing Baseline (GPT-2): GPT-2, which failed most reasoning tasks (performing near or below chance), did not exhibit fine-grained modularity across the three reasoning domains. It only recovered a broad separation between language and non-language tasks, suggesting that functional specialization emerges as a solution to task performance requirements rather than being an artifact of the data or pipeline.

4. Independence from Semantic Similarity

Control analyses demonstrated that the observed modularity was not driven by semantic similarity in the prompts. Semantic similarity matrices (using TF-IDF, GloVe, SBERT, and input token embeddings) failed to recover the four-domain structure as effectively as the neuron overlap matrix. Partial regression confirmed that neuron overlap carried significant domain-specific information independent of lexical or semantic overlap.

Significance and Claims

The paper argues that the convergent emergence of modularity in both biological brains and artificial neural networks suggests that functional specialization is a fundamental property of intelligent systems, likely arising from the computational demands of solving diverse, interacting problems.

• Beyond Metabolic Constraints: Since LLMs do not face the metabolic energy constraints that biological brains do, the emergence of modularity in LLMs suggests that metabolic pressure is not a necessary condition for functional specialization.
• Computational Necessity: The authors propose that modularity arises to prevent interference. When a system must simultaneously engage in multiple forms of reasoning (e.g., parsing syntax while performing logical deduction), segregating computations into distinct subpopulations of neurons protects against representational collision and catastrophic interference during learning.
• LLMs as a Testbed: The study positions LLMs as a valuable second class of intelligent systems for testing claims about the structure of cognition. The fact that modularity emerges in systems shaped by different pressures (gradient descent vs. evolution) strengthens the hypothesis that modularity is a general principle of intelligence.

The authors conclude that while they have demonstrated the existence of this modular organization in LLMs, the specific mechanisms driving it (e.g., whether it is an optimization artifact or a necessary solution to the "next-token prediction" problem in heterogeneous corpora) remain an open question for future research.