Psychological Imagination Networks Show Cross-Population Centrality and Clustering Alignment in Humans That Large Language Models Fail to Replicate

Technical Summary: Imagination Vividness Network Analysis in Humans vs. Large Language Models

Problem Statement
While mental imagery vividness is established as a stable individual trait with replicable latent factor structures, the relational architecture of imagined scenarios remains unexplored. Specifically, it is unknown whether the covariation structure among vividness ratings for different scenarios (e.g., visual scenes or sensory modalities) forms a consistent network topology across diverse human populations. Furthermore, it is unknown whether Large Language Models (LLMs), which acquire knowledge of imagery solely through linguistic training rather than embodied experience, can replicate the centrality and community structure of human imagination networks. The core question is whether LLMs can reproduce the "relational structure" of imagination that arises from shared memory organization, or if they merely produce plausible linguistic ratings without underlying structural coherence.

Methodology
The study employed psychological network analysis to map the partial correlation structure of vividness ratings from two validated instruments:

1. VVIQ-2 (Vividness of Visual Imagery Questionnaire): 32 items across 8 environmental scene contexts.
2. PSIQ (Plymouth Sensory Imagery Questionnaire): 21 items across 7 sensory modalities.

Human Data:

• Samples: Geographically and linguistically distinct groups: Florida (USA, N=541), Poland (N=1,651), and London (UK, N=217).
• Analysis: Networks were constructed as regularized partial correlation graphs using the EBICglasso method with Spearman correlations. Node centrality (expected influence, strength, closeness, betweenness) and community structure (via the walktrap algorithm) were analyzed.

LLM Data:

• Models: Six variants were tested: Gemma3-12B, Gemma3-27B, their quantization-aware trained (QAT) counterparts, Llama3.3-70B, and Llama4-16x17B (272B parameters).
• Simulation: Each model generated 1,000 simulations by crossing 200 synthetic personas with five imagination ability conditions (aphantasia to hyperphantasia).
• Conditions: Two conversational modes were tested:
  • Independent: Items rated without access to prior responses.
  • Cumulative: Items rated with conversational history maintained (simulating episodic memory).
• Population Matching: LLM simulations were downsampled to N=600 per model to match the distribution of total vividness scores found in human populations, ensuring differences were not due to overall ability levels but representational organization.

Key Results

1. Human Network Consistency:

   • Centrality: Human networks exhibited robust cross-population alignment for local centrality measures. Expected influence and strength correlations ranged from $r = 0.31$ to $0.93$ across distinct samples (Florida, Poland, London). Closeness centrality also showed significant alignment.
   • Clustering: Community detection recovered clusters aligned with questionnaire structures. PSIQ networks showed near-perfect alignment with sensory modalities (Adjusted Rand Index, ARI = 0.87–1.0). VVIQ-2 networks showed moderate alignment with scene contexts (ARI = 0.27–0.40).
   • Instability: Betweenness centrality was unstable across all populations ($r \approx 0.08$), consistent with its sensitivity to individual experiential history.

2. LLM Failure to Replicate:

   • Centrality: LLM networks failed to replicate human structure. LLM-human centrality correlations were weak and largely non-significant after correction (median $r = 0.13$ for VVIQ-2 and $0.09$ for PSIQ).
   • Clustering: Most LLM configurations produced degenerate single-cluster topologies (median ARI = 0). Even with cumulative conversational memory, LLMs failed to achieve the clustering alignment seen in humans.
   • Robustness: This failure persisted across model architectures, parameter scales (12B to 272B), and conversational conditions.

Key Contributions

• Network Psychometrics of Imagination: The study demonstrates that imagination can be modeled as a network where item-level covariation reflects shared memory organization, distinct from latent factor models.
• Quantitative Benchmarks for AI: The paper establishes concrete thresholds for evaluating generative AI world models: a minimum cross-population centrality correlation of $r = 0.31$ and a minimum VVIQ-2 clustering alignment of ARI = 0.27.
• Differentiation of Representation vs. Language: The findings distinguish between the ability to generate plausible vividness language and the ability to organize that language according to the relational structure of embodied memory.

Significance and Claims
The authors posit that the observed human network structure reflects memory organization accumulated through embodied experience, a representational structure that linguistic training alone does not reproduce.

• Cognitive Science: The cross-population replication supports the view that the "rendering" component of human mental simulation (the phenomenological detail of imagery) is organized by shared environmental and sensory regularities, binding the human "physics engine" (simulation) to a "graphics engine" (rendering).
• AI Research: The results suggest that current LLMs, regardless of scale or conversational memory, lack the specific structural organization of human imagination. The failure is specific to phenomenologically-grounded knowledge organization rather than general semantic representation, as LLMs succeed in tasks where human similarity can be approximated from language statistics (e.g., color similarity) but fail in tasks requiring the relational structure of episodic memory.
• Future Direction: The paper concludes that producing plausible vividness language and organizing it according to human memory structure are separable capacities. Future work must test multimodal architectures to determine if richer sensorimotor grounding can close this gap.

The study maintains modesty regarding its scope, noting that human samples were Western and educated, and that objective measures (e.g., fMRI) are needed to confirm that network structure reflects phenomenological organization rather than scale-use differences. It explicitly states that the failure of LLMs is a qualitative difference in representational organization, not merely a difference of degree.