Free-form Association Tasks Reveal Stereotype Hallucination in Large Language Models

Technical Summary: Free-form Association Tasks Reveal Stereotype Hallucination in Large Language Models

Problem Statement
While recent literature suggests Large Language Models (LLMs) can predict human stereotypical judgments, it remains unclear whether LLMs emulate the cognitive processes underlying human stereotypes or merely retrieve learned associations to solve prediction tasks. Prior research has largely relied on controlled judgment tasks (e.g., multiple-choice surveys) or contexts constrained by entrenched, predictable group biases (e.g., partisan responses to news). These settings may allow LLMs to succeed by retrieving statistical correlations from training data rather than modeling genuine cognitive mechanisms. A critical gap exists in understanding how LLMs perform in novel, unconstrained interpretative environments where cultural meanings are not pre-established. Specifically, it is unknown whether LLMs can replicate the structure of human stereotypes when interpreting free-form stimuli that lack inherent semantic meaning.

Methodology
The authors employ a comparative experimental design involving human participants and two multimodal models (GPT-4o mini and Llama-3.2-11B-Vision-Instruct) across five social domains: gender, partisanship, personality, urbanicity, and lifestyle (routine).

1. Stimuli: The study utilizes 12 ambiguous visual stimuli (4 Rorschach inkblots and 8 abstract art pieces) deliberately chosen for their lack of pre-established cultural meanings, explicit symbols, or aesthetic neutrality.
2. Experimental Conditions:
   • First-Order Tasks: Participants (humans) and models (using persona-based prompts, e.g., "You are a [identity]") provide direct personal interpretations. Responses consist of 3 object associations, 3 concept associations, and 1 emotion.
   • Second-Order Tasks: Participants and models predict how members of specific social groups interpret the same stimuli.
3. Data Collection:
   • Humans: ~300 participants per domain were recruited via Prolific. First-order participants interpreted 4 random stimuli; second-order participants predicted interpretations for both groups within a domain.
   • LLMs: Models were queried 80 times per domain/stimulus combination across four temperature settings (0.3, 0.7, 0.9, 1.0) to capture model variability.
   • Leave-One-Out (LOO) Conditioning: To test if homogenization stems from insufficient individualization, GPT-4o mini was conditioned on the actual associations of specific human participants for three images to predict their association for a fourth held-out image.
4. Analysis Metrics:
   • Within-Group Diversity: Measured using Shannon entropy of word associations.
   • Between-Group Separability: Measured by training LLM classifiers (GPT-4o mini and Llama) to predict group membership based solely on the linguistic patterns of the associations.
   • Semantic Space Analysis: Word embeddings (Sentence-BERT) were projected into 2D space via PCA to visualize the topology and clustering of responses.

Key Results
The study reveals fundamental qualitative differences between human and LLM cognitive processes regarding stereotypes:

• Within-Group Diversity (Entropy): Human first-order responses exhibit high heterogeneity (high entropy), with demographic information being a weak predictor of interpretation. In contrast, LLMs produce homogeneous first-order responses (low entropy) regardless of the social identity prompted. This homogenization persists even under LOO conditioning, where models fail to capture individual human variation despite being given specific behavioral data.
• Between-Group Separability (Classification Accuracy):
  • Humans: First-order human responses are nearly indistinguishable by group (classification accuracy near random chance, 46–60%). In second-order tasks, humans exhibit "stereotype exaggeration," moderately increasing separability (57–79%) by amplifying weak first-order tendencies while preserving diversity.
  • LLMs: While first-order LLM responses show low separability (except in personality domains), second-order LLM responses exhibit extreme separability (often >90%, reaching 99.9% in some domains). LLMs generate highly polarized, group-specific associations that are easily distinguished by classifiers, far exceeding human levels of distinctiveness.
• Semantic Topology: PCA analysis shows human responses form broad, overlapping, continuous clusters. LLM responses, particularly in second-order tasks, form tightly concentrated, fragmented, and spatially distinct clusters that are disconnected from the semantic regions occupied by human first-order responses.
• Stereotype Hallucination: The authors term the LLM phenomenon "stereotype hallucination." Unlike human stereotype exaggeration, which reifies ground-truth biases, LLMs generate stark second-order stereotypes that neither amplify existing first-order tendencies nor reflect actual human group differences. These hallucinated prototypes are semantically untethered from human ground truth.
• Robustness: These findings hold even when models are fine-tuned or conditioned on individual participant data, suggesting the issue is an architectural constraint rather than a prompting artifact.

Key Contributions

1. Identification of "Stereotype Hallucination": The paper introduces a distinct mechanism where LLMs generate exaggerated group stereotypes in novel contexts that are fundamentally disconnected from human interpretative reality, contrasting with human "stereotype exaggeration."
2. Methodological Innovation: By utilizing free-form, ambiguous stimuli (Rorschach/abstract art), the study isolates the cognitive processes of meaning construction from the retrieval of entrenched cultural scripts, providing a more rigorous test of LLMs as cognitive models.
3. Evidence of Architectural Limitation: The study demonstrates that LLMs fail to preserve individual heterogeneity and instead default to socially available prototypes, even when provided with specific individual behavioral data.

Significance
The paper argues that LLMs cannot serve as reliable "model organisms" for social science in novel, ambiguous, or culturally unsettled contexts. While LLMs may successfully predict human behavior in domains where meanings are stabilized and group boundaries are entrenched (e.g., partisan reactions to news), they fail to model the active social construction of meaning from diverse interpretations. The findings suggest that LLMs are limited in their ability to simulate human cognition when faced with high-entropy environments where human responses are idiosyncratic and weakly structured by demographics. Consequently, using LLMs to predict human behavior in emerging social phenomena or before meanings have stabilized may yield systematically biased and unrealistic results due to the models' tendency to hallucinate group prototypes rather than emulate human interpretative diversity.