The role of System 1 and System 2 semantic memory structure in human and LLM biases

1. Problem Statement

Implicit biases in both humans and Large Language Models (LLMs) pose significant societal risks. While Dual Process Theory posits that biases arise from fast, associative System 1 thinking and are mitigated by slow, deliberative System 2 thinking, the underlying cognitive mechanisms remain poorly understood. Specifically, it is unclear whether the structural organization of knowledge (representational structure) or the processing dynamics drive these differences. Furthermore, it is unknown if LLMs, which are inherently associative (System 1-like), possess distinct "System 2-like" structural representations that regulate bias similarly to humans. This study aims to bridge this gap by modeling human and LLM cognition as multilayer semantic networks to compare how different forms of conceptual knowledge relate to implicit gender bias.

2. Methodology

A. Data Construction: Multilayer Semantic Networks

The authors constructed three parallel multilayer networks (one for Humans, one for Mistral-7b, and one for Llama3-8b). Each network consists of three distinct layers representing different types of conceptual knowledge:

1. Associative Layer (System 1 Proxy):

   • Source: Free word associations.
   • Human Data: Small World of Words (SWOW) dataset.
   • LLM Data: LLM World of Words (LWOW) dataset generated by prompting models with cue words.
   • Structure: Directed edges converted to undirected, weighted by response frequency. Represents implicit, statistical regularities.

2. Definitions Layer (System 2 Proxy - Feature-based):

   • Source: Dictionary definitions.
   • Human Data: WordNet definitions.
   • LLM Data: Generated by prompting models to identify word senses and provide definitions.
   • Structure: Unweighted, undirected edges connecting a word to words in its definition. Represents logical, necessary conditions of concepts.

3. Categorical Relations Layer (System 2 Proxy - Taxonomic):

   • Source: Taxonomic relations (synonyms, antonyms, hyponyms, hypernyms).
   • Human Data: WordNet relations.
   • LLM Data: Generated by prompting models for specific word senses.
   • Structure: Unweighted, undirected edges representing hierarchical and logical category constraints.

B. Analytical Frameworks

1. Structural Reducibility Analysis:

   • Goal: To determine if the three layers encode distinct structural information or if they are redundant.
   • Method: Used the multired Python package. Layers are represented by normalized Laplacian matrices. An information-theoretic distance measure (relative entropy) is used to iteratively merge layers.
   • Metric: If the optimal configuration keeps layers separate (maximizing relative entropy), the system is irreducible (distinct structures). If layers merge with minimal information loss, they are reducible (overlapping structures).

2. Implicit Bias Measurement (Spreading Activation):

   • Goal: To quantify gender stereotypes within each layer.
   • Method: Adapted the Implicit Association Test (IAT) logic using spreading activation (via the spreadr R library).
   • Process:
     • Primes: Gender identities (e.g., man, woman, mother, father).
     • Targets: 172 words across four categories: Traits, Home/Career, Art/Science, Professions.
     • Mechanism: Activate a prime node; activation spreads through the network edges over time steps. The final activation level of target nodes indicates association strength.
   • Metric: Effect sizes (Wilcoxon test) comparing the activation of stereotype-consistent pairs (e.g., man–doctor) vs. inconsistent pairs (e.g., woman–doctor). Positive values indicate bias; values near zero indicate neutrality.

3. Key Contributions

• Novel Representation: The first study to model System 1 and System 2 thinking as distinct, comparable multilayer semantic networks for both humans and LLMs.
• Structural vs. Functional Distinction: Moves beyond behavioral performance metrics to analyze the representational structure of knowledge, disentangling memory organization from processing dynamics.
• Comparative Framework: Establishes a shared methodology to directly compare the cognitive architectures of humans and LLMs regarding bias regulation.

4. Key Results

A. Structural Reducibility (Knowledge Organization)

• Humans: The multilayer network is irreducible. Merging any pair of layers (Associative, Definitions, Categorical) results in a significant loss of structural information. This confirms that human semantic memory maintains distinct, non-overlapping structures for associative, feature-based, and taxonomic knowledge.
• LLMs (Mistral & Llama3): The networks are partially reducible. The Associative and Categorical layers can be merged with minimal information loss, suggesting that LLMs do not maintain a distinct structural separation between free associations and categorical (taxonomic) knowledge.
  • Observation: LLMs often generate "associative" responses for categorical tasks (e.g., providing "table" as an antonym for "chair" rather than recognizing no antonym exists), leading to structural overlap.

B. Relationship Between Structure and Bias

• Humans: A consistent, systematic pattern exists.
  • The Associative layer exhibits the strongest gender bias.
  • The System 2 layers (Definitions and Categorical) show significantly reduced bias.
  • Implication: In humans, shifting from associative to deliberative/structured knowledge effectively regulates bias.
• LLMs: No consistent pattern is observed.
  • Bias levels fluctuate unpredictably across layers and topics.
  • There is no systematic reduction of bias in the Definitions or Categorical layers compared to the Associative layer.
  • Implication: The structural shift from associative to rule-based representations does not reliably regulate bias in LLMs, likely because their "rule-based" layers are structurally redundant with their associative layers.

C. Specific Anomalies

• Human Categorical Layer: Shows an asymmetry where male-related targets are strongly associated with the male gender, while female-related targets show negative (stereotype-inconsistent) associations. The authors attribute this to historical "male-defaults" in taxonomic hierarchies (e.g., man meaning "mankind" in WordNet).
• LLM Inconsistency: In Llama3, the Definitions layer showed male-consistent bias for professions but female-consistent bias for traits, indicating that bias in LLMs is tied to topic categories rather than a coherent semantic structure.

5. Significance and Conclusion

• Fundamental Cognitive Differences: The study provides evidence that LLMs lack the irreducible, distinct structural representations of taxonomic knowledge found in humans. LLMs appear to encode "categorical" knowledge in a way that is statistically indistinguishable from associative knowledge.
• Bias Regulation Mechanism: In humans, bias regulation is a structural property; accessing knowledge through deliberative (System 2) structures naturally dampens stereotypes. In LLMs, this mechanism is absent because their "System 2" layers are not structurally distinct from their "System 1" layers.
• Implications for AI: Simply adding structured data (like dictionaries or knowledge graphs) to LLMs may not be sufficient to regulate bias if the model's internal architecture cannot maintain distinct, non-overlapping representations for different types of knowledge. Future bias mitigation strategies must address the representational structure of the model, not just the training data or prompting strategies.

This work highlights that while LLMs mimic human behavior, their underlying cognitive architecture regarding conceptual knowledge and bias regulation is fundamentally different from human dual-process cognition.