💻 computer science

A Theory of Appropriateness That Accounts for Norms of Rationality

This paper proposes a society-first theory of normative appropriateness that models individuals as predictive pattern-completing agents analogous to LLMs, offering a parsimonious account of human norms and rationality as culturally contingent adherence to justification standards without relying on exogenous rational-choice assumptions.

Original authors: Joel Z. Leibo, Alexander Sasha Vezhnevets, Manfred Diaz, John P. Agapiou, William A. Cunningham, Peter Sunehag, Logan Cross, Raphael Koster, Stanley M. Bileschi, Minsuk Chang, Iyad Rahwan, Simon Osind

Published 2026-03-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Joel Z. Leibo, Alexander Sasha Vezhnevets, Manfred Diaz, John P. Agapiou, William A. Cunningham, Peter Sunehag, Logan Cross, Raphael Koster, Stanley M. Bileschi, Minsuk Chang, Iyad Rahwan, Simon Osindero, James A. Evans

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: We Are All "Pre-Loaded" Social Actors

Imagine you walk into a room. You don't stop to calculate the physics of the situation, weigh the pros and cons of every possible action, and then decide what to do. Instead, you just know what to do. You know to whisper in a library, to shake hands when introduced, and to stand a certain distance away from strangers.

This paper argues that human behavior isn't driven by a calculator inside our heads trying to maximize happiness or money. Instead, we are more like Large Language Models (LLMs)—the same technology that powers advanced AI chatbots.

We are "pre-trained" on human culture. From the moment we are born, we absorb the stories, rules, and habits of our society. When we face a situation, our brain doesn't run a complex math problem; it predicts the next word in a sentence. It asks itself: "What does a person like me do in a situation like this?" and then automatically generates the answer.

The Core Analogy: The "Autocomplete" Brain

Think of your brain as a super-smart text autocomplete feature (like the one on your phone), but instead of finishing your text messages, it finishes your actions.

The Input (Context): You see a friend crying.
The Pattern: Your brain instantly pulls up a pattern from your "training data" (your life experiences and culture).
The Output (Action): The most likely "completion" of that pattern is "hug them" or "offer a tissue." You do it without thinking.

You aren't calculating, "If I hug them, my social credit score goes up." You are just completing the pattern.

The Five "Weird" Things About Rules (and How This Theory Explains Them)

The authors say this "pattern completion" theory explains five strange things about human rules (norms) that other theories struggle with:

1. Context Dependence (The "Party vs. Funeral" Rule)

The Fact: Wearing a bright red dress is perfect at a wedding but terrible at a funeral.
The Old Theory: You must have a complex rulebook in your head that says "Red = Good at weddings, Red = Bad at funerals."
The New Theory: Your brain is like an AI that knows the whole story. It sees the context (the funeral) and predicts the next move. It doesn't need a rulebook; it just knows the "story" of a funeral usually involves dark colors. If you change the story (the context), the prediction changes instantly.

2. Arbitrariness (The "Drive on the Right" Rule)

The Fact: Why do we drive on the right side of the road? Why not the left? It doesn't matter physically; it just matters that we all agree.
The Old Theory: We must have calculated that driving on the right is the most efficient way to maximize traffic flow.
The New Theory: It's just a pattern we all learned. Once everyone is driving on the right, the "pattern completion" for a new driver is "drive on the right." If you tried to drive on the left, your brain would feel a huge "error signal" because it breaks the pattern. The rule is arbitrary, but the habit is strong.

3. Automaticity (The "Zombie" Mode)

The Fact: You can hold a conversation while thinking about your grocery list. You don't have to think about how to form sentences or how close to stand to someone.
The Old Theory: You have two systems: a fast "gut feeling" and a slow "thinking" system.
The New Theory: Most of the time, your brain is running on short, fast patterns (implicit norms). It's like a song you've heard a thousand times; you sing it without reading the sheet music. You only switch to "slow thinking" (deliberation) if something weird happens, like someone shouting in a library.

4. Dynamism (The "Tipping Point")

The Fact: Sometimes, a new rule spreads overnight (like wearing masks during a pandemic).
The Old Theory: People slowly calculate the benefits and switch.
The New Theory: It's a chain reaction. Once enough people start doing it, the "pattern" in everyone else's brain shifts. Suddenly, not wearing a mask feels "wrong" because the pattern has changed. It's like a wave; once it hits a certain size, it crashes over everyone.

5. Sanctioning (The "Gossip" Police)

The Fact: We follow rules because we fear being judged, gossiped about, or shunned.
The Old Theory: We are punished with a "negative reward" (like losing points).
The New Theory: Sanctions are social signals. When someone gets scolded for cutting in line, it's not just a punishment; it's a data update for everyone watching. It tells the observers: "Hey, update your pattern! Cutting in line is no longer a valid completion."

The Twist: What is "Rationality"?

The most shocking part of the paper is what it says about Rationality.

Usually, we think "being rational" means using logic to maximize your own benefit (like a robot calculating the best move).

This paper says: No.
"Being rational" is just following the local rules of your tribe.

If you are in a courtroom, "being rational" means using legal logic and evidence.
If you are in a religious community, "being rational" might mean following ancient traditions.
If you are in a scientific lab, "being rational" means running experiments.

There is no universal "Logic." There is only what your culture says is the right way to think.

The Analogy:
Imagine you are playing a game of Chess.

Old View: You are a genius calculating 10 moves ahead to win.
New View: You are just following the "Chess Pattern." You move the Knight in an 'L' shape because that's what Knights do in Chess. If you were playing Go, you would move differently. You aren't "rational" in a universal sense; you are just good at following the specific rules of the game you are currently playing.

Why This Matters

This theory changes how we see ourselves and society:

We aren't isolated robots: We are deeply shaped by the culture we grew up in. Our "preferences" (what we like) aren't fixed; they are learned patterns.
Change is possible: If we want to change a bad habit in society (like racism or pollution), we don't need to convince everyone to do math. We just need to change the pattern. If enough people start acting differently, the "autocomplete" for everyone else will update, and the new behavior will become the automatic norm.
Simplicity: We don't need to assume humans have complex, invisible "utility functions" or "rewards" in their heads. We just need to assume we are pattern-matching machines that learn from our environment.

In short: We are not calculating machines trying to win a game. We are storytellers who are constantly finishing the sentence: "What does a person like me do in a situation like this?" And the answer is usually: "Whatever everyone else is doing."

1. Problem Statement

The paper addresses a foundational question in the sciences of mind and society: How does social order emerge?

The Conflict: Existing theories are divided between "individual-first" approaches (Methodological Individualism, Rational Choice Theory) and "society-first" approaches (Structuralism).
- Rational Choice Theory (RCT) assumes agents are "blank slates" or under-socialized learners who maximize exogenous scalar rewards (utility) based on fixed preferences. It struggles to explain the context-dependence, arbitrariness, and automaticity of human norms without artificially baking in specific utility functions.
- Sociological approaches often lack a mechanistic cognitive model explaining how macro-level social structures translate into micro-level individual behavior.
The Gap: There is a need for a unified theory that explains five key empirical regularities (stylized facts) of human normative behavior without relying on exogenous utility maximization:
1. Context-dependence: Behavior varies by situation, role, and culture.
2. Arbitrariness: Norms often have no inherent functional necessity (e.g., driving on the left vs. right).
3. Automaticity: Norm adherence is often habitual and intuitive, not deliberative.
4. Dynamism: Norms can change rapidly or remain fixed for long periods.
5. Sanctioning: Norms are maintained through social approval/disapproval (gossip, punishment).

2. Methodology and Theoretical Framework

The authors propose a Society-First Theory where individuals are modeled as pre-trained actors with cognitive architectures analogous to Large Language Models (LLMs).

Core Mechanism: Predictive Pattern Completion

The Agent: Individuals are not blank slates but are initialized with substantial cultural knowledge (pre-trained on human language and cultural products).
Decision Logic: Behavior is generated via predictive pattern completion. The agent answers the question: "What does a person such as I do in a situation such as this?"
Formalism (LMAE):
- The environment is a Linguistic Multi-Actor Environment (LMAE).
- Agents operate within a Global Workspace ( $z_t$ ), a transient representation of observations, memories, and thoughts.
- Assemblies: The workspace contains sequences of symbols (thoughts, goals) predicted autoregressively.
- Memory ( $M_t$ ): Acts as an associative memory (analogous to the hippocampus) storing explicit rules and episodic memories.
- Pattern-Completion Network ( $p$ ): Analogous to the neocortex/LLM weights. It generates behavior based on the current context and consolidated implicit knowledge.
- Consolidation: Explicit memories (hippocampal) are gradually integrated into the weights of $p$ (neocortical) through repetition, turning explicit rules into implicit habits.

Distinction: Explicit vs. Implicit Norms

Explicit Norms: Articulable rules (laws, regulations). Stored in memory ( $M_t$ ) and retrieved into the global workspace for deliberative reasoning (analogous to "System 2" or long chains of thought).
Implicit Norms: Unarticulable, automatic behaviors (conversational distance). Embedded in the weights of $p$ (analogous to "System 1" or short pattern completions).
Sanctions: Defined as symbolic information added to the context that alters the probability distribution of the pattern-completion network, making norm-consistent behaviors more likely.

3. Key Contributions

A. Reconceptualizing Rationality

The paper argues that rationality is not a universal drive to maximize utility, but a culturally contingent epistemic norm.

"Rational" behavior (e.g., cost-benefit analysis) is simply a specific Decision Logic (Logic A) learned and enforced by specific communities (e.g., economists, engineers).
Most human behavior follows Decision Logic B (Appropriateness): "What does a person such as I do in a situation such as this?"
Rational choice is thus a learned script, not a fundamental cognitive architecture.

B. Explaining the Five Stylized Facts

The theory provides a unified mechanistic explanation for the five empirical regularities:

Context-dependence: The global workspace conditions behavior on the specific identity, role, and situational cues present in the prompt.
Arbitrariness: Norms emerge from conventions (patterns of precedent). Once a pattern is established, the "weight of precedent" makes it self-reinforcing, regardless of its material utility.
Automaticity: Implicit norms are consolidated into the weights of $p$ . They require only short autoregressive chains (fast processing), whereas deliberation requires long chains (slow processing) that are easily disrupted by cognitive load.
Dynamism: Norm change occurs via collective action. When a critical mass of agents shifts their behavior, the contextual input for all agents changes, triggering a cascade (tipping point) where the new pattern becomes the new high-probability completion.
Sanctioning: Sanctions are not external rewards/punishments but contextual signals that update the agent's memory and pattern-completion probabilities. They signal to the community what is "appropriate" for a specific role.

C. Biological and Cognitive Plausibility

The model aligns with Complementary Learning Systems (CLS) theory in neuroscience (Hippocampus for fast learning/explicit memory vs. Neocortex for slow consolidation/implicit knowledge).
It supports the Predictive Processing framework, where the brain acts as a generative model predicting sensory input and action.
It explains Motivated Reasoning: Deliberation (long chains of thought) does not always correct bias; it can be co-opted by implicit identity norms to generate sophisticated rationalizations for pre-existing beliefs.

D. Parsimony Argument

The authors apply Occam's Razor:

Rational Choice Theories require two assumptions: (1) a fundamental utility maximization mechanism, and (2) exogenous specification of utility functions/preferences.
Pattern Completion Theory requires only one mechanism: Predictive Pattern Completion.
Preferences and goals are endogenous, emerging from the interaction of the agent's pre-trained cultural weights and current context, rather than being fixed inputs.

4. Results and Implications

Unified Explanation: The theory successfully models how social order emerges from the interactions of culturally competent agents without needing to posit exogenous rewards.
Dual-Process Reinterpretation: It reframes the "System 1 vs. System 2" debate not as two separate neural systems, but as short vs. long autoregressive chains of thought within a single predictive network.
Norm Change: It provides a computational mechanism for how norms evolve, distinguishing between top-down institutional changes (explicit norms) and bottom-up cultural drift (implicit norms).
Simulation Potential: The framework enables the use of Generative Agent-Based Modeling (e.g., using LLMs as agents) to simulate complex social dynamics, counterfactual histories, and cultural evolution.

5. Significance

This paper represents a paradigm shift in social science and cognitive modeling:

From Individual-First to Society-First: It posits that the "self" and "rationality" are constituted by society, not the other way around. Individuals are initialized with culture; society is the aggregate of their interactions.
Demystifying Rationality: It challenges the notion of the "Homo Economicus," suggesting that what we call "rationality" is merely adherence to specific, culturally evolved epistemic norms.
Bridging Disciplines: It creates a rigorous bridge between Sociology (norms, social structure), Cognitive Science (predictive processing, memory consolidation), and AI (LLMs, pattern completion).
Methodological Advancement: It offers a path to simulate human social behavior using pre-trained LLMs, moving beyond the limitations of traditional game-theoretic models that rely on arbitrary utility functions.

In conclusion, the authors argue that predictive pattern completion is "all you need" to explain human normative behavior, offering a more parsimonious and culturally grounded alternative to rational-choice theories.