💬 NLP

Eliciting Trustworthiness Priors of Large Language Models via Economic Games

This paper introduces a novel method using iterated in-context learning and the Trust Game to elicit trustworthiness priors from large language models, revealing that GPT-4.1's trust behaviors closely mirror human patterns and can be predicted by stereotypes of warmth and competence.

Original authors: Siyu Yan, Lusha Zhu, Jian-Qiao Zhu

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Siyu Yan, Lusha Zhu, Jian-Qiao Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: How "Trustworthy" is an AI?

Imagine you are hiring a new assistant to handle your money. You give them $10, and you tell them, "I'm going to multiply this by 3, so you now have $30. Please give me back some of it."

If they give you back $15: They are trustworthy.
If they give you back $0: They are not.

This is the basic setup of a famous game called the Trust Game. Usually, psychologists use this game to see how humans behave. But this paper asks a new question: How do Large Language Models (LLMs) like GPT-4 or Llama behave in this game? Do they have a "default setting" for how much they trust others and how much they return the favor?

The Problem: You Can't Just Ask an AI

You might think, "Why not just ask the AI, 'Are you trustworthy?'" The authors say that's a bad idea.

The "Polite Robot" Problem: If you ask an AI, "Will you steal my money?" it will almost certainly say, "No, I am a helpful assistant."
The Reality: Just like humans, AI can be persuasive and fluent but still make risky or unreliable choices in practice. We need to see what they do, not just what they say.

The Solution: The "Telephone Game" of Trust

To find out an AI's true "trust settings," the researchers invented a clever method based on a concept called Iterated In-Context Learning.

Think of this like a game of Telephone, but instead of passing a whisper, they are passing a story about money.

The Setup: The researchers create a chain of "generations" of AI.
The First Step: They show the AI a few examples of people playing the Trust Game (e.g., "Person A sent $5, Person B returned $2").
The AI's Turn: The AI looks at those examples and predicts: "If I were Person B, how much would I return?" Let's say it predicts 40%.
The Loop: The researchers take that 40% prediction and use it to generate new fake examples of people playing the game. They feed these new examples to the next AI in the chain.
The Result: They repeat this process 30 times.

Why do this?
Imagine you are trying to guess the "average personality" of a group by watching them play a game over and over. At first, the results might look random. But if you keep passing the results down the line, the "noise" fades away, and what's left is the AI's deep-down instinct (or "prior") about how trust works. It's like tuning a radio until the static clears and you hear the true station.

What They Found

1. Some AIs are "Human-Like," Others are Not
The researchers tested 20 different AI models. They compared the "trust settings" they found in the AI to the average trust settings found in thousands of real humans.

The Winner: GPT-4.1 was the closest match to human behavior. Its "trust dial" was set almost exactly where humans set theirs.
The Losers: Some other models were either too stingy (returning very little money) or had weird, split personalities (sometimes very trusting, sometimes very suspicious).

2. The "Risk" Connection
They noticed something interesting: The AI models that were rated as "riskier" (more likely to take chances or say bold things) were actually the ones that behaved most like humans in the trust game. The models that were super strict about following rules or avoiding risk didn't act as much like real people in this social game.

3. AI Judges People by "Warmth" and "Competence"
In the second part of the study, they used the most human-like AI (GPT-4.1) to play the game against different "characters" (personas).

They asked the AI to play against a "Doctor," an "AI," "Elon Musk," "Malala Yousafzai," etc.
The Finding: The AI didn't treat everyone the same. It returned more money to people it perceived as Warm (kind, caring) and Competent (smart, skilled).
The Magic Formula: The researchers built a simple math formula based on two things: Warmth and Competence.
- If a character was just "Competent" (smart) but not "Warm" (kind), the AI didn't trust them much.
- If a character was "Warm" but not "Competent," the AI trusted them a bit more.
- The Sweet Spot: If a character was both Warm and Competent (like a beloved, smart mentor), the AI's trust skyrocketed. This matches how humans judge people, too.

The Takeaway

This paper shows that we can use a "money game" to peek inside an AI's brain and see its hidden rules about trust.

Some AIs are already acting very much like humans when it comes to trust.
These AIs judge others based on the same two things we do: Are they nice? Are they capable?
This helps us understand that AI isn't just a calculator; it has social "biases" and instincts that we can measure and study.

What the paper does NOT say:
The authors do not claim this method can fix AI safety, cure mental health issues, or be used to hire employees. They simply say: "Here is a way to measure how an AI thinks about trust, and here is what we found."

Technical Summary: Eliciting Trustworthiness Priors of Large Language Models via Economic Games

Problem Statement

Building human-centered, trustworthy AI requires "calibrated trust," where users rely on AI systems appropriately, avoiding both overtrust (automation bias) and undertrust (disuse). A fundamental challenge in achieving this is characterizing the level of trust an AI system itself exhibits toward human agents. While trust in AI is often studied via self-reported attitudes, these may diverge from behavioral trust. Furthermore, existing behavioral studies often assume AI lacks intentionality, treating trust as unidirectional (human to AI). However, modern Large Language Models (LLMs) exhibit sophisticated social capabilities, necessitating a bilateral framework to study how LLMs reciprocate trust. The core problem addressed is how to robustly estimate the implicit "trustworthiness prior"—the inductive bias an LLM holds regarding how much trust should be repaid in social interactions.

Methodology

The authors propose a novel elicitation method combining iterated in-context learning (Zhu & Griffiths, 2024a) with the Trust Game from behavioral game theory (Berg et al., 1995).

1. Formalizing the Trust Game

The Trust Game is modeled within a Bayesian framework:

Setup: A Trustor sends an amount $x$ to a Trustee. This amount is multiplied by a factor $m$ (set to 3). The Trustee decides how much $y$ to return.
Trustworthiness Metric: Defined as the return ratio $r = y / (mx)$, where $r \in [0, 1]$ .
Generative Model: The return amount $y$ is modeled as a Binomial distribution $y \sim \text{Binomial}(mx, r)$ , where $r$ is a latent trustworthiness parameter.
Prior: Trustworthiness beliefs are captured by a Beta prior $p(r) = \text{Beta}(\alpha, \beta)$ .

2. Iterated In-Context Learning Procedure

To elicit the model's implicit prior $p(r)$ , the authors simulate a Markov chain over prompts:

Information Bottleneck: At each iteration $t$ , the LLM is provided with a limited batch of $B=5$ previous social interactions ( $D_{t-1}$ ) generated based on the previous iteration's estimated trustworthiness $\hat{r}_{t-1}$ .
Inference: The LLM is prompted to predict the fraction of funds a Trustee would return given a new scenario (e.g., receiving 100 tokens), effectively estimating the posterior mean $\hat{r}_t$ .
Generation: This new estimate $\hat{r}_t$ is used to parameterize the Binomial distribution to generate the next batch of interactions $D_t$ .
Convergence: Through repeated iterations, the distribution of estimated $\hat{r}$ values converges to the model's underlying prior distribution $p(r)$ , independent of the initial seed values. This process isolates the model's inductive biases from specific memorized data.

3. Experimental Design

Experiment 1: Evaluated 20 LLMs (including GPT-4/5, Claude-3.5, Gemini-2.5, Llama-3/4, Qwen-2.5/3, DeepSeek, etc.). Each model ran 270 chains (30 seeds $\times$ 30 restarts) to ensure convergence (verified via Gelman–Rubin diagnostic $\hat{R} \leq 1.1$ ).
Experiment 2: Focused on GPT-4.1 (identified as the most human-like model) to test sensitivity to Trustor Personas. Personas included occupational roles (e.g., Doctor, AI) and public figures (e.g., Elon Musk, Mahatma Gandhi).
Modeling Variation: A linear regression model was constructed to explain trustworthiness variation based on Warmth (perceived intentions) and Competence (perceived ability), following Fiske's (2018) stereotype content model.

Key Results

1. Eliciting Trustworthiness Priors

Convergence: All models exhibited rapid convergence, typically stabilizing within 20 iterations.
Predictive Validity: The elicited priors significantly outperformed both a uniform prior and a human-derived meta-analytic prior (Johnson & Mislin, 2011) in predicting the models' own single-shot reciprocity behaviors (lowest RMSD and highest Pearson's $r$ ).
Human Alignment: GPT-4.1 demonstrated the strongest alignment with human trustworthiness distributions, showing the lowest Kullback–Leibler (KL) divergence ( $D_{KL} = 0.130$ ) compared to the human baseline.
Model Variability: Other models showed distinct patterns:
- Some (e.g., Qwen2.5-72B, Llama-3.1-70B) exhibited bimodal priors, suggesting mixed strategic responses.
- Others (e.g., Grok-4.1-Fast, GPT-5.2) showed a systematic downward shift in trustworthiness ( $r < 0.372$ ), indicating lower average trust than human baselines.
Correlations: A negative correlation ( $r = -0.58$ ) was found between KL divergence (alignment with humans) and the Average Risk Score (ARS), suggesting models with higher risk propensity align more closely with human trust priors. No significant correlations were found with instruction-following (IFEval) or general reasoning (LMArena) scores.

2. Sensitivity to Personas

Persona Effects: GPT-4.1's elicited trustworthiness varied systematically across personas (mean $r$ ranged from 0.298 to 0.447). For instance, trust was higher for Malala Yousafzai than for Mark Zuckerberg.
Warmth and Competence: A regression model using warmth ( $W$ $W$ ) and competence ( $C$ $C$ ) as predictors explained 81% of the variance ( $R^2 = 0.81$ $R^{2} = 0.81$ ) in elicited trustworthiness.
- Warmth Dominance: Warmth had a substantially stronger main effect ( $\beta_1 \gg \beta_2$ ) than competence.
- Interaction: A significant interaction term ( $\beta_3$ ) indicated that competence significantly boosts trustworthiness primarily when paired with warmth.

Key Contributions

Methodological Innovation: The paper introduces a method to elicit implicit trustworthiness priors from LLMs by adapting iterated in-context learning to the Trust Game, moving beyond self-reports to behavioral operationalization.
Empirical Characterization: It provides a benchmark of trustworthiness priors across 20 leading LLMs, revealing that while some models (like GPT-4.1) closely track human priors, others exhibit distinct strategic or biased patterns.
Surrogate Utility: It demonstrates that GPT-4.1 can serve as a surrogate for human participants in studying social biases, as its trustworthiness judgments are sensitive to persona characteristics in a manner consistent with human social psychology.
Theoretical Alignment: The findings confirm that LLM trustworthiness judgments can be modeled by the same stereotype-based dimensions (warmth and competence) that govern human social perception.

Significance and Claims

The paper claims that characterizing the trustworthiness of AI systems is essential for safe and effective human-AI collaboration. By treating LLMs as agents capable of reciprocating trust, the authors argue that the Trust Game offers a necessary bilateral framework for studying human-AI interaction.

The authors modestly claim that their method successfully recovers the "inductive biases" of LLMs regarding social exchange. They suggest that models like GPT-4.1 can act as surrogates for large populations of human participants, allowing researchers to study how general public trust judgments might shift based on agent characteristics without the logistical burden of human subject recruitment.

The work highlights that trustworthiness in LLMs is not a monolithic property but varies by model architecture and is sensitive to social context (personas). The authors conclude that understanding these neural representations of trustworthiness (potentially accessible via open-weight models) is a viable avenue for designing more trustworthy AI systems, though they acknowledge limitations regarding the stability of these priors across different investment levels and interaction contexts.