Do people expect different behavior from large language models acting on their behalf? Evidence from norm elicitations in two canonical economic games

Technical Summary: Do people expect different behavior from large language models acting on their behalf? Evidence from norm elicitations in two canonical economic games

Problem Statement
As large language models (LLMs) increasingly replace humans in tasks involving social interaction and resource allocation, a "human-machine cooperation gap" has emerged where people behave differently when interacting with AI compared to humans. While existing literature documents these behavioral differences, the underlying mechanisms remain unclear. This paper posits that the gap stems from a shift in social norms: people may apply different standards of "social appropriateness" to decisions made by LLMs acting on behalf of humans versus decisions made by humans directly. The study investigates whether delegating resource-allocation decisions to LLMs alters the perceived social norms governing those decisions, specifically examining whether LLMs are judged differently when acting as initiators (first-movers) versus enforcers (second-movers) of fairness.

Methodology
The authors conducted two pre-registered, incentivized experimental studies using representative samples from the United Kingdom (Experiment 1, N=1,153) and the United States (Experiment 2, N=1,505). The studies utilized the Krupka-Weber norm elicitation procedure, where participants rate the social appropriateness of specific behaviors on a four-point scale (from "Very Socially Inappropriate" to "Very Socially Appropriate"). Participants were incentivized to identify the modal response of the group to ensure the elicitation of shared normative beliefs.

• Experiment 1 (Dictator Game): Participants rated the social appropriateness of 11 possible resource splits (£0 to £10) in a Dictator Game. The between-subjects factor was the identity of the "Dictator": either a human or an LLM acting on behalf of a human. This design isolates the appropriateness of making an offer without the possibility of retaliation.
• Experiment 2 (Ultimatum Game): Participants rated the social appropriateness of rejecting offers in an Ultimatum Game. This experiment employed three between-subjects conditions:
  1. Human Proposer – Human Responder (Control).
  2. Human Proposer – LLM Responder (LLM as enforcer).
  3. LLM Proposer – Human Responder (LLM as initiator).
     This design allowed the authors to test norms regarding both the enforcement of fairness (rejecting offers) and the initiation of offers by machines.

Statistical analyses included clustered Wilcoxon rank-sum tests, participant-clustered bootstrap tests, and linear mixed-effects models to account for the repeated measures within participants.

Key Contributions and Results

1. Shift in Appropriateness, Not Norms (Experiment 1):

   • Modal Response: The study found no shift in the modal social norm. In both Human and LLM conditions, the 50-50 split was identified as the most socially appropriate action.
   • Downward Shift in Appropriateness: Despite the modal norm remaining constant, there was a significant downward shift in the overall social appropriateness ratings for LLMs. Offers made by LLMs were judged as less socially appropriate than identical offers made by humans.
   • Asymmetry: This effect was most pronounced for "low" splits (favoring the Dictator/LLM). For splits where the LLM kept more than half the resources, the appropriateness rating dropped significantly compared to the human condition. Conversely, for "high" splits (favoring the Recipient), the difference was negligible or non-existent.

2. Norms of Enforcement vs. Initiation (Experiment 2):

   • LLMs as Enforcers (Human Proposer/LLM Responder): When an LLM acted as the Responder (rejecting an offer made by a human), there was no significant difference in the social appropriateness of rejection compared to a human Responder. People do not view it as less appropriate for an LLM to enforce fairness norms on behalf of a human.
   • LLMs as Initiators (LLM Proposer/Human Responder): When an LLM acted as the Proposer, the pattern of rejection appropriateness shifted.
     • Low Offers: Rejecting a low (unfair) offer from an LLM was judged as equally appropriate as rejecting one from a human. The authors note this contrasts with some prior behavioral studies suggesting higher rejection rates for AI; here, the norm of rejection did not change.
     • High Offers: Unexpectedly, rejecting a generous offer (e.g., giving the Responder £8 and keeping £2) was judged as more socially appropriate when the offer came from an LLM than from a human. This suggests a norm that LLMs should not be "too generous" to the detriment of the human they represent.

Significance and Claims
The paper claims to identify a specific mechanism for human-AI interaction friction: a divergence in social appropriateness norms rather than a change in the definition of the "most appropriate" action. The findings suggest that:

• People apply different norms to machines acting as first-movers (initiators) versus second-movers (enforcers).
• When machines initiate economic transactions, they are held to a stricter standard of "not being ungenerous" but also a standard of "not being overly generous" (which would be unfair to the human principal).
• However, people do not object to machines acting as enforcers of fairness; delegating the role of rejecting unfair offers to an LLM is socially acceptable.

The authors conclude that these results imply that offers from machines are now viewed as having both a cognitive and an emotional component. While people may accept machines enforcing norms, they are sensitive to the emotional and moral implications of machines making unilateral decisions about resource distribution. The study cautions that policy and delegation strategies must account for these nuanced expectations, as mass delegation of decision-making roles to LLMs could lead to different social equilibria than currently anticipated. The authors explicitly limit their claims to the context of economic games and Western, developed nations, noting that external validity and cross-cultural replication are necessary future steps.