Tacit Coordination of Large Language Models

Technical Summary: Tacit Coordination of Large Language Models

Problem Statement
Large Language Models (LLMs) are increasingly deployed in multi-agent settings requiring coordination without explicit communication, ranging from human–AI interaction to safety-critical scenarios like search and rescue. A core challenge in such settings is tacit coordination: how agents converge on a preferred outcome when choices must be made independently, without binding commitments or communication. While humans routinely solve this by relying on focal points—salient solutions that stand out due to shared cultural or perceptual cues (Schelling, 1980)—it remains unclear whether LLMs possess similar capabilities. The paper investigates how, when, and why focal points emerge in LLMs, comparing their behavior against human baselines across cooperative and competitive games. It specifically addresses the risk that LLM salience may diverge from human salience, leading to coordination failures in heterogeneous human–AI systems.

Methodology
The authors employ a theoretical framework grounded in Schelling’s focal points and game theory to evaluate tacit coordination. They define a salience function to quantify how focal an equilibrium is for a player and use the Coordination Index (CI) and Normalised Coordination Index (NCI) to measure the concentration of strategy choices in cooperative games.

The evaluation spans three distinct experimental settings:

1. Amsterdam and Nottingham Coordination Games: Based on human experiments by Bardsley et al. (2010), these are purely cooperative, symmetric games where participants choose from multiple answers. The study evaluates over 20 open- and closed-source LLMs (including Llama-3 variants, Qwen-2/2.5, and GPT-oss) against human baselines under three instructions: “pick,” “guess,” and “coordinate.”
2. The Bargaining Table: A semi-competitive game adapted from Mizrahi et al. (2020), where two players assign disks to themselves or each other. Coordination yields payoff; disagreement yields a penalty. This tests coordination under payoff asymmetry and competition.
3. Search & Rescue (SAR) Missions: A spatial testbed using 65 missing-hiker incidents from Hashimoto et al. (2022). LLMs predict a single coordinate for a missing hiker based on terrain maps. The study classifies incidents as “focal” (where the find location is near a salient terrain cue) or non-focal to assess the impact of spatial salience.

The authors also test three learning-free prompting techniques to improve coordination: “saliency” (approximating Theory of Mind), “all-features” (reasoning over focality principles), and “culture” (prioritizing culturally relevant options).

Key Contributions

• First Large-Scale Evaluation: The paper presents the first comprehensive assessment of LLM tacit collaboration, comparing more than 20 models against human data across cooperative, competitive, and realistic spatial scenarios.
• Analysis of Salience Mechanisms: It identifies specific heuristics LLMs use for coordination, revealing a bias toward structural cues (e.g., centrality, option order) rather than cultural or contextual cues.
• Prompting Strategies: The study evaluates and identifies simple, learning-free prompting techniques that can align LLM behavior with human salience, particularly through cultural prompting.
• Reasoning vs. Coordination: The paper investigates the role of Chain-of-Thought (CoT) reasoning, finding that increased reasoning depth does not necessarily improve tacit coordination and may even degrade performance.

Results

• Cooperative Games (Amsterdam/Nottingham): LLMs generally outperform humans in the “pick” condition but narrow the gap in “guess” and “coordinate” conditions. While LLMs remain competitive in the Amsterdam dataset, they significantly underperform humans in the Nottingham dataset. Analysis reveals that LLMs rely heavily on structural regularities and centrality, whereas humans leverage cultural and contextual salience. The “culture” prompt substantially improves LLM performance in Nottingham, matching or surpassing human baselines, whereas “saliency” and “all-features” prompts have little effect.
• Reasoning Depth: Contrary to the hypothesis that more reasoning aids coordination, increasing the reasoning level in GPT-oss models did not improve NCI and sometimes degraded performance. Log analysis indicates that elaborate rationales often collapse into arbitrary positional heuristics (e.g., selecting the first option).
• Bargaining Table: LLMs behave similarly to cooperative agents, achieving high social welfare and minimizing contested assignments. Model scaling (e.g., GPT-oss-120B vs. 20B) improves payoff and coordination, but increased reasoning levels do not.
• Search & Rescue: Behavior-aware prompting (specifically variant S3) improves prediction accuracy only in incidents classified as “focal” (where the endpoint is tied to a salient terrain cue). For non-focal incidents or the full dataset, prompting does not reliably improve performance. This suggests LLM coordination succeeds when model salience aligns with the structural salience of the environment.

Significance and Conclusions
The paper concludes that LLMs exhibit remarkable abilities to coordinate tacitly, often matching or outperforming humans in structurally clear tasks. However, they consistently fail in tasks requiring numerical common sense or culturally nuanced notions of salience. The findings caution against assuming that LLMs share the same cultural and perceptual substrate as humans. Instead, LLM coordination is best understood as implicit social and spatial modeling driven by structural heuristics.

The authors emphasize that for mixed human–AI systems, the central challenge is not merely enhancing reasoning capabilities but aligning the salience of LLMs with human expectations. They highlight that while cultural prompting can mitigate coordination failures, it also reveals that LLM salience can be steered, raising considerations for how such mechanisms might be used in deployment. The work underscores the need for caution in deploying LLMs in coordination settings where communication is sparse, as divergent notions of salience can lead to systemic failures.