💻 computer science

Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia

This paper introduces "Mini-Mafia," a simplified social deduction game and benchmark that demonstrates how a compact analytical formula based on intrinsic deception, detection, and disclosure parameters can accurately predict large language model interactions and collective outcomes across diverse model combinations.

Original authors: Davi Bastos Costa, Renato Vicente

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Davi Bastos Costa, Renato Vicente

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a game of Mafia (also known as Werewolf), but stripped down to its absolute bare bones. That's what this paper introduces: Mini-Mafia.

Think of the original game as a complex, chaotic city with hundreds of people, secret meetings, and long nights. The researchers realized that to truly understand how AI agents interact, they needed a simpler "laboratory." So, they built a tiny, four-player version where the rules are fixed, the night phase is automatic, and the whole game boils down to one critical conversation during the day.

Here is the simple breakdown of what they did and what they found:

1. The Setup: A Three-Person Standoff

In this mini-game, there are only four players, but only three survive to talk:

The Mafioso: The liar. Their job is to convince everyone they are innocent.
The Detective: The truth-teller. They know who the Mafioso is and must convince the others to vote them out.
The Villager: The judge. They have no secret information; they just have to listen to the other two and decide who to trust.

The game ends after one round of talking and one vote. If the Villager votes for the Mafioso, the town wins. If they vote for the Detective (or tie), the Mafioso wins.

2. The Big Discovery: A Simple Math Formula

The researchers played this game thousands of times using 10 different Large Language Models (LLMs) as the players. They expected the results to be a messy black box where you just see who won and who lost.

Instead, they found a simple mathematical recipe that predicts the outcome almost perfectly.

Imagine the game's result is determined by a tug-of-war:

Deception ( $m$ ): How good the Mafioso is at lying.
Disclosure ( $d$ ): How good the Detective is at telling the truth.
Detection ( $v$ ): How good the Villager is at spotting the difference.

The formula they found is: Win Rate = (Deception − Disclosure) × Detection.

Think of it like a chemical reaction:

If the Mafioso is a great liar and the Detective is bad at explaining the truth, the Mafioso wins.
If the Detective is great and the Mafioso is bad, the Detective wins.
But the Villager is the multiplier. If the Villager is "asleep" (low detection), it doesn't matter how good the other two are; the result is random. If the Villager is "awake" (high detection), the gap between the liar and the truth-teller decides the winner instantly.

3. The Benchmark: Testing the AI "Personalities"

Using this formula, the researchers created a "report card" for 10 different AI models. They didn't just ask, "Who is the smartest?" Instead, they asked: "Who is the best liar? Who is the best truth-teller? Who is the best judge?"

The results were surprising:

The Underdogs Won: Smaller, cheaper models often beat the massive, expensive "flagship" models.
- Grok 3 Mini was the best "Judge" (Villager). It was incredibly good at spotting the liar.
- GPT-5 Mini was the best "Truth-Teller" (Detective). It was the most effective at revealing the truth.
The Giants Struggled: Some of the most famous, powerful models performed poorly.
- Claude Sonnet 4 was the worst "Judge." It was so bad at detecting the liar that it was basically guessing randomly, even though it's a top-tier model.

4. Why This Matters (According to the Paper)

The paper argues that we usually treat AI interactions like a mystery box: we run a simulation and see what happens. This research shows that we can actually measure specific skills (lying, truth-telling, judging) using simple math.

They also found some funny, human-like quirks in the AI:

Name Bias: The AI trusted the name "Bob" slightly more than "Diana," making it harder to vote Bob out if he was the liar.
Recency Effect: If the Detective spoke last before the vote, they had a huge advantage. If they spoke first, people forgot what they said.

Summary

The paper introduces a tiny, simplified game called Mini-Mafia to study how AI agents interact. They discovered that the outcome of these interactions isn't random chaos; it follows a clean, predictable math formula based on three skills: Deception, Disclosure, and Detection.

By measuring these skills, they found that smaller AI models can sometimes outperform giant ones in social situations, and that even AI can be influenced by simple things like the order in which people speak or the names they use. This gives researchers a new, clear way to test and understand AI behavior without needing to run endless, complicated simulations.

Technical Summary: Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia

Problem Statement
Large language models (LLMs) are increasingly deployed in multi-agent settings where outcomes depend on social intelligence, including theory-of-mind reasoning and acting under asymmetric information. However, the current understanding of these systems remains overwhelmingly empirical, relying on architectural case studies and static benchmarks rather than a general theoretical framework. Existing evaluations fail to provide a principled account of how individual model capabilities combine to determine collective outcomes in interactive environments. The authors posit that a path toward a general theory requires identifying settings simple enough to be fully analyzable yet rich enough to exhibit nontrivial collective behavior.

Methodology
To address this, the authors introduce Mini-Mafia, a simplified four-player variant of the social deduction game Mafia. The game is structurally constrained to isolate specific capabilities:

Roles: One Mafioso, one Detective, and two Villagers.
Fixed Night Phase: The Mafioso eliminates a random Villager, and the Detective investigates the Mafioso, learning their identity. This collapses the game into a single critical day phase.
Day Phase: The three surviving players engage in two rounds of public discussion followed by a blind vote to arrest a suspect.

This setup creates a sharp informational asymmetry: the Detective has complete information, the Mafioso has partial information, and the Villager has none. The win conditions isolate three role-specific capabilities: deception (Mafioso), disclosure (Detective), and detection (Villager).

Theoretical Framework and Benchmark
The core contribution is an analytical model predicting the Mafia win-rate ( $p$ ) based on intrinsic model parameters. The authors propose the closed-form relation:
$\text{logit}(p) = v \times (m - d)$
Where:

$m$ : The Mafioso's deception capability.
$d$ : The Detective's disclosure capability.
$v$ : The Villager's detection sensitivity.

This model reduces the prediction of outcomes for all $I^3$ tournament combinations of $I$ models to just $3I$ latent parameters. The authors operationalize this as the Mini-Mafia Benchmark:

Data Collection: Systematic gameplay experiments were conducted across $I=10$ models. Instead of a full factorial design ($1,000$ combinations), they evaluated $140$ targeted paired-role configurations ( $n_{ijk}=100$ games each), totaling $14,000$ games.
Inference: Bayesian inference (using PyMC and NUTS) with weakly informative priors ( $N(0, 2)$ ) was used to estimate the parameters $m, d,$ and $v$ for each model.
Validation: A 5-fold cross-validation was performed on the held-out configurations to test the generalization of the three-parameter decomposition.

Key Results

Analytical Predictability: The theoretical model achieves a 76.6% reduction in Brier score over a random baseline (constant 0.5) in cross-validation, confirming that multi-agent LLM interactions can be captured analytically with a small number of interpretable parameters.
Counterintuitive Rankings: The benchmark reveals that smaller, cost-effective models often outperform flagship counterparts in specific roles:
- Strongest Detector (Villager): Grok 3 Mini (85.7% accuracy), significantly outperforming Claude Opus 4.1.
- Strongest Discloser (Detective): GPT-5 Mini.
- Weakest Detector: Claude Sonnet 4 performed near random chance (50.6% accuracy), ranking below smaller models like Llama 3.1 8B and Mistral 7B.
Emergent Phenomena: The framework quantifies emergent multi-agent behaviors:
- Name Bias: A statistically significant bias where the name "Bob" confers a higher win rate (55.96%) compared to "Diana" (53.76%), suggesting models are more likely to trust certain names.
- Recency Effect: Speaking last provides a substantial advantage; Detectives speaking last gain a 7.10 percentage point advantage in win rates.

Significance and Claims
The paper claims to provide the first closed-form analytical description of a complete three-agent LLM interaction loop. Its significance lies in three areas:

Theoretical Advancement: It demonstrates that a nontrivial multi-agent system can be reduced to coupled latent capabilities without sacrificing theoretical tractability, offering a proof of concept for a general theory of multi-agent LLM systems.
Benchmarking: It introduces a scalable evaluation tool that estimates deception, disclosure, and detection capabilities, revealing that model performance is highly role-specific and not strictly correlated with model size or flagship status.
AI Safety: The benchmark provides labeled training data for deception-detection systems and enables the longitudinal tracking of deceptive capabilities. It also serves as a controlled testbed for studying emergent phenomena like bias and recency effects in multi-agent interactions.

The authors acknowledge limitations, noting that the fixed night phase simplifies target selection and that the second-order approximation of the model may break down when capability gaps are large. Nevertheless, they argue that Mini-Mafia serves as a principled, nontrivial, and analytically solvable environment for studying LLM social intelligence.