🤖 AI

CoupVisor: Strategy Optimization by Round and Challenge Decision Support

This paper introduces CoupVisor, a unified decision-support system for the card game Coup that integrates belief tracking and simulation to optimize turn-based actions and challenge timing, demonstrating that policies trained with win-oriented rewards outperform both rule-based and short-term gain-focused baselines.

Original authors: Cris Huynh

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cris Huynh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of games, some contests are decided by the cards you hold, while others are won by the stories you tell. The card game Coup belongs to the latter category, a compact arena where every player hides two secret roles and must bluff, bluff, and bluff their way to victory. The core tension lies in a simple but dangerous question: when a player claims to be a powerful character, are they telling the truth, or are they lying to steal a turn? In a real game, players rely on gut feelings, table talk, and vague suspicions to decide whether to call out a liar. This instinctive approach leaves a gap; the evidence is all there in the public record of every move made, yet no one turns that record into a clear number to guide the next decision. This is where the field of artificial intelligence steps in, specifically the branch that teaches computers to make decisions when they cannot see the whole picture. By treating a game as a puzzle of hidden information, researchers can build systems that track probabilities, weigh risks, and learn from thousands of simulated matches to find strategies that human intuition might miss.

Enter CoupVisor, a new decision-support system designed to bridge the gap between the messy reality of a card table and the cold logic of a computer. Rather than playing the game itself, CoupVisor acts as an observer and advisor, watching the public record of every action, challenge, and reveal to build a mathematical picture of what every opponent is likely holding. The system does not guess; it calculates. It takes the known facts—who has claimed what, who has been challenged, and how many cards remain in the deck—and combines them with a running estimate of the likelihood that any specific player holds a specific role. This allows it to answer the two hardest questions a player faces on any given turn: what move should I make, and is it time to challenge an opponent's claim? The system is built on a single, unified framework that works whether a human is typing in moves, a computer is replaying a recorded game, or a simulation is running thousands of matches to train a learning algorithm.

The researchers behind CoupVisor discovered that the way a computer learns to play depends entirely on what it is told to value. They tested two very different approaches to teaching the system. In the first approach, the system was rewarded for small, immediate gains, such as collecting a few coins or keeping a card safe for the moment. Under these conditions, a method that simply copied the moves of past players performed best, while more complex learning methods struggled to improve. However, when the researchers changed the goal to focus solely on winning the game, the results flipped completely. The complex learning method, which had previously lagged behind, surged ahead of all other strategies, including the copying method and simple rule-based bots. This finding suggests that the choice of reward is more critical than the choice of algorithm; if you want a system to learn to win, you must reward it for winning, not just for doing well in the short term.

To make its advice useful, CoupVisor had to solve a specific problem that had tripped up previous attempts: the system was too eager to challenge claims right at the start of a game. Because no one had revealed any cards yet, the computer initially thought every claim was suspicious, leading it to recommend challenges that were statistically unjustified. The researchers fixed this by changing how the system calculated probability. Instead of looking at the chance that a player held a single card, the system began calculating the chance that a player held at least one copy of a role within their two-card hand. This small adjustment corrected the early-game bias, ensuring that the system only recommended a challenge when the evidence truly supported it. The system also learned to adjust its confidence based on the situation, becoming more cautious when a player had only one card left to lose and more aggressive when an opponent was close to winning.

The testing phase involved running five hundred simulated games, pitting the system against different types of opponents, from those who played honestly to those who bluffed constantly. The results showed that while the system's advice was stable in terms of overall game outcomes, its ability to correctly identify a bluff varied significantly depending on who it was playing against. When facing aggressive bluffers, the system became much better at spotting lies, but when facing honest players, it struggled to find false claims because they were rare. This sensitivity to the opponent's style highlights a key limitation: the system is not a universal oracle but a tool that must adapt to the table. The researchers also found that the system's biggest errors came from being too eager to challenge, often accusing honest players of lying when the math was simply too close to call.

Ultimately, CoupVisor demonstrates that building a helpful advisor for a game of hidden information requires more than just a powerful computer. It requires a careful balance between hard rules, such as the forced move a player must make when they have too many coins, and soft probabilities that estimate the truthfulness of a claim. The system separates these two worlds clearly, telling the user what is impossible by the rules and what is merely unlikely by the odds. By converting the chaotic flow of a card game into a structured stream of data, the researchers created a tool that can audit a player's decisions, explain why a challenge was or was not wise, and provide a clear path for learning. The work suggests that in any situation where decisions must be made under uncertainty, the most important factor is not the complexity of the tool, but the clarity of the goal it is trying to achieve.

Technical Summary: CoupVisor – Strategy Optimization by Round and Challenge Decision Support

1. Problem Definition

The paper addresses decision-making in Coup, a hidden-information card game characterized by imperfect information and adversarial bluffing. In Coup, players hold two hidden role cards but may claim any role to perform actions. The core challenge lies in two coupled decisions made without knowledge of opponents' hands:

Action Selection: What action to take on one's own turn.
Challenge Decision: Whether to demand proof (challenge) when an opponent claims a role.

The paper identifies a gap in existing play: while all game events (actions, blocks, challenges, reveals) are public and theoretically sufficient to constrain hidden states, players typically rely on instinct rather than formalizing this evidence. The goal of CoupVisor is to bridge this gap by converting public records into explicit probability distributions and actionable advice, serving as an observer and advisor rather than a game-playing bot.

2. Methodology

A. Unified Event Schema

The system is built around a single event schema that describes all public game events. This schema is consumed uniformly by:

Manual turn entry.
Replay of recorded games.
Synthetic game simulation.
Bayesian belief tracking.
Advisor recommendation logic.
Offline-trained learning policies.

This architecture ensures that recommendations produced during live play, replay, or training are generated by identical code, eliminating drift between evaluation and deployment.

B. Belief Modeling (Bayesian Update)

CoupVisor maintains a belief state $b_i(r)$ , representing the probability that player $i$ holds role $r$ given the public history $h_t$ .

Update Mechanism: The system uses Bayes' rule, updating priors with likelihood ratios based on observed events (e.g., claiming a role, blocking an action, surviving a challenge).
Likelihood Calibration: Ratios are calibrated against estimated honest claim rates (~~0.85) and bluff rates (~~0.15). For instance, surviving a challenge multiplies the probability of the claimed role by 10.0, while losing multiplies it by 0.05.
Deck Pressure: Probabilities are scaled by a factor $(c_r/3)^{0.7}$ , where $c_r$ is the number of unrevealed copies of role $r$ , ensuring roles with exhausted copies become unlikely.

C. From Per-Card to Per-Hand Probability

A critical methodological contribution is the conversion of per-card beliefs into per-hand probabilities. Since a player holds two cards, the probability that a claim is truthful ( $p_{truth}$ ) is not simply the per-card belief.

The system blends a hypergeometric probability (accounting for the finite deck and drawing without replacement) with an independent probability model.
Formula: $p_{truth} = 0.7 \cdot p_{hyp} + 0.3 \cdot p_{ind}$ .
Significance: This correction prevents the system from flagging the very first claim of a game as suspicious. A uniform per-card prior of 0.20 would incorrectly suggest a challenge; the per-hand calculation correctly raises the probability above the challenge threshold.

D. Decision Logic

Challenge Advisor: Uses a threshold rule ( $\theta$ ). A challenge is recommended if $p_{truth} < \theta$ . The threshold is context-dependent (e.g., 0.25 for standard claims, 0.35 for Assassinate, 0.99 for roles with no remaining copies) and adjusted by game state factors (number of surviving players, coin counts).
Action Advisor: Uses a two-ply expected-value tree to score legal actions, considering immediate coin/influence swings and likely opponent responses (blocks/challenges).
Risk Modes: The system offers Conservative, Aggressive, and Balanced modes, which shift the challenge threshold to align with user risk tolerance.

E. Learning Approaches

The paper evaluates several learning-based policies trained on simulator logs (Offline RL):

Behavior Cloning (BC): Supervised imitation of logged actions.
Deep Q-Network (DQN): Reinforcement learning maximizing expected return.
Baselines: Random policy, rule-based heuristics, and belief-based expected-value agents.

Two reward definitions were tested:

Shaped Reward: Credits intermediate gains (coins, influence retained) and action matching.
Win-Aligned Reward: Credits the policy only when its chosen action matches the action taken by the eventual winner of the logged game.

3. Key Contributions

Unified Pipeline: A single event schema that synchronizes manual play, replay, simulation, belief tracking, and policy training, ensuring consistency across all system components.
Per-Hand Probability Correction: A method to convert per-card beliefs into per-hand probabilities, resolving a systematic error where initial game claims were incorrectly flagged as bluffs.
Reward Alignment Finding: A demonstration that the choice of reward function, rather than the algorithm itself, dictates the performance hierarchy of learned policies.
Transparent Decision Support: An advisor that separates "uncertain belief" (probabilistic) from "hard rules" (legal constraints like forced Coup), providing auditable reasoning for recommendations.

4. Results

A. Advisor Performance

Stability: Outcome accuracy (whether the recommendation matched the better hindsight outcome) was stable at 0.5694 across seeds.
Sensitivity: Challenge F1 scores varied significantly based on opponent composition. Against "aggressive-heavy" opponents (who bluff more), the F1 was 0.5641. Against "honest-heavy" opponents, it dropped to 0.2831.
Error Analysis: The system exhibited a precision-recall imbalance, tending to challenge too eagerly. False positives (challenging a true claim) were dominant, particularly on "Steal" and "Tax" actions.

B. Learning Policy Benchmarks

The results inverted depending on the reward definition:

Under Shaped Reward: Behavior Cloning (BC) significantly outperformed DQN (Reward/episode: +1.04 vs -0.29). BC learned to imitate the average logged behavior, which included intermediate coin gains.
Under Win-Aligned Reward: DQN outperformed all baselines, including BC (Reward/episode: 6.74 vs 3.38). DQN successfully learned to prioritize the ultimate goal (winning) over intermediate proxies.
Conclusion: The algorithm choice (BC vs. DQN) did not determine the winner; the reward definition did.

C. Ablation Studies

Belief Mode: Comparing "Standard" vs. "Strict no-duplicate-hand" belief updates showed no statistically significant difference, suggesting the duplicate-hand constraint rarely binds at critical decision points.
Advisor On/Off: Turning the advisor off (never challenging) increased outcome accuracy to 0.8192 in the simulator. However, the author notes this is an artifact of the simulator where most claims are true; it does not prove challenging is harmful, but rather highlights the cost of false challenges in a low-bluff environment.

5. Significance and Claims

The paper claims three primary contributions:

For Players: It transforms intuition into stated probabilities with explicit reasons, separating uncertain beliefs from hard rule constraints. This makes the advice teachable and auditable.
For Research: It provides a frozen evaluation testbed where belief modeling, decision rules, and learned policies can be varied independently.
For Decision Support (Generalizable Insight): The paper asserts that reward alignment is more critical than algorithm choice in offline decision support systems trained from logged behavior.
- When logs contain heterogeneous quality (average players vs. winners), shaping rewards toward local proxies (coins) teaches the model to imitate the average.
- Conditioning rewards on the final outcome (winning) teaches the model to imitate the successful agents.
- Practitioners observing flat offline RL curves should audit the reward definition before concluding the algorithm or data is inadequate.

The paper remains modest regarding its limitations, acknowledging that the decision rule is myopic (does not model long-term retaliation or signaling), the opponents are scripted rather than adaptive humans, and the evaluation is simulation-based without human-in-the-loop validation. It identifies the dominant error mode as over-eager challenging due to uncontrolled multiplicative belief updates, suggesting future work in belief calibration and Thompson sampling.