Adaptive Noise in Human Game Play: An Information-Theoretic Signature that Separates Learning Models on Matching-Pennies Data

In the study of how people learn to make decisions, scientists often look at games where two players compete, trying to outsmart one another. A classic example is a simple game of matching pennies, where one player wins if the coins match and the other wins if they do not. To succeed, a player must be unpredictable; if they fall into a pattern, their opponent will spot it and exploit it. At the same time, a player must pay attention to what their opponent is doing to adjust their own strategy. This creates a delicate balance: a player needs to be random enough to avoid being predictable, but smart enough to react to the other person. For decades, the standard computer models used to simulate human learning have struggled with this balance. These models typically start with a lot of random guessing but gradually become more confident and deterministic as they learn, eventually settling into a rigid, predictable routine. This behavior, however, does not match what real people actually do in the laboratory.

A new study by Pratyush Mahadevaiah investigates this discrepancy by analyzing data from hundreds of people playing these games over long sessions. Instead of trying to fit a complex mathematical formula to the results, the researcher used a method from information theory to measure two specific qualities of the players' behavior over time. The first quality is the amount of randomness, or "noise," in a player's choices. The second is a measure of how much a player's next move is influenced by their opponent's last move. By tracking these two factors across 108 different groups of players, the study reveals a distinct pattern in human behavior that standard computer models fail to replicate. The research shows that humans maintain a steady level of unpredictability throughout the entire game while simultaneously staying highly responsive to their opponent's actions. In contrast, the most common learning models used by scientists become too predictable and stop reacting to their opponents as the game progresses.

The data comes from a large set of experiments where groups of eight people played a series of matching-pennies games for 200 rounds each. The researchers recorded every single move made by every player, creating a detailed timeline of decisions. They then applied a specific measurement to see how much information a player's current move held about their opponent's previous move. This measurement, known as transfer entropy, acts like a detector for responsiveness. If a player is truly paying attention and adjusting their strategy based on what the other person just did, this number will be high. If the player is just acting on their own internal logic without regard for the opponent, the number will be low. When the researchers looked at the human data, they found that players maintained a high level of randomness, with an average measure of unpredictability staying around 0.64 bits throughout the game. More importantly, they showed a strong signal of responsiveness, with their choices clearly influenced by their opponent's last move.

When the same analysis was applied to a standard computer model called Q-learning, the results were starkly different. This model, which is widely used to simulate how agents learn from rewards, started with some randomness but quickly lost it. Within about thirty rounds, the model's behavior became almost entirely predictable, dropping its randomness measure to near zero. Because it became so predictable, it also stopped reacting to its opponent; the measure of responsiveness fell to almost nothing. The researchers tested whether simply changing the settings of this computer model could fix the problem. They tried making the model more random by adjusting its internal "temperature" settings, but this only created a trade-off. When the model was made more random, it became less responsive to the opponent. When it was made more responsive, it became less random. No single setting allowed the model to be both unpredictable and attentive at the same time, a combination that humans achieved naturally.

The study then tested four other common learning models to see if any of them could reproduce the human pattern. Most of these models failed in similar ways: some became too rigid and predictable, while others remained random but failed to pay attention to the opponent. However, one specific model, known as self-tuning experience-weighted attraction, succeeded where the others failed. This model, which updates its strategy based on both past rewards and a mechanism that detects changes in the game, managed to stay both random and responsive. Its behavior closely matched the human data, occupying the same space on the graph where the human players were found. While this model did not match the human numbers perfectly, it was the only one that came close to capturing the dual nature of human play.

The findings suggest that the way humans learn in competitive situations is structurally different from how standard reinforcement learning models operate. Humans do not simply become more confident and less random as they gain experience; instead, they maintain a steady level of adaptive noise that allows them to remain unpredictable while still tracking their opponent. The study does not claim to prove that this specific computer model is exactly how the human brain works, but it does identify a crucial feature that any successful model of human learning must possess. To accurately describe how people play these games, a model must be able to be noisy and attentive at the same time. This insight helps narrow down the search for better theories of human behavior, moving beyond simple predictions of what people will do next to understanding the dynamic qualities of how they learn and adapt over time.