🧠 neuroscience

Humans use a dual policy to improve inferences during epistemic information seeking

Through three studies involving 702 participants, the authors demonstrate that humans improve epistemic inference by employing a distinct two-stage policy—initially "streaking" to test hypotheses before shifting to uncertainty-guided exploration—which outperforms standard neural network strategies and reveals unique psychological traits beyond traditional reward-based exploration.

Original authors: Cao, Y., Almeras, C., Lee, J. K., Maye, I., Wyart, V.

Published 2026-02-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cao, Y., Almeras, C., Lee, J. K., Maye, I., Wyart, V.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The Big Idea: How We Learn When There's No Prize

Imagine you are walking through a massive, unfamiliar city. You have two ways to explore:

The "Hungry Tourist" (Reward-Seeking): You are starving. You want to find the best pizza place right now. You check a review, go there, and if it's good, you eat there again. You are focused on the immediate reward.
The "Curious Explorer" (Epistemic/Information-Seeking): You aren't hungry. You just want to understand the city's layout. You don't know which street leads where, and there's no "best" street yet. You just want to build a mental map.

This paper asks: How do we explore when we aren't looking for a reward, but just for knowledge?

The researchers found that humans have a unique, two-step strategy for learning that computers (specifically standard AI) don't naturally figure out on their own.

The Experiment: The Magic Gem Bags

To test this, the researchers created a game with two "bags" of gems.

Bag A has mostly blue gems (but sometimes orange ones).
Bag B has mostly orange gems (but sometimes blue ones).
You can't see inside; you have to pull one gem out at a time to guess which bag is which.

They played two versions of the game:

The "Match" Game (Reward): You are told, "Collect as many Blue gems as possible." Here, you are motivated by points. You quickly figure out which bag gives blue gems and stick to it.
The "Guess" Game (Knowledge): You are told, "Just learn what's inside the bags. At the end, I'll ask you to guess which bag is which." There are no points for picking the right gem during the game. You are just gathering information.

The Discovery: The "Streaking" Strategy

When playing the Guess game (learning for knowledge), humans did something weird and fascinating that the "Hungry Tourist" didn't do.

The "Streaking" Analogy:
Imagine you are tasting two new soups to figure out which one is spicy and which is salty.

The Logical AI approach: Taste a spoonful of Soup A, then a spoonful of Soup B, then A, then B. Compare them side-by-side constantly.
The Human "Streaking" approach: You take five spoonfuls of Soup A in a row. Sip, sip, sip, sip, sip. Then you switch and take five spoonfuls of Soup B. Sip, sip, sip, sip, sip.

The researchers call this "Streaking." Even though switching back and forth seems like it would give you more data faster, humans naturally group their learning. They focus on one option, test it repeatedly to get a "feel" for it, and then switch to the other.

Why do we do this?
The paper suggests our brains are a bit "noisy" (like a radio with static). If we switch too quickly, the static makes it hard to hear the signal. By "streaking" (sticking with one option), we drown out the noise and get a clearer picture of that specific option before moving on. It's like tuning a radio to one station for a while to make sure you've got the signal clear before switching channels.

The Two-Stage Policy

So, human learning in the "Guess" game follows a two-stage dance:

Stage 1 (The Streak): We pick an option and stick with it for a few turns to build a solid, clear hypothesis. "Okay, this bag is definitely mostly orange."
Stage 2 (The Uncertainty Dance): Once we have a decent idea, we start switching back and forth to the option we know the least about, just to fill in the gaps in our map.

The Computer vs. The Human

The researchers trained Artificial Neural Networks (AI) to play the same game.

The AI's Success: The AI was great at Stage 2. It quickly learned to switch to the "most uncertain" option to learn the fastest. It was a perfect, logical learner.
The AI's Failure: The AI never figured out Stage 1 (Streaking). It didn't naturally group its choices. It just kept switching back and forth.

The Takeaway: Streaking isn't a "bug" in human thinking; it's a feature. It's a specific human trick to deal with our "noisy" brains. We need to focus on one thing at a time to make sense of it, whereas a perfect computer doesn't need that crutch.

Why Do Some People Streak More Than Others?

The study also looked at personality. They found two different types of people:

The "Need for Closure" People: These are people who hate uncertainty and want answers fast. They tend to streak less. They switch quickly because they are anxious to finish the task.
The "Smart Explorers": People with higher general reasoning skills tended to streak more effectively. They used the streaking strategy to build better mental maps, which led to higher accuracy in guessing the bag colors.

Summary in a Nutshell

Reward vs. Curiosity: When we want a reward, we hunt for the best option. When we want knowledge, we explore.
The Human Quirk: When exploring for knowledge, humans naturally "streak"—we focus on one thing repeatedly before switching.
The Benefit: This seems inefficient, but it actually helps us learn better because it helps our brains filter out "noise" and confusion.
The AI Gap: Computers can learn to be efficient, but they don't naturally invent the "streaking" strategy. It's a uniquely human adaptation to our own messy, noisy brains.

In short, humans are not just inefficient computers. We have a special, two-step way of learning that turns our "noisy" brains into a strength, allowing us to build better mental maps of the world than a purely logical machine might.

1. Problem Statement

The paper addresses a gap in understanding human information-seeking behavior, specifically distinguishing between reward-driven exploration (seeking immediate or prospective payoff) and epistemic exploration (seeking knowledge for its own sake, independent of reward).

While the "explore-exploit" trade-off is well-studied in reward-maximization contexts (e.g., multi-armed bandit tasks), it remains unclear how humans gather information when the goal is purely to build an internal model of the environment without immediate or prospective valuation. Furthermore, existing computational models and Artificial Neural Networks (ANNs) often fail to capture the specific, sometimes suboptimal, heuristics humans employ in these epistemic contexts. The authors investigate whether humans adopt a single optimal policy or a complex, dual-policy strategy when the pressure of reward is removed.

2. Methodology

Experimental Design

The study utilized a sequential sampling task with a large sample size ( $N = 702$ total across three studies; primary analysis $N=420$ ). Participants sampled from two "bags" (options), each containing gems of a specific shape but varying in color shade (blue to orange). One color predominated in each bag (e.g., mostly blue), but outcomes were probabilistic (approx. 2/3 dominant, 1/3 noise).

Two conditions were compared:

MATCH Condition (Reward-Driven): Participants were instructed to collect gems matching a target color (e.g., blue). Choices were evaluated based on the immediate accumulation of target-matching gems. This mimics a standard bandit task with prospective valuation.
GUESS Condition (Epistemic): No target color was provided. Participants sampled to learn the dominant color of each bag. At the end of the sequence (8–20 trials), they guessed the dominant color of a randomly selected bag. Performance was evaluated solely on the accuracy of this final inference. Crucially, no intermediate rewards were given.

Computational Modeling

The authors developed a Suboptimal Evidence Accumulation Model to formalize human behavior. Key components included:

Noisy and Leaky Integration: Beliefs were updated via log-likelihood ratios (logLR) but corrupted by learning noise ( $\sigma$ ) and subject to memory leak ( $\delta$ ).
Global Policy Parameters:
- $\beta_v$ : Sensitivity to target-aligned evidence (MATCH only).
- $\beta_c$ : Sensitivity to uncertainty (inverse of absolute belief magnitude).
Local Policy Parameters (The "Streaking" Mechanism):
- $\theta$ : A confidence threshold.
- $\epsilon$ : Repetition strength.
- Mechanism: Before the accumulated evidence for the current option crossed $\theta$ , the agent was biased to repeat the same choice (streak) with probability $\epsilon$ , ignoring the global utility.
Repetition Bias ( $\beta_r$ ): A general tendency to repeat the previous choice regardless of evidence.

Neural Network Simulations

To test if these policies emerge from optimization alone, the authors trained Recurrent Neural Networks (RNNs) on the same task:

MATCH RNN: Trained via policy gradient to maximize reward.
GUESS RNN (State-only): Trained to predict the hidden state (bag color) at the end of the game.
GUESS Meta-RNN: Trained with a hybrid objective: predicting the state plus maximizing meta-cognitive certainty (reducing representational uncertainty) during sampling.

Individual Differences Analysis

The study correlated model-derived parameters with psychological traits:

Need for Cognitive Closure (NFCS): Preference for rapid, definite answers.
ICAR (International Cognitive Ability Resource): General cognitive ability/IQ.
Big Five Personality Traits: Used for bottom-up prediction.

3. Key Contributions

Identification of "Streaking": The discovery of a distinct, two-stage information-seeking policy in epistemic contexts. Humans do not immediately switch to uncertainty-guided exploration; instead, they engage in "streaking"—repeatedly sampling the same novel option in short bursts to test a provisional hypothesis before switching.
Dual-Policy Architecture: The demonstration that human information seeking combines a global policy (uncertainty-guided exploration in GUESS, reward-guided in MATCH) with a local policy (early streaking).
Human vs. AI Dissociation: Showing that while ANNs can learn optimal uncertainty-guided exploration when trained with appropriate objectives, they fail to spontaneously acquire early streaking. This suggests streaking is a human-specific heuristic shaped by cognitive constraints (e.g., noisy belief updating) rather than a default outcome of optimization.
Adaptive Suboptimality: Providing evidence that streaking, while statistically suboptimal in a noise-free world, actually improves inference accuracy in humans because it stabilizes beliefs against the noise inherent in neural computation.

4. Key Results

Behavioral Findings

Condition Differences: In the MATCH condition, participants converged on the target option. In the GUESS condition, they showed a stronger bias toward the most uncertain option (uncertainty-guided sampling).
The Streaking Effect: In the GUESS condition, participants exhibited a temporary bias toward the more certain option early in the game (trials 2–5), repeatedly sampling it (e.g., AAAAA) before switching to the other. This pattern was absent in the MATCH condition.
Performance: Participants who exhibited stronger streaking tendencies (higher Evidence of Streaking, EoS) also demonstrated higher overall decision accuracy in the GUESS condition.

Computational Modeling Results

Model Fit: The model including the streaking mechanism (threshold $\theta$ and repetition $\epsilon$ ) significantly outperformed models without it in the GUESS condition.
Noise and Leak: The GUESS condition was characterized by lower memory leak but higher learning noise compared to MATCH. "Knockout" simulations revealed that learning noise was the primary driver of suboptimality in GUESS.
Benefit of Streaking: Simulations showed that streaking improves accuracy specifically in the presence of noisy belief updating. By accumulating evidence on one option, streaking prevents the dilution of weak beliefs by random noise, allowing for a clearer signal before switching.

Neural Network Results

RNNs trained to maximize predictive accuracy (GUESS state-only) or meta-cognitive certainty (GUESS meta-policy) successfully learned uncertainty-guided sampling.
However, none of the RNNs (across various architectures and training objectives) spontaneously developed the early streaking behavior observed in humans.
Forcing streaking on the RNNs improved their performance only when their global sampling policy was suboptimal, suggesting streaking is a compensatory mechanism for imperfect global policies.

Individual Differences

Double Dissociation:
- Streaking (EoS) was negatively predicted by Need for Cognitive Closure (NFCS). Individuals with a high need for closure (who prefer quick answers) were less likely to engage in the patience required for streaking.
- Uncertainty-Guided Sampling ( $\beta_c$ ) was positively predicted by General Cognitive Ability (ICAR).
Mediation: NFCS did not directly predict accuracy but did so indirectly by reducing streaking. Higher cognitive ability predicted accuracy both directly and indirectly through better uncertainty-guided sampling.

5. Significance

Redefining Exploration: The paper moves beyond the standard "explore-exploit" trade-off by showing that epistemic exploration involves a structured, two-stage process (local hypothesis testing followed by global uncertainty reduction).
Cognitive Constraints as Features: It challenges the view that human suboptimality is purely a deficit. Instead, "streaking" is framed as an adaptive heuristic that compensates for the inherent noise in human belief updating. It allows the brain to stabilize weak signals before making costly switches.
Human vs. Machine Intelligence: The failure of RNNs to replicate streaking highlights a fundamental difference between biological and artificial intelligence. Human strategies are shaped by specific inductive biases and computational constraints (like noisy integration) that standard optimization objectives do not capture.
Trait-Policy Mapping: The study provides a granular link between stable psychological traits (NFCS, IQ) and specific latent computational processes (streaking vs. uncertainty sensitivity), suggesting that personality influences how we compute, not just what we choose.

In conclusion, the paper argues that human information seeking is a layered architecture where global, near-optimal policies are complemented by local, trait-dependent rules (like streaking) that are essential for robust inference in noisy environments.