🧠 neuroscience

The unique value of zero prediction errors in reinforcement learning

This study demonstrates that zero prediction errors are not merely neutral events but are psychologically and computationally significant, actively shaping momentary happiness, distinct belief states, and neural feedback processing to influence subsequent learning in human reinforcement learning.

Original authors: Lloyd, B., Kikumoto, A., Wurm, F., Vives, M.-L.

Published 2026-07-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lloyd, B., Kikumoto, A., Wurm, F., Vives, M.-L.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine your brain is a super-smart weather forecaster. Every morning, you guess if it will rain, and then you check the sky. Usually, learning happens when you get it wrong. If you say "sunny" and it pours, your brain goes, "Whoops! Update the forecast!" This is the classic idea of learning: mistakes drive us forward.

But what happens when you get it perfectly right? You guess "sunny," and the sun shines. In the old school of thought, this was just a boring "no news" moment. Your brain was supposed to shrug and move on, because there was no error to fix.

A new study suggests that getting it perfectly right is actually a huge, exciting event for your brain. It's not just a "zero error"; it's a special signal that makes you happier, changes how you learn, and even lights up your brain in a unique way.

The "Perfect Hit" Feeling

The researchers set up a game where people had to guess a number that would appear next. The numbers came from a cloud of possibilities—sometimes the cloud was tight (easy to guess), and sometimes it was wide and messy (hard to guess).

Here's the twist: The researchers secretly rigged 15 out of every 80 rounds so that the number that appeared matched the player's guess exactly. The players didn't know this was happening; they just thought they were lucky.

When they asked the players, "How happy are you right now?", the answer was surprising. People were the happiest not when they got a huge, unexpected bonus, but when their prediction was perfectly accurate. It was like a sweet spot of joy. If they were wrong by a little bit, or a lot, happiness dropped. But that perfect "bullseye" moment? That was the peak.

The study suggests that our brains treat this "perfect match" as a special reward in itself, separate from just getting a good prize. It's the feeling of, "I knew it! I really knew it!"

The "Two-Track" Brain

So, how does the brain use this perfect moment? The researchers tested three ideas:

The Old Way: Perfect matches do nothing special; learning only happens when you're wrong.
The "Sticky" Way: Perfect matches just make you stubborn, so you keep guessing the same thing.
The "Two-Track" Way (The Winner): When you get it perfect, your brain creates a secret, separate file just for those perfect moments.

The study found that the "Two-Track" idea was the best fit. It suggests that when you get a perfect prediction, your brain doesn't just ignore it. Instead, it opens a special "Zero Prediction Error" folder. It keeps a separate belief about how things work based only on the times you were right.

This is especially true when the world is confusing (high uncertainty). When things are messy and hard to predict, getting it right feels like a massive relief. The study found that people who generally hate uncertainty (they get anxious when things are unpredictable) relied on this "perfect match" folder even more. They clung tighter to the moments they were right to feel safe.

The Brain's "P3" Spark

To see what was happening inside the head, the researchers put electrodes on 40 volunteers' scalps to read their brain waves.

They found that when a perfect prediction happened, the brain didn't just go quiet. It fired off a distinct electrical spark, a big wave of activity called a P3 response, that looked different from when people were wrong.

Here is the coolest part: The size of this brain spark predicted what the person would do next, but in opposite directions depending on the situation:

If they were wrong: A bigger brain spark meant they would change their guess a lot next time. (The spark said, "Okay, big mistake, let's fix it!")
If they were right: A bigger brain spark meant they would stick to their guess even more. (The spark said, "Perfect! Lock this in! Don't change a thing!")

It's like the brain uses the same "alert" signal to either panic and fix a mistake or to celebrate and double down on a success.

What This Means

The study doesn't say this is a magic cure for anything or that it works in every single situation. It suggests that "getting it right" is a powerful, active learning tool, not just a pause button.

When we predict the world perfectly, we aren't just sitting still. We are feeling a rush of happiness, our brains are building a special "I was right" memory, and we are using that moment to stabilize our beliefs, especially when the world feels chaotic. It turns out, being right is just as important for learning as being wrong.

Technical Summary: The Unique Value of Zero Prediction Errors in Reinforcement Learning

Problem Statement

Reinforcement learning (RL) theory traditionally posits that learning is driven by prediction errors (PEs)—the discrepancy between expected and observed outcomes. In standard models (e.g., Rescorla & Wagner, 1), outcomes that perfectly match expectations (zero PEs) are treated as the absence of a learning signal, carrying no informational value beyond the reward itself. However, this framework leaves open the question of whether "perfect predictions" are psychologically neutral or if they constitute distinct, informative events that actively shape affect, belief updating, and neural processing. Specifically, it is unclear if zero PEs trigger unique computational mechanisms, such as the formation of separate latent belief states, particularly in uncertain environments or among individuals with high intolerance of uncertainty.

Methodology

The study employed a multi-modal approach combining behavioral experiments, computational modeling, and electrophysiology (EEG) to investigate the impact of zero PEs.

Experimental Design:

Task: Participants performed a "Number Generator Task" where they predicted rewards drawn from Gaussian distributions with varying widths (Low, Medium, High uncertainty).
Manipulation: Unbeknownst to participants, 18.75% of trials (15 out of 80 per block) were experimentally constrained so that the reward outcome exactly matched their prediction, creating "zero prediction errors."
Measures:
- Affect: Participants reported momentary happiness every three trials.
- Behavior: Trial-by-trial prediction updates were recorded.
- Neural: In a subset of participants ( $N=40$ ), reward-locked EEG was recorded to analyze event-related potentials (ERPs).
Participants: Two experiments were conducted: an online study ( $N=97$ ) and an EEG laboratory study ( $N=40$ ), totaling $N=137$ .

Computational Modeling:
Three reinforcement learning models were compared using hierarchical Bayesian inference:

PH-standard: A standard Pearce-Hall model where learning rates scale with PE magnitude; zero PEs produce no update.
PH-persist: An extension where zero PEs induce a "persistence" state, biasing future predictions toward the confirmed outcome.
ZePE (Zero Prediction Error) Model: A novel model proposing that zero PEs trigger the formation of a distinct latent belief state ( $B_{zero}$ ) tracked in parallel with the standard belief state ( $B_{std}$ ). The final prediction is a weighted mixture of these two states, controlled by a parameter $\omega$ .

Analysis:

Behavioral: Mixed-effects regression analyzed happiness ratings and prediction updates.
Model Comparison: Leave-one-out cross-validation (LOO) and expected log predictive density (ELPD) were used to determine the best-fitting model.
EEG: Mass univariate regression (GLM) and cluster-based permutation tests analyzed neural responses to PEs. Psychophysical interaction (PPI) analyses tested whether residual neural activity predicted subsequent behavioral updating.

Key Results

1. Affective Impact: Zero PEs Maximize Happiness

Contrary to the assumption that happiness scales linearly with positive PEs, participants reported the highest momentary happiness following zero PEs.

Happiness followed an inverted-U relationship with PE magnitude, peaking at zero and declining for both large positive and negative PEs.
Computational modeling of happiness confirmed that adding a specific term for zero PEs significantly improved model fit compared to models relying solely on reward magnitude and standard PEs. Zero PEs were the strongest predictor of happiness, outweighing the reward outcome itself.

2. Behavioral Adaptation: The ZePE Model

The ZePE model provided the best fit for behavioral data in both experiments, significantly outperforming the PH-standard and PH-persist models.

Distinct Latent State: The data supports the hypothesis that zero PEs induce a separate latent belief state that guides subsequent updating.
Uncertainty Modulation: The weight parameter $\omega$ (governing the influence of the zero-PE state) increased with environmental uncertainty. Participants relied more heavily on the zero-PE state in high-uncertainty blocks.
Individual Differences: Higher scores on the Intolerance of Uncertainty (IUS) scale correlated with higher $\omega$ values, suggesting that individuals averse to uncertainty place greater weight on perfect predictions to stabilize their beliefs.
Updating Patterns: Zero PEs were followed by larger absolute prediction updates in low-uncertainty blocks, and this effect intensified under higher uncertainty, indicating that perfect predictions actively shape learning dynamics rather than merely confirming existing beliefs.

3. Neural Signatures: Distinct EEG Responses

EEG analyses revealed that zero PEs elicit distinct neural responses that differ from standard PEs.

ERP Components: Zero PEs elicited a robust, sustained late positive response (P3-like) over frontocentral and centroparietal electrodes (500–800 ms), comparable in magnitude to large positive PEs and significantly larger than other PE conditions.
Latent State Encoding: Regression analyses showed that EEG activity on zero-PE trials tracked deviations from the zero-PE latent belief state ( $B_{zero}$ ), whereas standard trials tracked deviations from the standard belief state ( $B_{std}$ ).
Neural-Behavioral Coupling: Residual EEG activity (unexplained by the models) predicted subsequent updating in opposite directions depending on trial type:
- Standard Trials: Stronger residual activity predicted enhanced updating (learning).
- Zero-PE Trials: Stronger residual activity predicted attenuated updating (belief entrenchment).
- This coupling was modulated by $\omega$ , with higher $\omega$ amplifying the dissociation.

Significance and Claims

The authors claim that this study provides the first systematic evidence that zero prediction errors are not neutral events but are distinct, informative signals that actively shape human learning.

Challenge to Orthodoxy: The findings challenge the standard RL assumption that learning occurs exclusively through mismatches. Instead, "trial-and-hit" (perfect prediction) is a complementary learning mechanism.
Intrinsic Reward: Perfect predictions appear to be intrinsically rewarding, generating positive affect independent of external reward magnitude, likely due to the successful resolution of uncertainty.
Computational Mechanism: The study proposes that the brain maintains parallel latent belief states for standard and zero PEs. The weighting of these states ( $\omega$ ) is a flexible control parameter influenced by environmental uncertainty and individual traits (intolerance of uncertainty).
Neural Specificity: Zero PEs engage specific neural circuits (reflected in P3-like activity) that serve to stabilize beliefs rather than update them, acting as a mechanism for "belief entrenchment" when the environment is successfully predicted.

The paper concludes that perfectly predicted outcomes are essential for learning, serving to stabilize beliefs and reduce uncertainty, particularly for individuals who find uncertainty aversive. This suggests a more complex learning architecture where the confirmation of expectations is as computationally significant as the violation of them.