Rapid offline integration of learned experiences is shaped by valence and biases future decisions

Technical Summary: Rapid Offline Integration of Learned Experiences is Shaped by Valence and Biases Future Decisions

Problem Statement
While it is established that the brain continues to process recently learned experiences during offline periods (awake rest or sleep) to support adaptive behavior, the specific mechanisms governing how experiences of opposing valence (gain vs. loss) are differentially reactivated remain unclear. Previous research has largely examined reward or punishment in isolation, leaving open questions regarding whether offline reactivation is valence-sensitive, how distinct components of learned experiences (observable stimuli/outcomes vs. latent variables like expected value and prediction errors) are represented, and whether these reactivation patterns differentially influence subsequent decision-making in novel contexts.

Methodology
The study employed a probabilistic reward-learning (PRL) paradigm combined with functional magnetic resonance imaging (fMRI), computational modeling, and multivariate predictive decoding.

• Participants and Task: Forty-six healthy participants underwent an fMRI session involving a sequential learning and transfer design. In the learning stage, participants learned probabilistic stimulus-outcome associations involving monetary gains and losses across three stimulus pairs (AB, CD, EF). This was followed by a transfer stage where learned stimuli were recombined into novel pairs without feedback, requiring generalization of learned values to optimize reward-seeking and loss-avoidance.
• Computational Modeling: To dissociate observable components (stimuli, outcomes) from latent learning signals, the authors fitted four computational models to behavioral data. The Reinforcement Learning (RW) model with dual learning rates (M2) was selected as the best fit. This model generated trial-wise estimates of Expected Value (Q-value) and Prediction Errors (PEs) for both gain and loss contexts.
• Multivariate Pattern Analysis (MVPA):
  • Template Creation: Linear Support Vector Machines (SVM) were trained to distinguish gain- vs. loss-related stimuli and outcomes. Linear Support Vector Regression (SVR) was used to predict continuous levels of Q-values and PEs. These models identified whole-brain neural signatures for observable and latent experience components.
  • Offline Reactivation: Classifiers trained on learning-phase data were applied to the final 10 TRs of post-learning rest periods to estimate the probability of reactivation for gain- and loss-related experiences (stimuli, outcomes, Q-values, PEs).
  • Network Dynamics: A whole-brain SVR analysis tracked neural pattern changes across successive rest periods to identify networks showing progressive engagement.
• Behavioral Correlation: Reactivation probabilities and network pattern expressions were correlated with choice accuracy in the subsequent transfer stage (specifically selecting the high-gain stimulus "A" and avoiding the high-loss stimulus "B").

Key Contributions and Results

1. Dissociable Neural Signatures: The study demonstrated that gain- and loss-related experiences are encoded in partially dissociable multivariate neural patterns. Observable components (stimuli/outcomes) and latent variables (Q-values/PEs) exhibited distinct spatial representations. For instance, Q-values were primarily predicted by the right superior frontal gyrus (SFG) and anterior insula, while PEs were predicted by the ventral striatum (VS) and bilateral anterior insula.
2. Valence-Sensitive Reactivation Asymmetry:
   • Strength vs. Stability: Gain-related experiences showed significantly stronger overall reactivation probabilities during rest compared to loss-related experiences. However, loss-related reactivation exhibited greater temporal coherence and stability across rest periods.
   • Observable vs. Latent: Observable components (stimuli/outcomes) were reactivated more strongly than latent components (Q-values/PEs).
3. Behavioral Prediction: Crucially, the study found a functional asymmetry in how reactivation influenced future decisions.
   • Loss Avoidance: Higher reactivation probabilities of loss-related experiences during the first rest period significantly predicted more accurate avoidance of the loss-associated stimulus (B) in the subsequent transfer stage.
   • Reward Seeking: No significant association was found between gain-related reactivation strength and the selection of the reward-associated stimulus (A).
4. Distributed Network Engagement: A distributed hippocampal-cortical network, including the posterior cingulate cortex (PCC), middle temporal gyrus (MTG), and a small hippocampal cluster, showed progressively increasing engagement across post-learning rest periods. Conversely, value-evaluation regions (VS, ventral mPFC) showed decreasing activation.
   • Behavioral Link: Increased pattern expression in the PCC and MTG during rest was positively associated with accurate loss avoidance in the transfer stage, but not with reward-seeking behavior.

Significance and Claims
The paper claims to provide evidence that rapid offline integration of learned experiences is both valence-sensitive and functionally asymmetric. The findings suggest that while the brain may broadly restate positive experiences, the offline reactivation of negative experiences is more stable and specifically drives adaptive loss-avoidance behaviors in novel contexts.

The authors posit that this mechanism may offer a neural basis for understanding how negative experiences exert a disproportionate influence on future cognition and behavior, potentially relevant to the persistence of negative thoughts in affective disorders. The study extends reinforcement learning models by integrating offline processes, suggesting that the "memory triage" of salient experiences is not merely a function of reward magnitude but involves distinct, valence-dependent reactivation dynamics that prioritize threat avoidance.

The authors maintain modesty regarding their claims, acknowledging limitations such as the temporal resolution of fMRI (which captures representational reinstatement rather than single-neuron replay sequences), the reliance on motivational contexts, and the relatively brief learning duration. They do not claim to have resolved the translational gap between human fMRI findings and rodent electrophysiology but suggest their representational framework provides a necessary step toward understanding the functional role of offline integration in adaptive decision-making.