Cholinergic modulation of reinforcement learning and prefrontal value computations under uncertainty

Technical Summary: Cholinergic Modulation of Reinforcement Learning and Prefrontal Value Computations under Uncertainty

Problem Statement
The neuromodulator acetylcholine has been hypothesized to play a critical role in governing learning processes under conditions of uncertainty. However, the specific mechanisms by which muscarinic acetylcholine receptors influence reward-guided learning and decision-making across varying degrees of uncertainty remain to be fully elucidated. This study investigates the causal role of muscarinic transmission in these cognitive processes, specifically examining how its blockade affects behavioral adaptation and neural representations of value.

Methodology
The study employed a within-subjects, placebo-controlled design involving 43 healthy male participants. Participants were administered 4 mg of biperiden, a muscarinic antagonist, to transiently block muscarinic acetylcholine receptors. The experimental protocol consisted of two distinct tasks:

1. A Gambling Task: Designed to assess decision-making without the requirement for learning.
2. A Learning Task: Involving option values associated with varying levels of uncertainty, including scenarios where choice-outcome contingencies switched frequently.

During task performance, magnetoencephalography (MEG) was recorded to capture neural dynamics. The behavioral data were analyzed using reinforcement learning models to quantify learning rates and probability sensitivity. Neural data were specifically examined for lateral prefrontal representations of learned reward probability, with a focus on high-beta (20–30 Hz) power.

Key Results
The administration of biperiden yielded dissociable effects depending on the task demands:

• Behavioral Findings: In the gambling task, where no learning was required, biperiden had no significant effect on decision-making. Conversely, in the learning task, biperiden reduced participants' sensitivity to probabilities. This reduction was most pronounced when choice-outcome contingencies switched frequently.
• Modeling Insights: Reinforcement learning modeling indicated that the observed behavioral changes were driven by noisier estimates of probabilities. Specifically, biperiden induced increased learning rates for rewarded choices, leading to less stable probability estimations in uncertain contexts.
• Neural Findings: The behavioral alterations were paralleled by specific neural changes. Under biperiden, the lateral prefrontal representation of learned reward probability, typically observed in the high-beta (20–30 Hz) frequency band, was eliminated.

Key Contributions
This study provides empirical evidence linking muscarinic acetylcholine transmission directly to the calibration of learning rates in uncertain environments. By combining pharmacological manipulation, behavioral modeling, and MEG, the authors demonstrate that muscarinic blockade disrupts the stability of probability estimates during learning, a process that is distinct from static decision-making. Furthermore, the work identifies a specific neural signature—the loss of high-beta power representation of reward probability in the lateral prefrontal cortex—as a correlate of this cholinergic modulation.

Significance and Claims
The authors conclude that muscarinic acetylcholine transmission is essential for controlling learning in highly uncertain contexts. Specifically, the system appears to be critical when the cognitive demand for carefully calibrated adjustments is highest. The findings suggest that intact muscarinic signaling is necessary to maintain stable probability estimates and appropriate learning rates when environmental contingencies are volatile, thereby supporting adaptive behavior in complex, uncertain situations.