Dopamine dips during unrewarded actions promote punishment-resistant reward seeking

Technical Summary: Dopamine Dips During Unrewarded Actions Promote Punishment-Resistant Reward Seeking

Problem Statement
Punishment-resistant reward-seeking is a defining characteristic of addiction, yet it is less prevalent in females than in males. While sex hormones, particularly estradiol, are known to modulate dopamine signaling, the specific neural mechanisms by which estradiol influences the development of punishment-resistant behavior in females remain unclear. Previous work established that phasic dopamine peaks in the dorsomedial striatum (DMS) during rewarded actions predict the emergence of punishment-resistant reward-seeking. However, it was unknown whether sex differences in this behavior arise from estradiol-dependent modulation of these peaks or other dopamine signal components, such as dips following unrewarded actions.

Methodology
The study employed a combination of fiber photometry, optogenetics, and behavioral pharmacology in mice to investigate the causal link between DMS dopamine dynamics and punishment resistance.

1. Hormonal Manipulation and Fiber Photometry:

   • Female mice were divided into three groups: Sham (intact cycling), Ovariectomized with Placebo (OVX-P, mimicking low estradiol/diestrus), and Ovariectomized with chronic 17β-estradiol replacement (OVX-E, mimicking high estradiol/proestrus).
   • All groups underwent Random Interval 60 (RI60) reinforcement training to acquire a nosepoke behavior for sugar rewards.
   • Fiber photometry using the GRAB-gDA3m sensor recorded DMS dopamine signals during rewarded and unrewarded nosepokes.
   • Punishment resistance was assessed via "shock probes" where a subset of rewarded nosepokes was paired with mild footshocks.

2. Optogenetic Causal Testing:

   • To test the causal role of dopamine dips, the authors used DAT-IRES-Cre mice (both males and females) injected with AAV-eNpHR3.0 (inhibitory opsin) or a control virus (EYFP) into the medial substantia nigra pars compacta (SNc), targeting DMS-projecting neurons.
   • Fiber optic cannulas were implanted in the DMS.
   • During RI60 training, unrewarded nosepokes were paired with a 1-second continuous 625nm light pulse to inhibit dopamine terminals, artificially creating or exaggerating dopamine dips.
   • Shock probes were administered to measure the development of punishment resistance.

Key Results

• Behavioral Phenotypes: While all groups acquired the RI60 task similarly, hormonal status significantly altered punishment resistance. OVX-E females (chronic high estradiol) were 100% categorized as punishment-resistant (PR), whereas Sham females were predominantly punishment-sensitive (PS) or delayed PR. OVX-P females showed an intermediate but significant shift toward PR compared to Sham.
• Dopamine Signal Dynamics: Fiber photometry revealed no significant differences in dopamine peaks following rewarded nosepokes across groups. However, significant differences emerged in dopamine dips following unrewarded nosepokes. Both OVX-P and OVX-E groups exhibited significantly larger and more prolonged dopamine dips compared to Sham mice. The magnitude of the dip in OVX-E mice was the most pronounced.
• Optogenetic Causality: Artificially inducing dopamine dips via NpHR inhibition during unrewarded nosepokes was sufficient to accelerate the development of punishment-resistant reward-seeking in both male and female mice. NpHR-treated mice received significantly more shocks during probe sessions compared to controls.
• Sex Differences in Optogenetic Effects: While the optogenetic effect was significant in both sexes, the effect size was larger in females ($\eta^2_p = 0.38$) than in males ($\eta^2_p = 0.17$), suggesting a heightened sensitivity to dopamine dip modulation in females.

Key Contributions

1. Identification of a Novel Mechanism: The study identifies exaggerated dopamine dips during unrewarded actions, rather than enhanced dopamine peaks during rewarded actions, as a critical driver of punishment-resistant reward-seeking.
2. Hormonal Modulation: It demonstrates that chronic high estradiol states (mimicking a non-cycling high-estrogen state) predispose female mice to punishment resistance specifically by amplifying these dopamine dips.
3. Causal Validation: Using optogenetics, the authors provide causal evidence that mimicking these dopamine dips is sufficient to induce punishment-resistant behavior in both sexes.
4. Theoretical Refinement: The findings challenge the standard Reward Prediction Error (RPE) framework, which posits that dopamine dips should suppress action performance. Instead, the data suggest that dopamine dips may suppress learning about the value of an action, thereby promoting persistence (policy learning) despite negative outcomes.

Significance and Claims
The paper claims that these findings offer a mechanistic explanation for sex differences in addiction vulnerability, specifically linking chronic estradiol states to altered dopamine signaling that fosters compulsive behavior. The authors suggest that the suppression of learning from unrewarded actions via dopamine dips allows reward-seeking to persist in the face of punishment.

From a translational perspective, the study highlights a gap in modeling female dopamine function, noting that many women experience non-cycling hormonal states due to birth control or menopause. The authors propose that understanding how these states regulate dopamine transporter (DAT) function and subsequent dip magnitude is crucial for understanding vulnerability to psychiatric disorders. The work supports policy-based accounts of dopamine function, suggesting dopamine controls the rate of learning about action values rather than directly estimating action value.