📄 addiction medicine

Markov computational modeling to predict opioid vs. money choice and behavioral effort in regular heroin users

This study demonstrates that a Markov computational model accurately predicts opioid-seeking behavior and effort expenditure in regular heroin users by analyzing how decision latency and effort discrepancies influence the tendency to repeat or switch choices between drug and monetary rewards.

Original authors: Jhand, A. S., Greenwald, M. K.

Published 2026-08-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jhand, A. S., Greenwald, M. K.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Addiction is often described as a battle between two forces: the overwhelming pull of a drug and the competing pull of a normal life. For decades, scientists have tried to understand how the brain weighs these options. Is the choice to use drugs a rigid, automatic reflex, like a machine that cannot be stopped? Or is it a flexible decision that changes based on the situation, the cost, and the available alternatives? To answer this, researchers look at how people make choices when they can earn a drug or something else, like money, by doing work. They measure how fast a person decides and how much effort they are willing to spend. By studying these tiny, moment-to-moment decisions, scientists hope to see the hidden rules that guide behavior, moving beyond simple observations to a deeper understanding of why a person might keep reaching for a drug even when other options are right there.

A team of researchers at Wayne State University set out to build a computer model that could predict these choices in real time. They studied twenty-three to thirty-six regular heroin users at a time, depending on the specific group, who were not currently in treatment. To ensure the participants were not choosing drugs just to stop the pain of withdrawal, the researchers first stabilized them on a daily dose of buprenorphine, a medication that eases withdrawal symptoms but does not block the effects of other opioids. Once the participants were stable, they entered a controlled room where they faced a series of choices. On a computer screen, they could click a button to earn either a small amount of money or a dose of hydromorphone, a powerful opioid. The catch was that the work required to earn the reward got harder with every single choice. If a participant chose the drug once, the next time they wanted it, they had to click the mouse many more times. The same rule applied to the money. This setup, known as a progressive ratio, forced the participants to decide how much effort they were willing to expend for each reward.

The researchers recorded every click, every pause, and every switch between the drug and the money. They noticed a clear pattern: most of the time, people stuck with their previous choice. If they chose the drug once, they were very likely to choose it again immediately. This repetition happened about eighty percent of the time. However, the researchers wanted to know what made someone break that pattern and switch to the other option. They built a mathematical model, a type of computer simulation, to test two main ideas. First, they looked at the "effort discrepancy," which is the difference in how hard it was to get the drug versus the money on the next turn. Second, they looked at how fast the person made their decision compared to the turn before. They fed this data into a model designed to predict whether a person would repeat their choice or switch.

The results were striking. The computer model was able to predict whether a person would stick with their choice or switch to the other option with ninety-three percent accuracy. This high level of success suggests that the decision to keep using drugs or to switch to something else is not random or purely compulsive. Instead, it is highly sensitive to the specific conditions of the moment. The model showed that when the effort required to get the drug became much higher than the effort for money, people were more likely to switch. Similarly, if a person took a long time to make a decision, it signaled that they were more likely to change their mind on the next turn. The speed of the decision mattered, too. People who chose the drug very quickly tended to keep choosing it and were willing to do more work to get it. Those who hesitated before choosing the drug were more likely to switch to money on the next try.

The study also revealed how individual differences shaped these choices. Participants who had used cocaine recently took much longer to decide on the money option compared to the drug, suggesting they valued the money less. However, when the researchers doubled the amount of money from two dollars to four dollars, this difference disappeared, and the participants started making money choices much faster. This finding challenges the idea that addiction is an unchangeable state. Instead, it suggests that the value of a drug is relative; if the alternative reward becomes attractive enough, the brain recalculates the choice. The very first choice a person made in a session was a powerful predictor of their behavior for the rest of the time. Those who chose the drug first tended to work harder and longer to get it, while those who chose money first were more likely to stick with that path.

The researchers concluded that drug-seeking behavior is a mix of habit and goal-directed thinking. It is not a blind compulsion that ignores the world around it. The participants were constantly weighing the cost of their actions against the reward, and their behavior shifted when the balance changed. By using a model that tracks these tiny shifts in effort and speed, the study provides a new way to see addiction not as a fixed trait, but as a dynamic process that responds to the environment. The work suggests that if we can understand the specific factors that make a person switch from a drug to a non-drug reward, we might be able to design better ways to help them make that switch in real life. The study did not claim to have solved the problem of addiction, but it did offer a clear, data-driven map of how the choices are actually made, one click at a time.

Technical Summary: Markov Computational Modeling of Opioid vs. Money Choice and Behavioral Effort

Problem Statement
Opioid use disorder (OUD) remains a critical public health crisis, with a significant portion of individuals failing to achieve sustained abstinence despite available treatments. While behavioral economic frameworks have successfully modeled drug vs. non-drug choice at the "molar" level (session-level demand curves), there is a scarcity of human data regarding "pico-economic" or trial-by-trial decision-making processes. Specifically, there is a lack of computational models that quantify the dynamic interplay between effort, decision latency, and the persistence of drug-seeking behavior in real-time. This study addresses the need to formalize the mechanisms underlying repetitive drug-seeking behavior, distinguishing between habitual (automatic) and goal-directed (deliberative) processes in a controlled setting that mimics real-world trade-offs between opioid reinforcement and monetary alternatives.

Methodology
The study is a secondary data analysis of four programmatically linked clinical trials involving out-of-treatment, regular heroin users ( $N=83$ across three experimental conditions). Participants were stabilized on a fixed dose of buprenorphine (8 mg/day) to suppress withdrawal symptoms, thereby isolating positive reinforcement mechanisms from negative reinforcement (withdrawal alleviation).

Experimental Design: Participants engaged in a 3-hour choice session where they could earn either money or hydromorphone (HYD) via a progressive ratio (PR) schedule. The effort required to earn the same commodity increased exponentially with each successive choice on that option, while the alternative option reset to a lower effort level.
Conditions: Three distinct conditions varied the unit magnitudes of the reinforcers:
1. $2 vs. 1-mg HYD ( $n=23$ )
2. $2 vs. 2-mg HYD ( $n=36$ )
3. $4 vs. 2-mg HYD ( $n=24$ )
Data Collection: Trial-level outcomes included decision latency (time from option appearance to first click) and the specific choice made. Session-level outcomes included breakpoints (maximum effort expended) and the difference in breakpoints between drug and money.
Computational Model: A Markov chain framework was developed to predict the probability of a participant repeating their previous choice ("stick") versus switching to the alternative ("switch"). The model utilized a logistic regression function (Equation 1) with two primary inputs:
1. Same-choice effort discrepancy: The difference in required effort to earn the same commodity on the next trial versus the alternative commodity.
2. Log latency ratio: The logarithm of the ratio of the current decision speed to the previous decision speed.
- The model included fixed parameters for baseline bias ( $\beta_0$ ) and nonlinear scaling ( $\alpha_1, \alpha_2$ ), with free parameters ( $\beta_1, \beta_2$ ) fitted to individual participant data to determine sensitivity to effort and latency discrepancies.

Key Results

Behavioral Patterns: Participants repeated their previous choice on 80% of trials. The computational model accurately predicted "stick vs. switch" behavior on 93% of trials, significantly outperforming a baseline heuristic that assumed simple repetition.
Individual Differences:
- First-Trial Choice: Participants who chose hydromorphone on the first trial ("drug-first") made significantly more consecutive drug choices and expended greater effort (higher breakpoint differences) compared to "money-first" responders.
- Cocaine Use: Recent cocaine users (Coc+) exhibited slower decision latencies for money relative to hydromorphone compared to non-users, particularly in the $2 vs. 2-mg condition. This devaluation of non-drug rewards was mitigated when the monetary reward was increased to $4.
- Correlations: Faster relative decision latencies for drug vs. money correlated inversely with breakpoint differences, indicating that individuals who decided on drugs more rapidly were willing to expend significantly more effort to obtain them.
Model Coefficients:
- In the $2 vs. 2-mg condition, "money-first" responders showed a significantly stronger sensitivity to latency discrepancies ( $\beta_2$ ) than "drug-first" responders, suggesting drug-first responders rely more on automatic processing and are less prone to internal deliberation when hesitating.
- Increasing the hydromorphone dose from 1-mg to 2-mg (Condition 1 to 2) reduced participants' sensitivity to effort discrepancies ( $\beta_1$ became less negative), indicating a greater willingness to persist in effortful drug-seeking when the drug dose was higher.
- Individual case studies of participants completing all three conditions revealed that the drivers of choice switching (effort vs. latency) were context-dependent and varied between individuals.

Significance and Claims
The authors claim this is the first study to provide a computational model of dynamic, trial-by-trial drug-seeking behavior in chronic regular users under experimentally controlled conditions. The study demonstrates that:

Predictive Validity: A Markov computational model can accurately predict effortful drug-seeking behavior in a choice paradigm, capturing the "stick vs. switch" dynamics with high precision.
Mechanistic Insight: Drug-seeking behavior in this population is not purely compulsive; it is malleable and sensitive to environmental contingencies (e.g., reinforcer magnitude) and individual differences (e.g., first-trial choice, polysubstance use).
Habitual vs. Goal-Directed: The findings support a mixed model of behavior where drug-seeking involves both habitual sequences (shorter latencies, high repetition) and goal-directed sensitivity to effort and reward value.
Clinical Relevance: The results suggest that initial choices may serve as a proxy for a subject's motivational state and willingness to expend effort. Furthermore, increasing the value of non-drug alternatives (e.g., higher monetary rewards) can effectively disrupt drug-choice chains, supporting the utility of contingency management approaches.

The authors maintain a modest stance, acknowledging limitations such as small subgroup sizes for sex differences, the pre-fentanyl era context of the data, and the restriction of the model to dynamic parameters to avoid overfitting. They conclude that while the model does not capture the full complexity of decision-making, it provides a valuable tool for identifying underlying psychological processes in opioid seeking.

Technical Summary: Markov Computational Modeling of Opioid vs. Money Choice and Behavioral Effort

More like this