Estimating the Smallest Worthwhile Difference (SWD) of Psychotherapy for Alcohol Use Disorder: Protocol for a Cross-Sectional Survey

This protocol outlines a cross-sectional survey using the Prolific platform to estimate the smallest worthwhile difference (SWD) of psychotherapy for alcohol use disorder by assessing trade-offs between benefits and burdens among American respondents, with the goal of informing clinical decision-making and understanding variations across patient subgroups and professional stakeholders.

The Gist

The Big Question: "Is It Worth the Hike?"

Imagine you are standing at the bottom of a mountain. You have two choices to get to the top:

1. The Easy Path (Natural Recovery): You just walk up the gentle slope on your own. It takes time, and maybe only 3 out of 10 people make it to the top, but it's free and requires no special gear.
2. The Guided Hike (Psychotherapy): You hire a guide, buy expensive boots, and carry a heavy backpack. It costs money, takes up your weekends, and can be emotionally exhausting.

The researchers want to answer a very specific question: How much better does the "Guided Hike" have to be than the "Easy Path" before you decide it's worth the trouble?

If the guide only helps 3 out of 10 people reach the top (the same as walking alone), nobody would pay for the guide. But if the guide helps 9 out of 10 people, everyone would sign up. The researchers are trying to find that "tipping point"—the exact moment where the extra effort and cost become worth it.

What They Are Measuring: The "Smallest Worthwhile Difference" (SWD)

In the medical world, this tipping point is called the Smallest Worthwhile Difference (SWD).

Think of it like buying a new video game.

• The Cost: You have to pay $70 and spend 50 hours playing it.
• The Alternative: You could just watch a movie for free.
• The Question: How much more fun does the game have to be than the movie for you to say, "Okay, I'll spend the money and the time"?

For people with Alcohol Use Disorder (AUD), the "game" is therapy. The "movie" is trying to quit drinking on your own. The study wants to know: What is the minimum success rate therapy needs to have for a patient to say, "Yes, I'll do the therapy"?

How They Are Doing It: The "Trade-Off Game"

The researchers aren't just asking people, "Do you like therapy?" Instead, they are playing a game of "What If?" with thousands of people (mostly Americans).

They use a method called the Benefit-Harm Trade-Off. Here is how the conversation goes:

1. The Setup: "Imagine 100 people try to quit drinking on their own. About 30 of them succeed. Now, imagine you join a therapy group. It costs money and takes time."
2. The First Guess: "If therapy helped 100% of the people succeed, would you do it?" (Most people say yes).
3. The Lowering: "Okay, what if it only helped 90%?" (Still yes).
4. The Tipping Point: They keep lowering the number (80%, 70%, 60%...) until the person finally says, "No, that's not good enough to bother with the cost and hassle."

That final number is their SWD. If someone says, "I only want therapy if it helps at least 60% of people," then the "Smallest Worthwhile Difference" for them is 30% (60% therapy success minus the 30% natural success).

Who Are They Asking?

They are asking three different groups of people, because everyone sees the "hike" differently:

1. The Hikers (Patients): People who drink too much but have never tried therapy before. They are the ones who have to carry the backpack.
2. The Guides (Therapists): The doctors and counselors. They want to know: "How much proof do I need before I recommend this expensive, time-consuming path to my patient?"
3. The Judges (Criminal Justice): Judges and probation officers often force people into therapy. They need to know: "Is this therapy actually worth ordering someone to do, or should we just let them try on their own?"

Why Does This Matter?

Right now, doctors and judges are guessing. They might say, "Therapy is great!" and recommend it. But if the patient thinks, "That's too much work for such a small chance of getting better," they might quit halfway through.

By finding the SWD, the researchers hope to create a "rule of thumb" for decision-making.

• If a new therapy only helps 35% of people (just barely better than doing nothing), the study might show that patients won't find it "worth it."
• If a therapy helps 50% of people, it might be the perfect balance.

The Bottom Line

This study is like a price tag calculator for hope. It's trying to figure out exactly how much "better" therapy needs to be than doing nothing, so that patients, doctors, and judges can all agree on when it's time to start the hard work of treatment.

They expect to find that different people have different "price tags." A person who has been through therapy before might be willing to pay a higher "price" (accept a lower success rate) because they know the value, while someone new to it might need a much higher guarantee of success before they sign up.

Technical Summary

Based on the provided preprint, here is a detailed technical summary of the study protocol:

1. Problem Statement

Alcohol Use Disorder (AUD) is a prevalent condition with significant morbidity, yet the majority of affected individuals remain untreated. While psychotherapy is a proven, NIAAA-recommended treatment, its adoption is hindered by patient-perceived burdens (cost, time, inconvenience) and a lack of data regarding the Smallest Worthwhile Difference (SWD).

Current metrics like the Minimal Important Change (MIC) are insufficient for this context because:

• They measure within-person change (pre- to post-treatment) rather than between-person differences.
• They are specific to outcome tools and not tied to specific interventions.
• They fail to account for the trade-off between treatment benefits and burdens (harms, expenses, inconveniences) relative to alternatives (e.g., natural recovery).

There is a critical gap in understanding the threshold at which patients, providers, and criminal justice professionals deem the benefits of AUD psychotherapy to outweigh its burdens compared to natural recovery.

2. Methodology

The study employs a cross-sectional survey design utilizing the Benefit-Harm Trade-off Method (BHTM) to estimate the SWD and the Smallest Recommendable Difference (SRD).

• Study Design & Recruitment:

  • Primary Sample: English-speaking US residents (18+) recruited via Prolific (crowdsourcing). The primary focus is on individuals with moderate-to-severe AUD symptoms (AUDIT ≥ 8) who have never received treatment.
  • Secondary Samples:
    • Psychotherapy Providers: Recruited via APA listservs.
    • Criminal Justice Professionals: (Judges, probation officers) recruited via ABA, federal court systems, and government websites.
  • Subgroups: The primary analysis will stratify participants by treatment history and symptom severity (Risky drinking/never treated, risky drinking/previous treatment, low-risk/previous treatment, low-risk/never treated).

• Intervention & Comparator:

  • Intervention: NIAAA-recommended psychotherapies (CBT, Brief Intervention, DBT, MBRP, etc.).
  • Comparator: Natural recovery (no treatment).
  • Baseline Control Event Rate (CER): The study assumes a 30% natural recovery rate within one year, derived from a synthesis of epidemiological studies and RCTs.

• The BHTM Algorithm (SWD Estimation):

  • Participants are presented with a clinical scenario detailing the burdens of psychotherapy (risks, duration, cost, wait times).
  • They are asked if they would find the treatment "worthwhile" at varying hypothetical recovery rates (Experimental Event Rates or EERs) compared to the 30% natural recovery baseline.
  • Iterative Process: The algorithm adjusts the hypothetical recovery rate up or down based on participant responses (Yes/No) until the smallest recovery rate is identified where the participant still deems the treatment worthwhile.
  • Calculation: $SWD = (\text{Smallest Worthwhile Recovery Rate}) - 30\% (\text{Natural Recovery})$.
  • SRD Calculation: Providers and criminal justice professionals answer similar questions regarding whether they would recommend or mandate the treatment.

• Data Analysis:

  • Primary outcome: Median and Interquartile Range (IQR) of SWD for the "Risky Drinking but Never Treated" group.
  • Statistical comparisons: Kruskal-Wallis test and Dunn's post-hoc test for subgroup differences.
  • Exploratory analysis: Multivariable regression to identify demographic and clinical predictors of SWD/SRD.
  • Sensitivity analyses: Including "burden-sensitive decliners" (those rejecting 100% efficacy) and excluding those with prior non-AUD psychotherapy.

3. Key Contributions

• Novel Metric Application: This is the first study to apply the Benefit-Harm Trade-off Method (BHTM) specifically to estimate the SWD for AUD psychotherapy.
• Stakeholder Diversity: Unlike previous studies focusing solely on patients, this protocol explicitly includes providers and criminal justice professionals to determine the Smallest Recommendable Difference (SRD), addressing the unique decision-making power of third parties in mandated treatment.
• Patient-Centered Benchmarking: It moves beyond statistical significance to establish a patient-perceived threshold for "worthwhile" treatment, facilitating shared decision-making.
• Subgroup Granularity: The study plans to analyze how SWD varies based on treatment history (naïve vs. experienced) and current symptom severity, providing nuanced data for clinical triage.

4. Expected Results

• Quantitative Estimates: The study anticipates producing a specific numerical estimate for the SWD (e.g., a specific percentage point increase over the 30% baseline) required for patients to accept psychotherapy.
• Heterogeneity: The authors expect significant variance in SWD values across subgroups (e.g., treatment-naïve vs. previously treated) and between stakeholder groups (patients vs. providers vs. criminal justice).
• Predictors: The regression models aim to identify which demographic or clinical factors (e.g., insurance status, severity of AUDIT scores) most strongly influence the willingness to accept treatment burdens.
• Qualitative Insights: Open-ended responses from "burden-sensitive decliners" (who reject treatment even at 100% efficacy) and "burden-tolerant accepters" will provide qualitative context on decision-making drivers.

5. Significance

• Clinical Practice: The findings will provide an evidence-based benchmark for clinicians to discuss treatment expectations with patients, helping to align treatment goals with patient values and tolerance for burden.
• Trial Design: The SWD estimate can serve as a target for future Randomized Controlled Trials (RCTs) to determine if a new intervention meets the threshold of clinical meaningfulness.
• Policy & Mandates: By quantifying the SRD for criminal justice professionals, the study can inform policies regarding mandated addiction treatment, ensuring that court-ordered therapies meet a threshold of perceived efficacy and worthiness.
• Shared Decision-Making: The results will enhance the framework for shared decision-making by explicitly quantifying the "benefit-burden" trade-off, moving beyond abstract efficacy rates to tangible patient preferences.

Note: As this is a protocol paper posted on medRxiv, it outlines the plan for the research. The actual data collection, analysis, and final results have not yet been reported in this document.