Remote, bivariate expert elicitation to determine the prior probability distribution for sample size calculation in a Bayesian non-inferiority multicenter randomized controlled trial (Croup Dosing Trial)

This paper demonstrates the feasibility of using remote, bivariate expert elicitation to construct prior probability distributions for sample size calculation in the Croup Dosing Trial, successfully aggregating the beliefs of twelve emergency medicine physicians to determine a required sample size of 1,850 participants for assessing the non-inferiority of a lower dexamethasone dose.

The Gist

Imagine you are planning a massive, high-stakes cooking competition to see if a small pinch of salt works just as well as a heaping spoonful to make a soup taste good. You want to prove the small pinch is "not worse" than the big spoonful, so you can save money and reduce waste. But before you start cooking, you need to know: How many contestants do you need to enter to be sure your results are real?

If you guess the number wrong, you might waste years of time and millions of dollars, or worse, you might miss the truth entirely.

This paper is about how a team of researchers solved this "guessing game" for a real medical trial involving croup (a scary-sounding cough in kids) and dexamethasone (a steroid medicine). Here is the story of how they did it, explained simply.

The Problem: The "Blind" Trial

The researchers wanted to test if a low dose of medicine (0.15 mg/kg) worked just as well as the standard high dose (0.60 mg/kg) for treating croup.

• The Goal: Prove the low dose isn't "bad enough" to cause more kids to come back to the Emergency Room (ED) within a week.
• The Hurdle: To design a proper experiment, they needed a "map" of what to expect. In statistics, this map is called a Prior Distribution. It's basically a "best guess" based on what we already know, before the new trial starts.
• The Dilemma: There wasn't enough past data to draw a perfect map. The old studies were too few or too different. So, they had to ask the experts: "What do you think will happen?"

The Solution: The "Remote Taste-Test"

Usually, asking experts for their opinions involves flying them all to one room, sitting them around a table, and having long, intense meetings. This is expensive and hard to schedule.

Instead, the researchers used a Remote, Digital Workshop. Think of it like a virtual "Taste-Test" Zoom call.

1. The Experts: They gathered 12 top doctors from Canada, the US, Australia, and New Zealand. These were the "Master Chefs" of pediatric emergency rooms.
2. The Setup: They sent the doctors a "recipe card" (a case study of a sick child) and asked them to predict: "Out of 100 kids like this, how many will come back to the ER if we give them the low dose? How many if we give the high dose?"
3. The Twist (The Bivariate Part): Usually, people guess these numbers separately. But the researchers knew that if a doctor thinks the low dose is risky, they probably think the high dose is safe, and vice versa. It's like guessing the weather: if you think it will rain heavily, you probably don't think it will be sunny.
   • They used a special statistical trick (a bivariate distribution) to link these two guesses together, acknowledging that the doctors' minds were connected.
4. The Discussion (The "Round 2"):
   • Round 1: Doctors gave their private guesses.
   • The Chat: The researchers showed everyone a "cloud" of guesses (anonymized) and asked, "Why did you guess 10% while your neighbor guessed 20%?"
   • Round 2: After hearing each other's logic (e.g., "I guessed high because winter viruses are stronger this year"), the doctors adjusted their guesses. The group moved closer together, like a flock of birds finding a formation.

The Result: The "Magic Number"

After the digital workshop, the team combined all the doctors' adjusted opinions into a single, super-smart "map."

• The Prediction: They estimated that about 6% of kids on the high dose would return to the ER, and 8% on the low dose would return.
• The Sample Size: Using this map, they calculated that they needed 1,850 children in the trial to be 80% sure they would find the right answer.

Why This Matters (The Analogy)

Imagine trying to build a bridge.

• Old Way: You look at three old, blurry photos of bridges and guess how much steel you need. You might build a bridge that collapses or one that costs a billion dollars too much.
• This Paper's Way: You call 12 of the world's best bridge engineers. You show them the blueprints. You let them argue about the wind and the soil. You listen to their collective wisdom. Then, you build a bridge that is perfectly sized for the job.

The Takeaway

This paper proves that you don't need to be in the same room to get brilliant expert advice. By using remote technology and smart statistics, they successfully gathered the "collective brainpower" of doctors from four different countries.

They created a reliable "best guess" that will help them run a fair, efficient, and life-saving trial to see if kids can get away with a smaller dose of medicine. It's a win for science, a win for the budget, and most importantly, a win for the families of kids with croup.

Technical Summary

1. Problem Statement

Context: Bayesian clinical trials require the specification of prior probability distributions for parameters of interest. While historical data is often used, it may be limited, inconclusive, or non-existent for specific trial designs. In such cases, expert elicitation is used to quantify uncertainty.
Challenges:

• Logistical Barriers: Traditional expert elicitation requires face-to-face interaction and intensive pre-elicitation training, which is often infeasible due to geographical dispersion, time zone differences, and cost.
• Statistical Dependencies: Conventional elicitation often assumes independence between treatment arms. However, experts often use "anchoring," where their estimate for one arm influences the other. Ignoring this correlation can lead to biased priors.
• Specific Application: The Croup Dosing Trial aims to test the non-inferiority of a lower dose of dexamethasone (0.15 mg/kg) versus the standard dose (0.60 mg/kg) for treating croup in children. The primary outcome is the rate of return visits to the Emergency Department (ED) within 7 days. A robust prior distribution was needed to calculate the Bayesian sample size, but existing literature was sparse and potentially imprecise.

2. Methodology

The study employed a remote, bivariate expert elicitation framework based on the IDEA protocol (Investigate, Discuss, Estimate, Aggregate), adapted for a distributed, international setting.

A. Expert Selection and Recruitment:

• Target: 10–15 experts in pediatric emergency medicine and croup treatment from Canada, the USA, Australia, and New Zealand.
• Criteria: Experts must have experience in pediatric emergency medicine and croup treatment but no prior involvement in the Croup Dosing Trial or authorship of studies favoring specific doses.
• Outcome: 12 clinicians participated (8 from Canada, 4 from the USA).

B. Elicitation Process:

• Preparation: Experts reviewed a case study (a 1-year-old with mild-moderate croup) and provided literature (one clinical trial and one meta-analysis) regarding dexamethasone efficacy.
• Remote Workshops: Three 90-minute workshops were conducted via Zoom to accommodate time zones.
• Tooling: An interactive Shiny (R) application was used. Experts provided:
  • Lower and upper plausible values (assumed to be the 95% central credible interval).
  • Most plausible value (assumed to be the mode).
  • These values were fitted to Beta distributions using the betaExpert package.
• Protocol Structure (IDEA Adaptation):
  1. Round 1: Individual estimation of probabilities ($p_1$ for 0.60 mg/kg, $p_2$ for 0.15 mg/kg).
  2. Discussion: De-identified boxplots of Round 1 results were shared. Experts discussed reasoning, clinical factors (e.g., seasonality, virus virulence, regional primary care access), and uncertainties.
  3. Round 2: Experts re-estimated probabilities individually after reflection.

C. Statistical Aggregation (Bivariate Approach):

• Model: Instead of aggregating marginal distributions independently, the authors used a bivariate prior distribution with latent expert effects (based on Thall et al.).
• Mechanism: This approach models the correlation between the two treatment arms ($p_1$ and $p_2$) for each expert. It accounts for the fact that an expert's estimate for one dose likely informs their estimate for the other.
• Covariates: Physician experience and training were included as covariates to explain variability.
• Weighting: Each expert's bivariate prior was weighted evenly in the final aggregate.
• Sample Size Calculation: The aggregated prior was used to determine the sample size via assurance (probability of trial success) using simulation in R (INLA). The goal was 80% relative assurance and 5% assurance under the null hypothesis.

3. Key Contributions

• Remote Feasibility: Demonstrated that high-quality, bivariate expert elicitation can be successfully conducted remotely across multiple time zones without face-to-face interaction, overcoming logistical barriers in multinational trials.
• Bivariate Modeling: Moved beyond independent univariate priors by explicitly modeling the dependence (correlation) between treatment arms using latent effects. This addresses the "anchoring" bias common in expert judgment.
• Integration of Evidence: Successfully combined empirical literature with expert opinion to create a prior that reflects both data and clinical context (e.g., regional differences in healthcare access).
• Tool Development: Developed and validated an interactive, real-time visualization tool (Shiny) that allows experts to adjust their beliefs dynamically, improving the calibration of their subjective probabilities.

4. Results

Participant Characteristics:

• 12 experts with 11–38 years of experience.
• All had prescribed the standard dose (0.60 mg/kg) in the last year; only 2 had prescribed the lower dose (0.15 mg/kg).
• Most had training in clinical trial methodology and statistics.

Elicitation Outcomes:

• Shift in Beliefs: Variability in estimates decreased from Round 1 to Round 2 following group discussions. Experts refined their understanding of plausible ranges and clinical contexts.
• Aggregated Prior Distributions (Round 2):
  • Standard Dose (0.60 mg/kg, $p_1$): Median = 6.44%, Mean = 8.01%.
  • Lower Dose (0.15 mg/kg, $p_2$): Median = 8.19%, Mean = 1.00% (Note: Table 3 lists Mean 1.00% but Median 8.19%; the text summary clarifies the median is 8.2%).
  • Risk Difference ($p_2 - p_1$): Median = 1.52%.
• Correlation: The posterior estimates showed a moderate positive correlation (0.40) between the two arms, validating the necessity of the bivariate approach.
• Sample Size Determination:
  • Non-inferiority margin: 4% (derived from a separate survey of 60 physicians).
  • Required Sample Size: 1,850 participants (providing 80% relative assurance).

Validation:

• The elicited probabilities (6.4% vs. 8.2%) were consistent with a previous study by Parker and Cooper (5.9% vs. 8.8%).
• The risk difference aligned with a meta-analysis, despite the meta-analysis including family doctor visits (which would inflate rates).

5. Significance

• Trial Design Impact: The elicited prior directly informed the sample size calculation for the Croup Dosing Trial (NCT06272383), ensuring the trial is adequately powered based on realistic expert consensus rather than arbitrary assumptions.
• Methodological Advancement: This study provides a blueprint for conducting complex, dependent (bivariate) elicitation in a remote setting. It proves that remote workshops can yield robust priors comparable to, or more representative than, those derived from limited historical data.
• Clinical Relevance: By incorporating diverse geographical perspectives (Canada, USA, Australia, NZ), the prior distribution is more generalizable to the multinational trial population, accounting for variations in healthcare systems and patient management.
• Future Directions: The authors highlight the need for future studies to include more extensive training on bias and subjective probability, and to document the rationale behind opinion shifts to mitigate "groupthink" risks.

In conclusion, this paper successfully demonstrates a rigorous, scalable methodology for deriving bivariate priors for Bayesian non-inferiority trials, bridging the gap between expert clinical judgment and statistical trial design in a remote, multinational context.