Estimating the impact of Shigella vaccines on growth outcomes and implications for clinical trial design

This paper demonstrates that while standard randomized trials are underpowered to detect Shigella vaccine impacts on linear growth due to small effect sizes, focusing analysis on the "naturally infected" subgroup and optimizing trial design significantly increases statistical power and reduces the risk of misleading null or inverse results.

The Gist

The Big Picture: A Race Against a Tiny Villain

Imagine Shigella as a tiny, invisible villain that attacks young children in many parts of the world. It causes severe diarrhea. While we can treat it with medicine, the bacteria are becoming harder to kill (antibiotic resistance), so scientists are trying to build a vaccine to stop it before it starts.

We know this villain doesn't just make kids sick for a few days; it steals their future. When kids get Shigella, they often stop growing as tall as they should. This is called "stunting," and it can affect their health and learning for the rest of their lives.

The big question for the scientists is: "If we give kids this new vaccine, will it help them grow taller?"

The Problem: The "Needle in a Haystack" Search

The researchers realized that trying to answer this question in a standard clinical trial is like trying to find a needle in a haystack, but with a twist: The needle is moving, and the haystack is huge.

Here is why standard trials fail at this specific task:

1. Most kids won't get sick: In a trial of 10,000 kids, maybe only a few hundred will actually catch Shigella.
2. The "Uninfected" crowd: The other 9,000+ kids won't get sick anyway, whether they get the vaccine or a fake placebo. Their growth won't change.
3. Dilution: If you compare the average height of the whole group (vaccine vs. placebo), the massive group of healthy kids "dilutes" the results. It's like trying to hear a whisper in a stadium full of people shouting. The signal (the vaccine helping the sick kids) gets lost in the noise (the healthy kids).

Because of this, a standard trial might conclude, "The vaccine did nothing," even if it actually saved the growth of the few kids who got sick. Or worse, random chance might make it look like the vaccine hurt growth.

The Solution: The "Naturally Infected" Super-Group

The authors propose a clever new way to look at the data. Instead of looking at the whole stadium, they suggest focusing only on the specific people who would have gotten sick if they hadn't been vaccinated. They call this group the "Naturally Infected."

The Analogy:
Imagine a fire drill.

• Standard Approach: You measure the average running speed of everyone in the building (including people who didn't need to run because they were on the top floor). You conclude the fire drill didn't make anyone run faster.
• New Approach: You only measure the speed of the people who actually had to run down the stairs. You see that the fire drill (the vaccine) helped them run much faster and safer.

By using advanced math (causal inference) to estimate who would have gotten sick, the researchers found that the vaccine's effect on growth looks 5 to 10 times stronger when you focus on this specific group. It's like turning up the volume on that whisper so you can finally hear it.

The Experiment: Testing Different Scenarios

The researchers ran thousands of computer simulations to see how to design the best possible trial. They tested three main variables:

1. When to Vaccinate: Should we vaccinate kids at 6 months old or wait until 12 months?
   • Finding: Vaccinating later (12 months) actually gave better results in the trial. Why? Because older kids in these simulations had a higher chance of getting sick during the study, creating a bigger "Naturally Infected" group to study.
2. Where to Recruit: Should we pick kids randomly, or target the ones most at risk?
   • Finding: Targeted recruitment is key. If you go to a neighborhood where Shigella is very common and pick kids who are already struggling with growth, you get a much clearer picture of the vaccine's power. It's like fishing in a pond full of fish rather than an empty lake.
3. When to Measure Height: Should we measure the kids halfway through the year and at the end?
   • Finding: Surprisingly, measuring them only at the end of the year was better. Measuring them halfway through didn't add enough new information to justify the extra cost and effort. The full effect of the infection (and the vaccine's protection) takes time to show up.

The Hard Truth: We Need a Bigger Net

Even with these clever tricks, the researchers found a sobering reality: Standard Phase 3 trials are likely too small to prove the vaccine helps growth.

To get a definitive "Yes, this vaccine helps kids grow," you would need a trial with 80,000 children. Most vaccine trials only have 2,500 to 20,000.

The Conclusion:

• Don't give up: The vaccine is still valuable.
• Change the strategy: Use the "Naturally Infected" math trick to get the best possible data from smaller trials.
• Plan for the future: We probably won't know for sure if the vaccine helps growth until after it is approved and used in the real world (post-marketing studies). We need to keep watching the kids closely once the vaccine is out there.

Summary in One Sentence

The paper argues that to prove a Shigella vaccine helps children grow, we can't just look at everyone; we must use smart math to focus on the kids who would have gotten sick, and even then, we might need to wait until the vaccine is widely used to see the full benefit.

Technical Summary

1. Problem Statement

Shigella is a leading cause of diarrheal disease and mortality in children in low- and middle-income countries, significantly contributing to long-term linear growth faltering (stunting). While Shigella vaccines are in development, evaluating their impact on growth is methodologically challenging for Phase 3 clinical trials:

• Low Incidence & Dilution: Shigella incidence is relatively low. In a standard randomized controlled trial (RCT), the majority of children will not contract Shigella. Since uninfected children cannot experience Shigella-attributable growth faltering, comparing Height-for-Age Z-scores (HAZ) between vaccine and placebo arms across the entire population dilutes the vaccine effect, leading to extremely low statistical power.
• Risk of Null or Inverse Results: Due to this dilution and random chance, standard analyses may fail to detect a true benefit (Type II error) or, more dangerously, yield statistically significant "inverse" results suggesting the vaccine harms growth, even when it does not.
• Selection Bias: Simply restricting analysis to children who actually became infected introduces selection bias, as it excludes children whom the vaccine successfully protected (the primary beneficiaries).

2. Methodology

The authors employed a causal inference approach combined with realistic simulation studies to evaluate statistical power under various trial designs.

• Causal Framework (The "Naturally Infected" Estimator):
  Instead of comparing all randomized participants, the study estimates the vaccine effect in the subgroup of children who would have been infected with Shigella in the absence of vaccination (the "naturally infected"). This approach uses Augmented Inverse Probability Weighted (AIPW) estimators to adjust for baseline covariates (specifically baseline HAZ) that influence both infection risk and growth. This isolates the biological impact of the vaccine on growth among those at risk.

• Simulation Design:

  • Data Sources: Parameters were derived from multi-site observational studies, specifically the Enterics for Global Health (EFGH) Shigella surveillance study (for incidence) and the MAL-ED birth cohort (for growth trajectories and infection effects).
  • Scenarios: The authors simulated 1:1 randomized, placebo-controlled trials with 12 months of follow-up. They varied three key design factors:
    1. Immunization Schedule: Complete protection by 6 months vs. 12 months.
    2. Recruitment Strategy:
       • General: Average incidence settings.
       • Targeted: Enrolling children at highest risk (using upper confidence bounds of incidence and larger growth effect estimates).
       • High Early Incidence: Sites with disproportionately high infection rates in the 6–12 month age group.
    3. Measurement Timing: Single endpoint at 12 months vs. interim measurements at 6 months vs. averaging both.
  • Sample Sizes: Simulations were run for $N = 2,500$ to $80,000$.
  • Primary Endpoint: PCR-attributable moderate-to-severe Shigella diarrhea (used to calculate sample sizes for 90% power on the primary endpoint).
  • Secondary Endpoint: Change in HAZ.

3. Key Contributions

• Methodological Innovation: The paper validates the use of the "naturally infected" causal estimand for growth outcomes in vaccine trials, demonstrating it is superior to population-level intent-to-treat (ITT) comparisons for detecting growth effects.
• Quantification of Power Deficits: The study quantifies that standard Phase 3 trial sizes (typically 2,500–20,000) are grossly underpowered to detect growth effects, even with optimized designs.
• Design Optimization: It provides specific guidance on how to maximize power through recruitment strategies (targeting high-risk groups) and immunization timing, while debunking the utility of interim growth measurements.

4. Key Results

• Effect Size & Power:

  • The estimated vaccine effect on HAZ in the naturally infected subgroup was 5 to 10 times larger than the population-level effect.
  • Power: At a realistic trial size of $N=20,000$, power to detect the effect in the naturally infected was only 11% (vs. 6% for population-level). Even at $N=80,000$, power reached only 25% for the naturally infected approach.
  • Sample Size Requirement: To achieve 80% power for the growth endpoint, a sample size of approximately 80,000 is required, which is likely unfeasible for a Phase 3 trial.

• Risk of Negative Estimates:

  • Population-level analyses had a high probability (43% at $N=20,000$) of yielding negative (harmful) point estimates due to random noise.
  • The naturally infected approach reduced this risk significantly (26% negative estimates) and minimized the chance of statistically significant false-positive harm claims.

• Trial Design Findings:

  • Recruitment: Targeted recruitment (enrolling high-risk children) significantly improved power (e.g., 67% power at $N=40,000$ in targeted vs. 27% in general recruitment) and reduced negative estimates.
  • Immunization Schedule: While earlier schedules (protection by 6 months) yielded slightly larger effect sizes in the naturally infected, later schedules (protection by 12 months) generally provided higher statistical power. This is because older children in many sites have higher incidence rates, leading to a larger "naturally infected" subgroup.
  • Measurement Timing: Adding interim measurements (e.g., at 6 months) did not improve power compared to a single 12-month endpoint. This is likely due to the time lag required for severe infections to manifest maximal growth deficits and the correlation between repeated measures.

5. Significance and Implications

• Clinical Trial Design: The study concludes that including linear growth as a secondary endpoint in standard Phase 3 Shigella vaccine trials is likely to be underpowered and prone to misleading results.
• Strategic Recommendations:
  • If growth is measured, trials must use the naturally infected causal estimand rather than simple ITT comparisons.
  • Trials should prioritize targeted recruitment in high-incidence settings to maximize the size of the naturally infected subgroup.
  • Interim growth measurements should be avoided as they add cost without statistical benefit.
• Policy Impact: Given the difficulty of detecting growth effects in Phase 3, the authors recommend that the full evaluation of Shigella vaccine impact on growth should be deferred to post-marketing (Phase 4) studies, where larger sample sizes and longer follow-up can provide well-powered estimates. This prevents the risk of prematurely discarding a beneficial vaccine based on a null or falsely negative growth result in a Phase 3 trial.