Comparison of methods for assessing effects of risk factors on disease progression in Mendelian randomization under index event bias

This paper evaluates statistical methods for mitigating index event bias in Mendelian randomization studies of disease progression, finding that while no single approach is universally effective, a strategic framework based on data availability and biological context can guide method selection.

The Gist

The Big Picture: The "Survivor" Problem

Imagine you are a detective trying to figure out why some people get sick and stay sick, while others get sick and recover quickly. You want to know if a specific habit (like eating too much sugar) causes the sickness to get worse.

To solve this, you decide to look at a group of people who already have the disease. You compare those who got very sick to those who got better.

Here is the trap: By only looking at people who already have the disease, you have accidentally created a biased group. You've filtered out everyone who was so healthy they never got sick in the first place.

In the world of genetics, this is called Index Event Bias. It's like trying to figure out why some cars break down by only looking at cars that are currently in the repair shop. You might conclude that "red cars break down more often," but that's only because the red cars that didn't break down are still driving around on the highway, and you didn't count them.

This paper asks: How do we fix our detective work when we are forced to only look at the "repair shop" (the sick people)?

---

The Tools in the Detective's Kit

The authors tested five different "tools" (statistical methods) to see which one could fix this bias. They ran thousands of computer simulations to see which tool worked best.

1. The "Re-Weighting" Scale (Inverse-Probability Weighting)

• The Analogy: Imagine you have a bag of marbles, but you only see the red ones because the blue ones fell out of the bag. To fix this, you take every red marble you see and say, "Okay, for every red marble I see, I'll pretend there are actually 10 red marbles in the whole bag." You are artificially inflating the weight of the people you do see to represent the people you don't see.
• The Result: This works pretty well, BUT it requires you to have the original, full list of everyone (individual-level data). If you only have a summary report (like a news headline), you can't use this tool. Also, if your guess about how many blue marbles fell out is wrong, your whole calculation is wrong.

2. The "Magic Mirror" (Heckman's Method)

• The Analogy: This method tries to use a "magic mirror" (a special genetic clue) that tells you who would have gotten sick but didn't. It tries to reconstruct the missing people.
• The Result: In this study, the mirror was a bit foggy. It struggled to give clear answers, especially when the data was complex. It's a bit too rigid for this specific job.

3. The "Slope Hunter" (Slope-Hunter)

• The Analogy: Imagine you are looking at a hill. You know some people are sliding down just because of gravity (the disease event), and some are sliding because they were pushed (the risk factor). This method tries to find the "slope" of the hill by looking at a huge crowd of people and guessing which ones are just sliding naturally.
• The Result: This tool failed miserably. In the simulations, it kept making things worse. It was like trying to find a needle in a haystack by throwing the whole haystack at the wall. It created a lot of false alarms.

4. The "Two-Pronged Fork" (Multivariable Methods)

• The Analogy: Instead of just looking at the "sick" group, this method tries to look at two things at once: "Who got sick?" AND "How sick did they get?" It uses a special genetic fork to separate the people who got sick just by bad luck from those who got sick because of the risk factor.
• The Result: This was the best tool, but with a catch. It only works if you have a special set of genetic clues that affect getting sick but don't affect how sick you get. If the same genetic clues affect both, the fork gets stuck. It's like trying to separate salt from sugar when they are already mixed in the same shaker.

5. The "Super-Fork" (CWBLS)

• The Analogy: This is a fancy version of the Two-Pronged Fork designed to handle weak clues.
• The Result: It worked well, but sometimes it was a bit too cautious, missing real effects.

---

The Real-World Test: COVID-19

The authors didn't just play with computer simulations; they tested these tools on real data about COVID-19. They asked:

1. Does being overweight (BMI) make you more likely to catch COVID?
2. Does being overweight make the disease worse if you already have it?

• The Finding: When they looked at people who were hospitalized (the "repair shop"), the bias made it look like being overweight was less dangerous than it actually was.
• The Fix: The "Two-Pronged Fork" (Multivariable) and the "Re-Weighting Scale" (IPW) were able to correct this. They showed that yes, being overweight does make the disease worse, even though the raw data from the hospital made it look like it didn't matter as much.

However, for a different drug target (IL6R), the tools struggled because the genetic clues were "confused" (they affected both catching the virus and the severity), proving that no single tool is perfect for every situation.

---

The Final Verdict: No Silver Bullet

The main takeaway from this paper is simple: There is no magic wand.

• If you have the full list of everyone (individual data), use the Re-Weighting Scale.
• If you only have summary reports (like news headlines) and you have special genetic clues that separate "getting sick" from "getting sicker," use the Two-Pronged Fork.
• If you don't have those special clues, you might be stuck. In that case, the authors suggest: Don't try to study how the disease gets worse. Just study how people get the disease in the first place. It's simpler, less biased, and often gives you the answer you need anyway.

In short: When studying disease progression, be very careful. The "sick" group you are looking at is a filtered, biased sample. You need the right tool to fix the filter, but sometimes the best advice is to stop looking at the filter and look at the whole picture instead.

Technical Summary

1. Problem Statement

Index Event Bias (IEB) is a specific form of selection bias that occurs when Mendelian Randomization (MR) is applied to study disease progression (e.g., severity, mortality, or time-to-event outcomes) rather than disease incidence.

• Mechanism: MR relies on the random allocation of genetic variants at conception. However, analyses of disease progression are restricted to individuals who have already experienced the initial disease event (the "index event").
• Consequence: Conditioning on the index event creates a "collider" between the genetic instrument and confounders of the disease. This induces a spurious association between the genetic instrument and the progression outcome, even if the risk factor has no causal effect on progression.
• Context: This is critical for drug target MR, where the goal is to validate whether modulating a biomarker (e.g., a protein) prevents disease progression in patients who already have the disease. Standard MR methods often fail in this setting, leading to false positives or false negatives.

2. Methodology

The authors conducted a comprehensive evaluation of five statistical methods designed to mitigate IEB, comparing them via simulation studies and applied examples.

A. Methods Evaluated

1. Inverse-Probability Weighting (IPW):
   • Data: Individual-level data.
   • Mechanism: Weights individuals by the inverse of their probability of being selected (i.e., having the disease event) to reconstruct the population distribution.
   • Requirement: Requires a correctly specified model for the disease event probability.
2. Heckman's Sample Selection Method:
   • Data: Individual-level data.
   • Mechanism: Uses an instrumental variable for the selection event (disease incidence) to correct the regression model.
   • Limitation: Requires a valid instrument for the disease event and is limited to continuous or binary outcomes (not time-to-event).
3. Slope-Hunter:
   • Data: Summary-level data (GWAS).
   • Mechanism: Uses model-based clustering to identify variants affecting only incidence (not progression) to estimate the "collider bias constant." It then corrects the slope of the progression association.
   • Assumption: Assumes a homogeneous form of bias and that some variants affect incidence but not progression.
4. Multivariable Mendelian Randomization (MVMR):
   • Data: Summary-level data.
   • Mechanism: Treats the risk factor and the disease event (incidence) as two simultaneous exposures. It estimates the effect of the risk factor on progression while adjusting for the genetic risk of the disease event.
   • Requirement: Requires genetic variants that predict disease incidence but do not directly affect progression (no pleiotropy).
5. Corrected Weighted Bivariate Least Squares (CWBLS):
   • Data: Summary-level data.
   • Mechanism: An extension of MVMR designed to correct for weak instrument bias.

B. Simulation Study Design

• Setup: Simulated 100,000 individuals with genetic variants, a risk factor, a binary disease event (incidence), and a time-to-event outcome (progression).
• Scenarios: Varied instrument strength, number of instruments, confounding structures, and the presence of gene-confounder interactions.
• Metrics: Evaluated median estimates, statistical power (for true effects), and Type 1 error rates (false positives under the null).
• Applied Examples: Analyzed the effects of Body Mass Index (BMI) and IL6R inhibition (via CRP) on COVID-19 severity using UK Biobank (individual-level) and COVID-19 Host Genetics Initiative (summary-level) data.

3. Key Results

A. Simulation Findings

• Naive Analysis (Selected Sample Only): Highly inflated Type 1 error rates (up to 41.6% under the null), confirming severe bias.
• Inverse-Probability Weighting (IPW):
  • Performance: Reduced bias and maintained high power.
  • Limitation: Still showed inflated Type 1 error rates (12.7%) because the selection model was not perfectly specified (unmeasured confounders). Requires individual-level data.
• Slope-Hunter:
  • Performance: Performed poorly across all scenarios.
  • Issue: Exhibited severe Type 1 error inflation (52.7% under the null), even when assumptions were met. It failed to correct bias effectively in the applied examples.
• Multivariable Methods (MVMR & CWBLS):
  • Scenario 1 (Instrumental Variants Only): Controlled Type 1 error but had very low power (e.g., 7.7%) to detect true effects.
  • Scenario 2 (Including Disease Variants): When genetic variants predicting disease incidence were included, power reached 100% with nominal Type 1 error rates.
  • Critical Caveat: If the variants used to predict disease incidence also directly affect progression (pleiotropy), the method becomes biased.
• Heckman's Method: Produced unstable results (infinite confidence intervals) for time-to-event data; performed reasonably only with binary outcomes and valid instruments.

B. Applied Examples (COVID-19)

• BMI: Showed a positive effect on progression in all analyses. Bias attenuated the estimate but did not change the direction. IPW and MVMR did not significantly shift the estimate toward the "gold standard" (population-based) estimate.
• IL6R: The naive analysis suggested no effect on progression. The gold-standard analysis suggested a protective effect. Methods attempting to correct for IEB (Slope-Hunter, MVMR) failed to recover the gold-standard estimate, likely due to pleiotropy in the disease incidence variants.
• Individual-Level IPW: In the UK Biobank analysis, IPW corrected the estimate to be larger and statistically significant, whereas the naive analysis was non-significant.

4. Key Contributions

1. Systematic Comparison: Provides the first head-to-head comparison of IPW, Heckman, Slope-Hunter, MVMR, and CWBLS specifically for disease progression under IEB.
2. Empirical Evidence of Failure: Demonstrates that Slope-Hunter is unreliable for this specific application, showing high false-positive rates even under ideal simulation conditions.
3. Conditional Utility of MVMR: Clarifies that Multivariable MR is only effective if distinct genetic variants exist that affect disease incidence without directly affecting progression. If the same biological mechanisms drive both incidence and progression, MVMR offers little utility.
4. Strategic Framework: Proposes a decision tree for researchers based on data availability and biological context.

5. Significance and Recommendations

The paper concludes that no single method is a universal solution for IEB in MR. The authors propose the following strategic framework:

1. If the same mechanisms drive incidence and progression: Do not analyze progression. Analyze disease incidence instead, as it is free of IEB and typically has larger sample sizes.
2. If distinct variants exist for incidence: Use Multivariable MR including variants that predict disease incidence.
3. If individual-level data is available: Use Inverse-Probability Weighting, provided a robust model for disease incidence can be built.
4. If neither is possible: Perform a naive analysis but use simulations to estimate the magnitude of potential bias to interpret results cautiously.

Impact: This work is crucial for the pharmaceutical industry and epidemiologists using MR to validate drug targets for treating established diseases, preventing the pursuit of false leads caused by selection bias.