Reassessing Instrument Strength in Two-Sample Mendelian Randomization Analysis

Technical Summary: Reassessing Instrument Strength in Two-Sample Mendelian Randomization Analysis

Problem Statement
Two-sample Mendelian randomization (MR) is a standard approach for estimating causal effects using genetic variants as instrumental variables (IVs). A critical methodological decision in this framework is the selection threshold for IVs. While the standard practice employs a stringent genome-wide significance threshold ($p < 5 \times 10^{-8}$) to minimize weak instrument bias and assumption violations, this strategy may limit applicability in contexts with underpowered Genome-Wide Association Studies (GWAS), such as those involving underrepresented populations or specific demographic biases. Relaxing the threshold to include weaker instruments (e.g., $p < 10^{-5}$) increases the number of available IVs but risks introducing weak instrument bias and horizontal pleiotropy, potentially attenuating effect estimates toward the null. The extent to which these risks manifest in real-world data versus controlled simulations remains an open empirical question.

Methodology
The authors employed a dual approach combining extensive simulation studies and real-data analyses to evaluate the impact of varying IV selection thresholds.

• Simulation Studies: Using the sim1000G R package, the authors simulated genotype data for 30,000 individuals based on the 1000 Genomes Project European reference panel. They generated continuous exposures and outcomes with varying parameters:

  • Exposure GWAS sample sizes ($N_{exposure}$): 10,000 and 20,000.
  • Number of causal variants ($p_{causal}$): 10 and 100.
  • Exposure heritability ($h^2$): 0.4 and 0.6.
  • Pleiotropy: Scenarios with 30% of variants having non-zero pleiotropic effects and scenarios with no pleiotropy.
  • The study compared "oracle" MR (using known true causal variants) against "non-oracle" MR (selecting variants based on p-value thresholds ranging from $5 \times 10^{-8}$ to $10^{-5}$).
  • Estimation was performed using MR-RAPS as the primary method, with comparisons to IVW, MR-Egger, and penalized robust variants.

• Real-Data Analysis: The authors analyzed six cardiovascular risk factors (exposures) against four aging-related chronic disease outcomes.

  • Exposures: Type 2 diabetes (T2D), hypertension (HTN), obesity, sleep disorder, chronic kidney disease (CKD), and lipoprotein metabolism disorders (LpA).
  • Outcomes: Atrial fibrillation (AF), coronary artery disease (CAD), stroke, and Alzheimer's disease (Alz).
  • Data Sources: Exposure GWAS summary statistics were drawn from two FinnGen releases: Release 3 (135k individuals, representing a smaller sample size) and Release 12 (480k individuals, representing a larger sample size). Outcome GWAS data were sourced from the Pan-UK Biobank and the International Genomics of Alzheimer's Project (IGAP).
  • Analysis Strategy: The authors applied p-value thresholds of $5 \times 10^{-8}$, $10^{-7}$, $10^{-6}$, and $10^{-5}$ for IV selection. They also conducted secondary analyses using non-overlapping p-value intervals to isolate the effect of weak instruments and performed multivariate MR (MVMR) to adjust for obesity.

Key Results

• Simulation Findings: In simulated settings, the inclusion of weaker instruments (relaxed p-value thresholds) generally produced causal effect estimates similar to those obtained with stringent thresholds. The direction and variability of estimates remained stable across different sample sizes and degrees of pleiotropy, with the primary source of increased variability being the presence of pleiotropy itself rather than the inclusion of weak IVs.
• Real-Data Findings: The empirical results diverged from the simulations.
  • Sample Size Dependence: When using the larger FinnGen Release 12, relaxed thresholds generally maintained consistent effect directions and statistical significance conclusions compared to the stringent threshold. However, when using the smaller FinnGen Release 3, the inclusion of weak IVs led to a pronounced attenuation of effect estimates toward the null.
  • Confidence Intervals: In the smaller sample size setting, relaxed thresholds often resulted in narrow confidence intervals that did not overlap with estimates derived from stringent thresholds, increasing the risk of falsely concluding a null association.
  • Specific Anomalies: While most associations remained consistent, the authors noted a specific case where including weak instruments for T2D on AF shifted the estimate from non-significant to statistically significant (OR=1.04), a finding supported by MR-PRESSO.
  • Pleiotropy: Comparisons between MR-PRESSO and MR-RAPS suggested no substantial increase in horizontal pleiotropy when relaxing thresholds, implying that the observed attenuation in real data may stem from factors other than simple pleiotropy, such as complex causal structures or ill-defined exposures.

Significance and Claims
The paper concludes that while simulation studies suggest the robustness of MR to relaxed IV selection thresholds, real-world applications are more sensitive, particularly when the exposure GWAS sample size is small. The authors claim that:

1. Contextual Risk: Incorporating weak IVs is reasonable when the exposure GWAS is sufficiently powered (large sample size), as it may improve power without substantially biasing results.
2. Caution in Underpowered Settings: In settings with small exposure GWAS sample sizes, relaxing p-value thresholds poses a significant risk of attenuating effect estimates toward the null, potentially leading to false conclusions of no causal association.
3. Methodological Recommendation: The authors suggest that when applying liberal IV selection thresholds in underpowered studies, researchers must accompany primary findings with sensitivity analyses across multiple thresholds and robust MR methods.
4. Interpretative Shift: The authors advocate for interpreting MR estimates not as precise causal effect sizes but as evidence supporting or rejecting a causal hypothesis. From this perspective, consistency in the direction of effect and statistical inference across thresholds is more critical than the precise magnitude of the point estimate.

The study does not propose new statistical methods but rather provides empirical guidance on the trade-offs of instrument selection strategies, highlighting that the "weak instrument bias" observed in real data may be compounded by complexities (e.g., exposure definition, residual confounding) not fully captured in standard simulation frameworks.