📊 statistics

Inverse probability weighting for auxiliary variable dependent sampling in observational studies of Long COVID

Motivated by the RECOVER cohort studies, this paper addresses the bias caused by ignoring auxiliary variable-dependent sampling in observational research and proposes an inverse probability weighting method to ensure valid estimation in Long COVID studies.

Original authors: Andrea S. Foulkes, Tanayott Thaweethai, Daniel O. Scharfstein, Weixing Huang, Harrison T. Reeder

Published 2026-08-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrea S. Foulkes, Tanayott Thaweethai, Daniel O. Scharfstein, Weixing Huang, Harrison T. Reeder

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery about how a specific virus affects the human body. You have a massive list of suspects (people who got sick), but you can't afford to run every single expensive test on every single person. So, you decide to be smart: you only send the expensive tests to people who show specific warning signs, like a cough or a fever. This is a common trick in science called "two-phase sampling." It saves money and time, but it creates a tricky trap. If you just look at the results from the people who got tested, you might think the disease is much worse than it really is, because you only tested the people who looked sick! To get the true picture, you need a special mathematical tool called "inverse probability weighting." Think of this tool as a magic magnifying glass that doesn't just look at the people you tested, but also imagines the people you didn't test, giving them the right amount of "weight" in your final story so the numbers balance out. This paper is about using that magic magnifying glass to solve a very real, very confusing mystery: Long COVID.

The authors of this paper, Andrea S. Foulkes and her team, are tackling a specific headache in the study of Long COVID. Long COVID is a condition where people feel sick for months or even years after their initial infection, but scientists don't fully understand why or how it works. To figure this out, researchers are running huge studies (like the RECOVER study) where they follow thousands of people. However, they can't run every single test on everyone. Instead, they use a "tiered" system. If a participant reports a specific symptom (like losing their sense of smell), they get a special, expensive test. If they don't report that symptom, they might not get the test at all, or they might get it with a much lower chance.

The problem is that this system is biased. It's like if a teacher only gave a pop quiz to students who looked confused, and then concluded that "most of the class is confused." The teacher missed the students who were actually confused but didn't look it, and the students who looked confused but were just daydreaming. In the RECOVER study, this "auxiliary variable dependent sampling" means that the people who get the tests are not a random slice of the population; they are a slice that has been filtered by their symptoms and other factors. If researchers ignore this filtering process, their conclusions about Long COVID could be completely wrong.

This paper doesn't just point out the problem; it offers a specific, step-by-step recipe to fix it. The authors describe two different ways this "tiered" sampling happens in the real world. In the "Adult" version of the study, people get repeated chances to be selected for tests over many visits. If they show a symptom today, they might get a test; if they don't, they might get another chance next month. In the "Pediatric" version, the selection happens once at the beginning, deciding which kids get to stay in the study for the long haul and get the special tests later.

The paper's main finding is a new mathematical method that combines several different "weights" to correct these biases. It's like a recipe that mixes three ingredients: the chance a person showed up for a visit, the chance they were picked for a test based on their symptoms, and the chance they actually finished the test. By using this complex weighting system, the authors show how to calculate the true average results for the whole group, not just the lucky few who got tested.

To prove their method works, the authors applied it to a real example involving the sense of smell. In the uncorrected data, 24.1% of people with Long COVID who took a smell test had a severe loss of smell. But after applying their "magic magnifying glass" (the inverse probability weighting), the number dropped to 14.6%. Why the drop? Because the study was designed to test people who already complained about smell problems. The uncorrected numbers were inflated because they over-represented the people with the worst symptoms. The corrected numbers give a much more honest estimate of how common severe smell loss really is in the entire Long COVID population.

The authors are careful to note that this is a method for analysis, not a new discovery of Long COVID itself. They aren't saying Long COVID is better or worse than we thought; they are saying, "Here is how to read the data correctly so we don't get tricked by our own study design." They suggest that without these rigorous corrections, we might draw the wrong conclusions about how the disease works. While they don't claim to have solved the entire mystery of Long COVID, they have provided a crucial tool to ensure that the clues we find in these massive studies are reliable. They argue that as science moves toward using more complex, real-world data, ignoring these sampling tricks will lead to errors, but using their approach can help scientists extract the truth from the noise.

Technical Summary: Inverse Probability Weighting for Auxiliary Variable Dependent Sampling in Observational Studies of Long COVID

Problem Statement
Observational studies, particularly those utilizing electronic health records (EHR) or large-scale cohort designs like the Researching COVID to Enhance Recovery (RECOVER) initiative, increasingly employ "auxiliary variable dependent sampling" (also termed "tiered" testing). In these designs, the selection of participants for expensive or intensive assessments (Phase 2) is driven by inexpensive auxiliary variables collected in an initial phase (Phase 1), such as symptom triggers or prior test results.

The core statistical challenge identified is that ignoring this underlying sampling mechanism leads to biased estimation and erroneous scientific conclusions. Specifically, standard analyses fail to account for:

Selection Bias: The probability of receiving a test is often higher for individuals exhibiting specific symptoms (e.g., loss of smell), creating an oversampling of cases with adverse outcomes.
Confounding: Imbalances in baseline covariates between exposure groups (e.g., Long COVID status vs. non-Long COVID) that are not adjusted for.
Complex Missingness: In designs with repeated sampling opportunities (e.g., RECOVER-Adult), participants may drop out, become ineligible, or complete tests at varying times based on longitudinal eligibility and prior sampling history.
Temporal Misalignment: In some designs (e.g., RECOVER-Pediatric), the exposure is measured only after the initial sampling selection, complicating the definition of the propensity score relative to the full cohort.

Methodology
The authors propose a unified inverse probability weighting (IPW) framework to estimate adjusted outcome means and mean differences between exposure groups, specifically addressing two distinct sampling scenarios observed in the RECOVER studies.

1. Repeated Sampling Setting (RECOVER-Adult)
In this design, participants have multiple opportunities over a sequence of visits ( $k = 1, \dots, K$ ) to be sampled for tiered tests based on current auxiliary variables (triggers) and eligibility.

Estimands: The goal is to estimate the pooled mean outcome for each exposure level ( $\mu_a$ ) and the pooled mean difference ( $\psi$ ) across all visits.
Weighting Components: The estimator combines weights derived from three conditional probabilities:
- $\pi_U$ : The probability of attending a visit and meeting eligibility criteria, conditional on prior attendance and non-completion.
- $\pi_R$ : The probability of being sampled and completing the test at visit $k$ , conditional on attendance and prior non-completion.
- $\pi_A$ : The propensity score (probability of exposure status $A=a$ ) conditional on baseline covariates $W$ .
Implementation: The authors define a cumulative probability of remaining in the study and eligible for testing without prior completion ( $g_U$ ) and the probability of first completion at visit $k$ ( $g_R$ ). The inverse probability weighted estimator for the mean at visit $k$ is constructed using the product of these probabilities.
Propensity Score Adjustment: A critical methodological nuance is that the propensity score for exposure ( $\pi_A$ ) is defined for the entire cohort but can only be estimated among those currently on-study and eligible. To correct for this dependent missingness, the models used to estimate the propensity score must themselves be inverse-weighted by the probability of remaining in the study ( $g_U$ ).
Aggregation: Results from individual visits are pooled using an inverse-variance weighted mean.

2. Single Sampling Setting with Post-Sampling Exposure (RECOVER-Pediatric)
In this design, a subset of the cohort is selected for longitudinal follow-up based on baseline auxiliary variables before the exposure (Long COVID status) is measured.

Estimands: Adjusted mean outcome and mean difference between exposure groups.
Weighting Components: The estimator utilizes a sequence of probabilities:
- $\pi_R$ : Probability of selection (promotion) to the Tier 2 visit based on baseline auxiliary variables $Z$ and covariates $W$ .
- $\pi_U$ : Probability of attendance at the Tier 2 visit.
- $\pi_A$ : Propensity score for exposure status, defined with respect to the entire original cohort.
- $\pi_C$ : Probability of completing the outcome assessment conditional on attendance.
Implementation: Similar to the adult model, the propensity score for exposure is estimated among the attending cohort but must be inverse-weighted by the joint probability of promotion and attendance ( $g_{R,U} = \pi_R \pi_U$ ) to correctly reflect the exposure distribution in the full cohort.

Key Contributions

Unified Framework: The paper provides a rigorous analytic strategy that simultaneously addresses loss-to-follow-up, complex sampling designs (both single and repeated), differential test completion rates, and covariate imbalance.
Handling Post-Exposure Sampling: The authors explicitly address the scenario where exposure is measured after the sampling selection, a common feature in pediatric and some adult observational designs, by deriving specific weighting corrections for the propensity score.
Correction for Dependent Missingness: The methodology highlights the necessity of weighting the propensity score models themselves to account for the fact that the sample used to estimate the score is a biased subset of the target population.
Application to Long COVID: The methods are demonstrated using real-world data from the RECOVER Adult and Pediatric cohorts, specifically analyzing the association between Long COVID status and severe microsmia/anosmia (olfactory dysfunction).

Results
The paper presents an illustrative analysis of the University of Pennsylvania Smell Identification Test (UPSIT) within the RECOVER-Adult cohort:

Unadjusted Findings: Among participants who completed the UPSIT, 24.1% of Long COVID (LC) participants and 12.9% of non-LC participants had severe microsmia or anosmia.
Weighted Findings: After applying the proposed IPW to account for attendance, sampling bias (which favored those with self-reported smell loss), and completion rates, the estimated frequencies decreased to 14.6% for LC and 8.0% for non-LC.
Effect Estimate: The fully weighted model estimated a difference of 6.3% (95% CI: 1.3%, 11.3%) between groups.
Interpretation: The reduction in estimated frequencies confirms that the sampling schema (which oversampled individuals with triggers like "loss of smell") significantly biased the unadjusted estimates upward. The weighting successfully corrected for this oversampling.

Significance and Claims
The authors position this work as a necessary response to the increasing prevalence of auxiliary variable dependent sampling in observational research, particularly in EHR-based studies and large cohorts like RECOVER.

Necessity of Rigor: The paper claims that ignoring these sampling mechanisms leads to biased conclusions, and that existing methods for two-phase sampling must be extended to handle the specific complexities of repeated sampling and post-exposure measurement found in modern observational studies.
First Step: The authors modestly frame their work as a "first step" toward addressing more general questions, such as time-varying exposure effects or associations between multiple tiered tests. They acknowledge that further advancements, such as doubly robust extensions, are needed to improve efficiency and robustness.
Broader Impact: The paper aims to raise awareness among epidemiologists and statisticians regarding the types of estimation strategies required to extract reliable evidence from complex, dependent sampling schemes, thereby facilitating proper inference into disease mechanisms like those of Long COVID.

Technical Summary: Inverse Probability Weighting for Auxiliary Variable Dependent Sampling in Observational Studies of Long COVID

More like this