📊 statistics

Two-Phase Sampling Designs and Analysis Approaches for Ordinal Outcomes

This paper extends two-phase sampling designs to studies with ordinal outcomes by proposing three outcome-informed sampling strategies and developing corresponding analysis methods that significantly improve estimation efficiency over simple random sampling, as demonstrated through simulations and a real-world sepsis trial application.

Original authors: Yunbi Nam, Nathan I. Shapiro, Eric P. Schmidt, Wesley H. Self, Ran Tao, Jonathan S. Schildcrout

Published 2026-05-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yunbi Nam, Nathan I. Shapiro, Eric P. Schmidt, Wesley H. Self, Ran Tao, Jonathan S. Schildcrout

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery about how a specific, expensive clue (let's call it the "Golden Clue") affects a patient's health outcome. You have a huge list of suspects (a study cohort), and for everyone on that list, you already know some basic, cheap information like their age, gender, and general health history. However, finding the "Golden Clue" for every single person is too expensive and time-consuming. You only have the budget to test it for a small handful of people.

This paper is a guide on how to pick that small handful of people smartly, and how to analyze the results correctly so you don't get the wrong answer.

The Problem: The "All-or-Nothing" Trap

In the past, if researchers had to pick a small group to test for the expensive clue, they might have just picked names out of a hat (Simple Random Sampling). But this is inefficient. If you pick a random group, you might miss the most interesting cases.

For example, if you are studying a disease that gets worse in stages (mild, moderate, severe, critical), picking a random person might give you someone with "moderate" symptoms, which doesn't tell you much about the extremes. You want to find the people at the very top and very bottom of the severity scale to see how the "Golden Clue" behaves in extreme situations.

The Solution: Three Smart Ways to Pick Your Team

The authors propose three new strategies to pick your small group (Phase 2) from the big list (Phase 1) to make your detective work more efficient:

Outcome-Dependent Sampling (ODS): "The Extremes Strategy"
- The Metaphor: Imagine you are looking for the best and worst students in a school. Instead of picking students randomly, you specifically look at the top 10% and bottom 10% of the report cards.
- How it works: You look at the health outcome everyone already has (the report card). You deliberately pick more people who have very mild outcomes and very severe outcomes, and fewer people with average outcomes. This gives you a "wider" view of the data without testing everyone.
Covariate-Stratified ODS (CSODS): "The Grouped Extremes Strategy"
- The Metaphor: Sometimes, the "report card" is heavily influenced by something else, like the student's grade level. If you just pick the top and bottom students from the whole school, you might accidentally pick only 1st graders and 12th graders, missing the middle grades.
- How it works: You first divide your suspects into groups based on a known factor (like age or a specific health score). Then, within each group, you pick the extremes. This ensures you get a balanced view of the "Golden Clue" across all different types of people, not just the extremes of the whole group.
Residual-Dependent Sampling (RDS): "The Surprise Factor Strategy"
- The Metaphor: Imagine you have a crystal ball that predicts a student's grade based on their study habits. Most students fit the prediction well. But some students are "surprises"—they studied hard but got a bad grade, or didn't study and got an A. These "surprises" are the most interesting.
- How it works: You use the cheap data to build a prediction model. Then, you look for the people whose actual health outcome was very different from what the model predicted (the "residuals"). You pick those "surprise" cases because they hold the most new information about the "Golden Clue."

The Analysis: How to Solve the Puzzle Without Cheating

Once you pick your smart group, you can't just analyze them like a normal group. If you ignore the fact that you chose them specifically, your math will be biased (like a judge only listening to the loudest witnesses).

The authors developed three mathematical "tools" to fix this:

ACML (The "Correction" Tool): This method looks only at the people you picked but adds a mathematical "correction factor" to account for the fact that you picked them on purpose. It's like saying, "I know I picked more extreme cases, so I will adjust my math to pretend I saw the average ones too."
Multiple Imputation (The "Fill-in-the-Blanks" Tool): This method is smarter. It uses the data from the people you didn't pick (who still have their cheap info) to guess what their "Golden Clue" value might have been. It fills in the missing pieces of the puzzle many times to create a complete picture, then averages the results.
Sieve Maximum Likelihood (The "Flexible Net" Tool): This is a sophisticated method that doesn't assume the "Golden Clue" follows a specific shape. Instead, it uses a flexible "net" (mathematical splines) to catch the true shape of the data, making it very robust even if the data is messy.

What They Found

The authors ran thousands of computer simulations to test these ideas. They found:

Efficiency: Using these smart picking strategies (ODS, CSODS, RDS) was much better than picking randomly. You get more accurate answers with the same amount of money.
The Best Combo: When a known factor (like age or a health score) strongly predicts the outcome, the "Grouped Extremes" (CSODS) and "Surprise Factor" (RDS) strategies worked best.
The Best Math: The "Fill-in-the-blanks" (Multiple Imputation) and "Flexible Net" (Sieve) methods generally gave more precise answers than the simple "Correction" method, especially when you had data on the people you didn't pick.

Real-World Test: The CLOVERS Trial

To prove this works in real life, they applied these methods to data from a real medical trial called CLOVERS. This trial looked at patients with sepsis (a severe infection).

The Goal: They wanted to see if a specific protein in the blood (Interleukin-6, or IL-6) was linked to how sick the patient was 14 days later.
The Outcome: They treated the full group of 1,351 patients as the "Phase 1" list. They then simulated picking only 400 patients to test for IL-6 using their new strategies.
The Result: The smart strategies (especially CSODS and RDS) allowed them to reconstruct the relationship between the protein and the illness almost as well as if they had tested everyone. They got a clearer picture with less data.

Summary

This paper provides a new rulebook for researchers who have limited money to test expensive things. Instead of guessing who to test, you can use the data you already have to pick the most informative people. Then, you use special math to ensure your conclusions are accurate. This saves money and time while getting better scientific answers.

Technical Summary: Two-Phase Sampling Designs and Analysis Approaches for Ordinal Outcomes

Problem Statement
Modern clinical trials and cohort studies routinely collect low-cost data (e.g., baseline covariates, outcomes) on all participants and store biospecimens for future analysis. However, measuring expensive exposures (e.g., biomarkers, genomic data) for the entire cohort is often infeasible due to resource constraints. Two-phase sampling designs address this by using Phase 1 data to guide the targeted selection of a subset for Phase 2 measurements. While efficient sampling designs and valid likelihood-based inference methods have been well-established for binary, continuous, and time-to-event outcomes, they have not yet been extended to ordinal outcomes. Ordinal endpoints are increasingly common in clinical research (e.g., disease severity scales, clinical status progression), yet existing two-phase frameworks do not accommodate their ordered categorical nature.

Methodology
The authors propose a unified framework for two-phase studies with ordinal outcomes under the Proportional Odds (PO) model. The framework consists of two main components: sampling designs and analysis methods.

Phase 2 Sampling Designs:
The authors develop three outcome-informed sampling strategies designed to enrich the Phase 2 sample with informative subjects by leveraging the ordered structure of the outcome:
- Outcome-Dependent Sampling (ODS): Oversamples subjects with extreme outcome values (e.g., the lowest and highest categories).
- Covariate-Stratified ODS (CSODS): Defines sampling strata jointly on the outcome and a baseline covariate ( $Z_1$ ) to increase outcome variability within covariate levels, addressing cases where covariates strongly predict the outcome.
- Residual-Dependent Sampling (RDS): Targets variability in the residual component of the outcome after accounting for inexpensive covariates. To handle the multiple intercepts inherent in the PO model, the authors utilize the Probability-Scale Residual (PSR), which provides a single residual value per subject capturing the direction and magnitude of deviation between observed and fitted outcomes.
Analysis Approaches:
To ensure valid inference under these biased sampling schemes, three likelihood-based methods are developed:
- Ascertainment-Corrected Maximum Likelihood (ACML): A complete-case approach that restricts analysis to Phase 2 subjects but corrects for the sampling bias by incorporating sampling probabilities into the conditional likelihood. This is applicable to ODS and CSODS.
- Multiple Imputation (MI): A full-data approach that treats unmeasured exposures in non-sampled Phase 1 subjects as missing. It imputes these values using a parametric model combining the response model (PO model) and an exposure model, preserving information from the full Phase 1 cohort. This is applicable to ODS and CSODS.
- Sieve Maximum Likelihood Estimation (SMLE): A full-data semiparametric approach that estimates the exposure distribution nonparametrically using B-spline sieves. This method integrates over the unobserved exposure for non-sampled subjects, offering robustness against misspecification of the exposure model. SMLE is applicable to all designs, including RDS.

Key Contributions
The paper makes three distinct contributions to the literature:

Novel Sampling Designs: It derives the first efficient Phase 2 sampling designs specifically tailored for ordinal outcomes under the PO model (ODS, CSODS, and RDS), exploiting the ordered structure to concentrate resources on informative subjects.
Valid Inference Procedures: It develops three complementary likelihood-based inference methods (ACML, MI, SMLE) that yield valid and efficient estimates under these specific sampling designs.
Comprehensive Evaluation: It evaluates the finite-sample operating characteristics of all design-analysis combinations through extensive simulation studies and a plasmode study using real-world data.

Results

Simulation Studies: Extensive simulations (3,000 replicates) across scenarios with varying strengths of covariate-outcome associations demonstrated that all proposed design-analysis combinations yielded negligible bias and approximately 95% confidence interval coverage.
- Efficiency Gains: The proposed designs substantially improved efficiency over Simple Random Sampling (SRS) with standard Maximum Likelihood (ML).
- Impact of Covariates: When covariates were weakly associated with the outcome, ODS, CSODS, and RDS all provided similar efficiency gains. When covariates were strongly associated, CSODS and RDS outperformed ODS. Specifically, RDS achieved the largest efficiency gains (Relative Efficiency up to 2.56) when multiple strong predictors were present.
- Analysis Methods: Full-data approaches (MI and SMLE) generally outperformed the complete-case ACML approach, particularly for estimating covariate effects. MI consistently improved efficiency over ACML under ODS and CSODS.
Plasmode Study (CLOVERS Trial): The methods were applied to data from the CLOVERS trial to assess the association between Interleukin-6 (IL-6) levels and a four-level clinical status outcome. The results mirrored the simulations: outcome-informed designs (ODS, CSODS, RDS) combined with full-data estimators (MI, SMLE) yielded significantly lower uncertainty (higher D-efficiency, smaller integrated confidence interval areas) compared to SRS, while accurately recovering the underlying association trend.

Significance
The paper addresses a critical gap in epidemiological and clinical research by providing the first unified framework for two-phase studies with ordinal outcomes. It offers investigators a cost-effective strategy to study expensive exposures when outcomes are ordinal, a common scenario in critical care and disease severity research. The proposed framework allows researchers to tailor their study design and analysis to their specific data constraints (e.g., availability of baseline covariates) and scientific objectives, ensuring valid and efficient estimation of exposure-outcome associations without requiring expensive measurements for the entire cohort. The authors note that while the current MI approach is not compatible with RDS, the framework opens avenues for future work, such as extending conditional exposure models to RDS and investigating robustness under model misspecification.