📊 statistics

Sample Size Determination Under Selection Bias: Robust Tolerance Limits for Prevalent Cohort Data

This paper derives and validates modified distribution-free tolerance limit formulae that accommodate various biased sampling schemes, such as weight bias and censoring, to address the limitations of traditional methods when unbiased representative samples are unavailable, demonstrating their application with dementia data from the Canadian Study of Health and Aging.

Original authors: James H. McVittie, Martin Lysy, Masoud Asgharian

Published 2026-05-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: James H. McVittie, Martin Lysy, Masoud Asgharian

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a baker trying to figure out how many cookies to bake to ensure that, if you grab two specific cookies from the tray (say, the 3rd smallest and the 10th largest), the space between them contains at least 80% of all the cookie sizes you could have made.

In the world of statistics, this is called finding a tolerance limit. For decades, statisticians have had a famous, simple recipe (the Scheffé-Tukey formula) to answer this question. The recipe works perfectly if your tray of cookies is a fair, random sample of the whole batch. If you pick cookies blindly, the math holds up.

The Problem: The "Biased" Tray
However, in real life—especially in medical studies like tracking dementia patients—you often can't get a fair, random tray. You might only get cookies that are "longer" or "heavier" because of how you collected them.

The Analogy: Imagine you are studying how long people live with a disease. If you only start looking at people after they have already been sick for a while (a "prevalent cohort"), you are more likely to find the people who survive longer. The short-lived patients have already passed away and are invisible to you.
The Result: Your sample is biased. It's like a tray where only the biggest cookies were allowed to stay. If you use the old, simple recipe on this biased tray, you will calculate that you need very few cookies to get your 80% coverage. But in reality, you will be way off. You'll think you have enough data, but you won't.

The Solution: New Recipes for a Biased World
The authors of this paper, James McVittie, Martin Lysy, and Masoud Asgharian, realized that the old recipe fails when the data is biased. They created two new "recipes" (methods) to fix the math so it works even when your sample is skewed.

The "Inequality" Method (The Over-Preparer):
This method tries to be safe by using a series of logical "if-then" rules. It's like a baker who, worried about the biased tray, decides to bake way more cookies than necessary just to be absolutely sure.
- The Catch: It works, but it's wasteful. It tells you to collect a massive amount of data (a huge sample size) to be safe, often overestimating what you actually need.
The "FFT" Method (The Precision Chef):
This is the paper's star player. "FFT" stands for Fast Fourier Transform, which is a fancy math tool that acts like a high-speed blender for probability curves.
- How it works: Instead of guessing or using loose rules, this method takes the shape of your biased data and mathematically "un-blends" it to see what the original, fair distribution looked like. It then calculates the exact number of cookies (sample size) needed.
- The Result: It is incredibly accurate. It tells you the exact number of people you need to study to get your 80% coverage, no more and no less.

The Proof: The Simulation and the Real Test
The authors tested these new recipes in two ways:

The Simulation: They created fake computer data where they knew the "true" answer. When they used the old recipe on biased data, it failed miserably (predicting they needed too few people). The "Inequality" method was too cautious (predicting too many). The FFT method hit the bullseye every time.
The Real World Test: They applied this to real data from the Canadian Study of Health and Aging (CSHA), which tracks people with dementia. Because this study only catches people who have already been diagnosed (a biased sample), the old math would have given the wrong answer. Using their new FFT method, they calculated exactly how many more participants would be needed to be statistically confident.

The Bottom Line
If you are doing a study where your data is naturally "biased" (like looking at people who have already survived a disease for a while), do not use the old, simple math formula. It will lie to you.

Instead, use the new FFT method proposed in this paper. It's like upgrading from a rough guess to a laser-guided calculator. It ensures you don't waste money and time collecting too much data, nor do you risk failing your study by collecting too little. It's the most efficient way to handle messy, real-world data.

Technical Summary: Sample Size Determination Under Selection Bias

Problem Statement
The determination of sufficient sample sizes for statistical studies is critical for ensuring hypothesis test power and capturing the majority of a population distribution. The classical framework for this problem, established by Scheffé and Tukey (1944), provides a distribution-free formula relating sample size to population coverage using order statistics. However, this formula relies on the strict assumption of an unbiased, random sample from the target population. In biological and medical applications, logistical constraints often preclude such sampling, leading to biased designs such as prevalent cohort studies, length-biased sampling, and censored failure time data. When the classical Scheffé-Tukey formula is applied to these biased samples, it yields incorrect sample size estimates and coverage probabilities. Existing recent approaches for censored data often rely on restrictive assumptions, such as exponential failure time distributions. This paper addresses the need for robust sample size formulae that accommodate general biased sampling schemes without being restricted to specific parametric forms like the exponential distribution.

Methodology
The authors propose extensions to the Scheffé-Tukey formula to handle a large class of biased sampling schemes, including weight bias and censoring. Unlike the unbiased case, the modified formulae are not distribution-free; they require a priori knowledge (or estimation) of both the target population distribution ( $F$ ) and the biased sampling distribution ( $G$ ). The paper outlines two primary approaches to solve the coverage problem inequality $P[F(Y_{n-m+1}) - F(Y_r) \geq q] \geq 1 - \alpha$ , where $Y$ represents order statistics from the biased distribution:

Analytic Inequality Approach: This method approximates the two-sided coverage problem by treating the order statistics as approximately independent and utilizing a sequence of one-sided inequalities. It employs a piecewise linear approximation of the quantile mapping $\Phi^{-1} = F \circ G^{-1}$ . While this yields an analytic solution, the authors note that the sequence of inequalities provides loose bounds, often leading to an overestimation of the required sample size.
Computational FFT Approach: To avoid the overestimation inherent in the inequality method, the authors propose a computational approach based on the Fast Fourier Transform (FFT). Assuming independence between order statistics for moderate sample sizes, the probability density function (PDF) of the difference between order statistics is derived via the convolution of their individual PDFs. This convolution is efficiently calculated using the FFT. The method approximates the integral of the resulting PDF using a Riemann sum on a grid, allowing for the numerical estimation of the coverage probability and the subsequent root-finding of the sample size $n$ .

The paper details how the necessary quantile mappings ( $\Phi$ and $\Phi^{-1}$ ) can be obtained in various settings: analytically for specific distributions, via Monte Carlo simulation when distributions are intractable, or through nonparametric maximum likelihood estimators (NPMLE) using pilot study data.

Key Results
The proposed methods were validated through an extensive simulation study and an application to real-world data:

Simulation Study: Using Gamma-distributed target populations with varying hazard rates and length-biased sampling with right-censoring, the study compared the classical Scheffé-Tukey formula, the Inequality approach, and the FFT approach.
- The Scheffé-Tukey formula consistently underestimated the required sample size, resulting in empirical coverage probabilities significantly lower than the nominal 95% level.
- The Inequality approach tended to overestimate the sample size, yielding coverage probabilities close to 100%, which becomes less efficient as censoring rates increase.
- The FFT approach successfully identified the correct sample size, achieving empirical coverage probabilities close to the nominal 0.95 level across all scenarios. The computational cost was found to be negligible (typically under 0.1 seconds).
Application to CSHA Data: The methods were applied to data from the Canadian Study of Health and Aging (CSHA), a prevalent cohort study of dementia patients with length-biased and right-censored failure times. Using the CSHA data as a pilot study to estimate the distributions, the authors calculated required sample sizes for various order statistic configurations.
- The results mirrored the simulation findings: Scheffé-Tukey produced the smallest sample sizes, the Inequality approach produced the largest, and the FFT approach yielded intermediate values.
- The authors note that while the calculated sample sizes were smaller than the existing CSHA cohort size, the comparison highlights the substantial divergence in estimates between methods when bias is present.

Significance and Claims
The paper claims that the classical Scheffé-Tukey formula is invalid for biased sampling regimes, leading to statistically unsound study designs. The primary contribution is the derivation and validation of modified sample size formulae that account for selection bias and censoring. The authors emphasize that while these methods require knowledge of the underlying distributions (or the ability to estimate them via pilot studies), they offer a necessary correction to standard practice.

The authors modestly conclude that the FFT-based approach is the recommended method for computing tolerance limit sample sizes under biased sampling. They argue that the negligible computational trade-off of the FFT method is justified by the significant increase in accuracy compared to both the classical formula and the conservative inequality approach. The work provides a practical framework for researchers dealing with prevalent cohort data, ensuring that sample size calculations reflect the complexities of biased sampling designs rather than relying on idealized assumptions of unbiasedness.

Technical Summary: Sample Size Determination Under Selection Bias

More like this