📊 statistics

Selection and Collider Restriction Bias Due to Predictor Availability in Prognostic Models

This methodological note investigates and discusses how predictor availability in prognostic models can introduce selection and collider restriction bias.

Original authors: Marc Delord

Published 2026-02-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marc Delord

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to build a "crystal ball" to predict which patients are most likely to get sick in the future. You want to create a score that looks at a patient's age, blood pressure, and kidney function to tell you: "This person has a 20% chance of needing a kidney transplant in five years."

This paper argues that many of these "crystal balls" are actually cracked, not because the math is wrong, but because of who gets to look into the ball in the first place.

Here is the story of the problem, explained with a few simple analogies.

1. The "Missing Ingredient" Problem

To build your crystal ball, you need specific ingredients (data points). Let's say you need to know a patient's Kidney Function and their Urine Protein Levels.

In the real world, doctors don't test everyone for everything. They usually only test the urine protein if the kidney function looks suspicious or if the patient seems very sick.

The Flaw: If you build your prediction model using only the patients who already had their urine tested, you are only looking at the "sick" people. You are ignoring the "healthy" people who never got tested because they seemed fine.
The Result: Your model thinks the disease is much more common or severe than it actually is, because it's only looking at the tip of the iceberg.

2. The "Gatekeeper" Analogy

Imagine a nightclub (the hospital) where you want to predict who will get into a fight (the bad health outcome).

The Predictor: You want to use "How loud they are shouting" as a clue.
The Gatekeeper: The bouncer only lets people into the VIP room (where you can measure the shouting) if they look like they are already getting agitated.

If you only study the people in the VIP room to see who fights, you will conclude that everyone in the club is about to fight. Why? Because the bouncer (the medical system) filtered out all the calm people. You created a biased sample by only looking at the people the gatekeeper let in.

In medicine, the "gatekeeper" is the rule that says, "We only test for this specific marker if the patient looks sick."

3. The "Double-Edged Sword" (Collider Bias)

The paper introduces a tricky concept called Collider Restriction Bias. This is the most important part.

Imagine a traffic intersection where two roads meet.

Road A: The patient's actual kidney health (getting worse).
Road B: The doctor's gut feeling that the patient is at risk (due to other symptoms like diabetes).

Both Road A and Road B lead to the same stop sign: "Order the Urine Test."

The Urine Test is the "Collider." It is the point where two different causes meet.

If you only study people who have the urine test results, you are effectively standing at that intersection and only counting the cars that stopped there.
The Magic Trick: By forcing yourself to only look at people who stopped at the intersection, you accidentally create a fake connection between Road A and Road B. You might start thinking that "Loud Shouting" (Road A) causes "Fighting" (Road B), when in reality, they are just both reasons to stop at the bouncer's booth.

In the paper's example (the Kidney Failure Risk Equation), this happens because:

Kidneys get worse $\rightarrow$ Doctor orders a test.
Patient looks very sick $\rightarrow$ Doctor orders a test.

If you build a model using only the people who got the test, you distort the relationship between the symptoms and the outcome. You might think a specific symptom is a stronger warning sign than it really is, or you might miss the fact that the model doesn't work for healthy people who never got tested.

4. The Real-World Example: The Kidney Equation

The authors use a real tool called the KFRE (Kidney Failure Risk Equation). It's supposed to predict kidney failure.

The Problem: In the UK and US, doctors often forget to test the urine protein for patients with mild kidney issues. They only test the "sick" ones.
The Consequence: The model was built on data from the "sick" group. When doctors try to use it on a regular patient in a clinic, the model might say, "This person has a 50% risk!" when they actually have a 5% risk. The model is over-predicting danger because it was trained on a group of people who were already selected for being sick.

The Big Takeaway

The paper is a warning to scientists and doctors:

"Just because you have the data, doesn't mean you have the whole picture."

When building medical prediction tools, we often assume that the data we have (like blood tests) is available for everyone. But in reality, we only have data for the people the system decided to test. If we don't account for why those tests were ordered, our "crystal balls" will be broken, leading to wrong predictions and bad medical decisions.

The Solution?
We need to be smarter about how we design these models. We should either:

Make sure we test everyone (not just the sick ones) to get a fair sample.
Build simpler models that use data we know is available for everyone (like age and basic blood pressure), rather than complex tests that only the "sick" get.

In short: Don't let the way we choose our patients trick us into thinking we understand the whole story.

1. Problem Statement

The paper addresses a critical, often overlooked assumption in the development and validation of clinical prognostic models: the routine availability of required predictors at the point of care.

The Context: Prognostic models (e.g., Framingham risk score, QRISK3) estimate the probability of health outcomes based on clinical characteristics. Methodological guidelines for developing these models assume that all necessary predictors are routinely measured and available for the target population.
The Flaw: In practice, especially when using retrospective data, inclusion in model development is restricted to patients who have recorded predictor values.
The Core Issue: If the measurement of a predictor is not independent of the outcome risk (i.e., if sicker patients are more likely to be tested), restricting the analysis to patients with recorded data introduces systematic bias.
- This creates Selection Bias when disease severity drives testing.
- This creates Collider Restriction Bias when predictor availability is influenced by multiple factors (e.g., disease severity and a proxy for severity), distorting the causal relationships between predictors and the outcome.

2. Methodology

The paper employs a causal inference framework using Directed Acyclic Graphs (DAGs) to illustrate the mechanisms of bias. It does not present new empirical data but rather re-analyzes existing methodological concepts through the lens of predictor availability.

Causal Modeling: The author constructs three specific scenarios (illustrated in Figure 1 of the paper) to map the relationships between:
- $U$ : Unobserved underlying disease severity.
- $Y$ : The clinical outcome.
- $P_1, P_2$ : Predictors.
- $I\{P \text{ is measured}\}$ : The indicator of whether a predictor was recorded.
Scenario Analysis:
- Panel A (Simple Selection Bias): Disease severity ( $U$ ) directly influences the likelihood of measuring a predictor ( $P$ ). Restricting the sample to those with $P$ selected on severity.
- Panel B (Proxy Selection): A measured predictor ( $P_1$ ) acts as a proxy for severity, triggering the measurement of a second predictor ( $P_2$ ).
- Panel C (Collider Restriction Bias): Both underlying severity ( $U$ ) and a proxy ( $P_1$ ) influence the measurement of $P_2$ . Here, the availability of $P_2$ becomes a collider. Conditioning on this collider (by only analyzing patients with $P_2$ ) induces a spurious association between $P_1$ and $U$ , distorting the estimated coefficients for all predictors.
Case Study Application: The theoretical framework is applied to the Kidney Failure Risk Equation (KFRE), a widely used model for predicting kidney failure in Chronic Kidney Disease (CKD) patients.

3. Key Contributions

Identification of a Hidden Assumption: The paper explicitly highlights that the assumption of "predictor availability" is a central yet implicit pillar of prognostic model methodology that, when violated, invalidates model performance.
Distinction Between Bias Types: It clarifies the difference between classical selection bias (where the sample represents a high-risk subset) and collider restriction bias (where conditioning on availability creates artificial correlations and distorts effect estimates across the board).
Mechanistic Explanation of KFRE Limitations: The paper explains why the KFRE, despite strong statistical performance in validation studies, has limited uptake in routine clinical practice. The model requires urine albumin-to-creatinine ratio (uACR), which is not systematically tested in primary care unless specific triggers (like declining eGFR or high perceived risk) are present.
Causal Diagrams for Prediction: It provides a visual and conceptual toolkit (DAGs) for researchers to diagnose whether their predictor data suffers from selection or collider bias before model development.

4. Results and Evidence

The KFRE Example:
- Data Gaps: In the UK and US, uACR testing is inconsistent. In the UK, fewer than 25% of CKD 3–5 patients undergo uACR testing annually, rising to ~37% only if formally registered with CKD. In the US, only ~17% of at-risk adults have ACR recorded.
- The Bias Mechanism: Testing for uACR is triggered by declining eGFR (a predictor) and by the clinician's perception of overall risk (influenced by symptoms/comorbidities).
- Consequence: Because uACR availability depends on both the predictor (eGFR) and the outcome risk, conditioning on uACR availability creates a collider. This biases the relationship between eGFR and kidney failure risk, potentially leading to inaccurate risk stratification when the model is applied to the general CKD population where testing is less frequent.
Generalizability: The paper argues that this issue is not unique to KFRE but is a systemic problem in prognostic modeling where predictors are not universally measured.

5. Significance and Implications

Methodological Rigor: The paper urges the statistical and clinical community to move beyond standard validation metrics (like C-statistics) and explicitly assess predictor availability patterns during the design phase.
Model Design Recommendations:
- Simplified Models: In settings where complex predictor data is sporadically available, the paper supports the development of simplified prognostic models that rely only on routinely available data.
- Prospective Cohorts: Future model development should ideally occur in prospective cohorts where predictor measurement is standardized and independent of disease severity, rather than relying on retrospective "real-world" data with missingness patterns.
Clinical Impact: Ignoring these biases leads to models that perform well in research settings (where data is complete) but fail in routine clinical practice (where data is missing), undermining trust in clinical prediction tools and potentially leading to incorrect treatment or referral decisions.

Conclusion:
Marc Delord's paper serves as a critical warning to the field of prognostic research. It demonstrates that predictor availability is not a neutral data constraint but a potential source of severe causal bias. By failing to account for why data is missing (specifically when missingness is driven by the very factors the model aims to predict), researchers risk developing models that are mathematically sound but clinically misleading.