📊 statistics

A Diagnostic to Find and Help Combat Stochastic Positivity Issues -- with a Focus on Continuous Treatments

This paper proposes a novel diagnostic tool to detect stochastic positivity violations in continuous treatment settings and guide researchers in adjusting their estimands, a method validated through simulations and applied to a pharmacoepidemiological study on HIV treatment in children.

Original authors: Katharina Ring, Michael Schomaker

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Katharina Ring, Michael Schomaker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: "The Missing Ingredients"

Imagine you are a chef trying to figure out exactly how much salt makes a soup taste perfect. You have a recipe book (your data) with 100 different soups. Some soups have a little salt, some have a lot, and some have none.

To know the perfect amount, you need to have tried every amount of salt on every type of soup base. But in real life, your recipe book is messy.

You have 50 soups with very little salt.
You have 50 soups with a medium amount.
But you have zero soups with a huge amount of salt.

If you try to guess what happens if you add a huge amount of salt to a soup you've never seen before, you are just guessing. In statistics, this is called a Stochastic Positivity Violation. It means there are "gaps" in your data where you simply don't have enough information to make a reliable prediction.

Most researchers know this is a problem, but they often don't have a good way to spot exactly where the gaps are, especially when dealing with continuous things (like exact milligrams of medicine) rather than simple "yes/no" choices.

The Solution: A "Data Map" (The Diagnostic)

The authors of this paper created a new tool, which they call a Diagnostic. Think of this as a "Data Map" or a "Flashlight" that researchers can use before they start cooking (running complex math models).

This tool answers two big questions:

"Is my recipe book good enough for this specific question?"
"If not, how should I change my question so I can answer it?"

How the Tool Works: The "Neighborhood" Analogy

The tool works by looking at every single person (or data point) in the study and asking: "How many neighbors do you have?"

The Concept: To predict what happens to a person with a specific trait (e.g., a specific weight and a specific drug dose), you need to look at other people who are very similar to them.
The "Kernel": The tool uses a mathematical "kernel" (think of it as a glowing circle or a neighborhood radius). If you are standing in the middle of a crowd, the tool counts how many people are standing inside your glowing circle.
The Score:
- High Score: You have lots of neighbors nearby. You can make a safe, reliable guess about what happens to you.
- Low Score: You are standing alone in the middle of a desert. There are no neighbors nearby. Any guess about you is just a wild guess (extrapolation) and is likely to be wrong.

The Two Versions of the Tool

The paper offers two ways to use this map:

The Simple Map (Data-Centric): This just looks at the data. It asks, "For this specific treatment plan, how many people have neighbors?" It gives a quick "traffic light" signal: Green (safe to proceed), Yellow (risky), or Red (don't do it).
The Detailed Map (Estimation-Focused): This looks deeper. It checks two specific things:
- The Outcome Model: Can we predict the result (like viral failure) based on the data?
- The Weight Model: Can we calculate the "weight" or importance of each person? If the tool finds that some people would need to be given a "super-weight" because they are so unique, it warns that the math might blow up and give a wrong answer.

What the Tool Found (The Simulation)

The authors tested this tool using fake data (simulations) to see how it behaved. They tried different "Treatment Plans" (like "give everyone 5mg more" or "give everyone a dose based on their weight").

The Result: The tool successfully predicted which plans would fail.
The Lesson: Just because a treatment plan sounds logical (like "increase everyone's dose by 5mg") doesn't mean you have the data to prove it works. If the plan pushes people into a "desert" where no data exists, the tool turns the light red. It showed that some complex plans (Modified Treatment Policies) are actually very risky if the data is sparse.

Real-World Test: The HIV Drug Study

The authors applied their tool to real medical data from a study on children with HIV (the CHAPAS-3 trial). They wanted to find the perfect drug concentration to stop the virus.

The Problem: When they ran the tool, it found that for certain types of children (specifically "slow metabolizers" who process drugs slowly), there was zero data at low drug concentrations. It was physically impossible for these children to have low drug levels based on how their bodies work.
The Fix: The tool told the researchers, "You cannot ask 'What if we gave slow metabolizers a low dose?' because that scenario never happened and isn't realistic."
The Solution: The researchers used the tool to change their question. Instead of forcing everyone to the same low dose, they changed the plan to: "Give slow metabolizers their natural dose, but give others a low dose." The tool confirmed that this new plan had plenty of "neighbors" (data support) and was safe to study.

The Takeaway

This paper gives researchers a "sanity check" before they do their math.

Before: Researchers would often pick a complex question, run the numbers, and only then realize the answer was garbage because of missing data.
Now: They can use this "Data Map" first. If the map shows a desert, they can change their question to one that fits the terrain.

It's like checking if you have enough ingredients in your pantry before you try to bake a cake. If you're missing flour, the tool tells you, "Don't bake that cake; bake cookies instead," saving you time and preventing a messy failure.

Technical Summary: A Diagnostic to Find and Help Combat Stochastic Positivity Issues

Problem Statement
The positivity assumption is fundamental to identifying causal effects, yet its stochastic variant—where certain treatment values are rarely or never observed for specific subgroups due to chance or low probability—is frequently overlooked, particularly in studies involving continuous treatments or Modified Treatment Policies (MTPs). While theoretical (deterministic) violations are often identifiable through logic, stochastic violations in continuous settings are difficult to diagnose because the number of potential treatment values exceeds the number of data points.

Current approaches to address this issue are limited. Existing tools, such as parametric bootstraps, are often estimator-specific and fail to distinguish between estimator failure and data sparsity. Generalized propensity score (GPS) approaches require density estimation, which is challenging in high dimensions and does not easily reveal positivity issues. Furthermore, while changing the estimand (e.g., to an MTP) is a common recommendation to mitigate bias, researchers lack a systematic way to determine which alternative estimand will successfully balance proximity to the original research question with the feasibility of estimation given the available data.

Methodology
The authors propose a novel, kernel-based sparsity diagnostic designed to be estimand-specific and estimator-independent. The core premise is that stochastic positivity violations manifest as a lack of data support in the regions of the covariate-treatment space required for a specific intervention.

Core Concept (Effective Data Points - EDP): The diagnostic quantifies the "effective data points" available to estimate the outcome for each observation under a specific intervention scheme. It utilizes a proximity measure (kernel function) to weight observed data points ( $o^{obs}$ ) based on their distance from the counterfactual intervention points ( $o^{int}$ ).
- Kernel Choice: The authors recommend a Gaussian kernel, where the steepness (bandwidth) is determined by how far a researcher is willing to extrapolate. Hyperparameters are set based on domain knowledge or rules of thumb (e.g., half a standard deviation for the treatment variable).
- Calculation: For each observation $i$ , the diagnostic sums the kernel weights of all observed data points $j$ relative to the intervened value $o^{int}_i$ . This yields an EDP score for every observation.
Two Variations:
- Data-Centric Diagnostic: A simplified version that assesses data support for the outcome model ( $\bar{Q}_n$ ) across the joint distribution of treatment and covariates. It is visualized via boxplots to identify observations with low support (low EDP).
- Estimation-Focused Diagnostic: A more complex version that separately evaluates support for the outcome model ( $\bar{Q}_n$ ) and the treatment model ( $g_n$ ). It also estimates the potential magnitude of weights ( $1/g_n$ ) for intervened points, helping to identify where inverse probability weighting might amplify bias due to extreme weights.
Application to MTPs: The diagnostic is applied to various intervention schemes, including static interventions, naive shift-response curves, stochastic shifts, threshold response curves, and conditional shift response curves.

Key Results
The paper validates the diagnostic through simulation studies and a practical application using data from the CHAPAS-3 trial (children with HIV).

Simulation Study:
- The study simulated three data scenarios: overlapping, adjacent, and divided (no overlap) distributions of treatment across confounder levels.
- Static Interventions: The diagnostic correctly identified that static dose-response curves suffer from severe bias in "divided" data scenarios where extrapolation is required across empty regions.
- MTPs: The diagnostic revealed that not all MTPs solve positivity issues. For instance, a "naive shift" (adding a constant to all doses) often shifts data into unsupported regions, leading to bias. Conversely, "conditional shift" or "threshold" policies, which restrict interventions to observed ranges, showed high support and lower expected bias.
- Predictive Power: The diagnostic successfully predicted where bias would occur in the estimated curves without access to the true data-generating process, highlighting regions where extrapolation was unstable.
Practical Application (CHAPAS-3):
- Applied to a study on efavirenz concentration and viral failure, the diagnostic identified specific subgroups (non-adherent patients and slow metabolizers) with extremely low EDP scores at the lower bound of the therapeutic window.
- It demonstrated how researchers can iteratively refine the estimand: by excluding non-adherent patients or restricting the target population to specific metabolization groups, the EDP scores improved, indicating more feasible estimation.
- In a longitudinal extension, the diagnostic showed that introducing a "spike" in treatment concentration at a specific time point resulted in near-zero support for subsequent time points, correctly flagging the estimand as unestimable without massive bias.

Significance and Claims
The authors claim that this diagnostic provides a crucial, pre-modeling tool for applied researchers. Its primary significance lies in:

Feasibility Assessment: It allows researchers to determine before model fitting whether a specific estimand is estimable given their data, saving time and computational resources.
Estimand Selection: It guides the selection of Modified Treatment Policies by visualizing the trade-off between the closeness of the intervention to the research question and the sparsity of data support.
Bias Diagnosis: It helps distinguish between bias caused by the estimator and bias caused by the data structure (sparsity), addressing a gap in current literature where tools are often tied to specific estimators.
Generality: Unlike density-based methods, this approach works directly with the data distribution and applies to static, dynamic, and MTP estimands in both binary and continuous treatment settings.

The paper concludes that while the diagnostic relies on user-defined hyperparameters (kernel bandwidth) which require domain knowledge, it offers a transparent and intuitive framework for navigating the complexities of stochastic positivity in continuous causal inference.