📊 statistics

Addressing errors in multiple variables using generalized raking and cumulative probability models

This paper develops and evaluates efficient generalized raking estimators for cumulative probability models to reduce bias and improve estimation efficiency when analyzing error-prone routinely collected data, such as electronic health records, by leveraging validation subsamples.

Original authors: Eric S. Kawaguchi, Chun Li, Frank E. Harrell Jr., Pamela A. Shaw, Thomas Lumley, Bryan E. Shepherd

Published 2026-06-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Eric S. Kawaguchi, Chun Li, Frank E. Harrell Jr., Pamela A. Shaw, Thomas Lumley, Bryan E. Shepherd

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand the health of a massive city by looking at a giant, slightly blurry map. This map is your Electronic Health Record (EHR) data. It covers everyone in the city (10,000+ people), but because it was filled out by busy doctors and automated systems, it's full of smudges, typos, and missing pieces. If you try to draw conclusions using only this blurry map, your results might be misleading.

However, you also have a small, high-definition photo album of just 700 people from that city. A team of experts carefully checked these specific records, corrected the smudges, and filled in the missing details. This is your validation subsample.

The problem is: The photo album is too small to represent the whole city, but the blurry map is too big to ignore. How do you combine them to get the best possible picture?

This paper introduces a clever mathematical "glue" called Generalized Raking to stick these two data sources together, specifically for a type of analysis called a Cumulative Probability Model (CPM).

Here is how the paper breaks it down, using simple analogies:

1. The Problem: The "Blurry Map" vs. The "Gold Standard"

In the real world, researchers often have to guess based on imperfect data. In this study, they were looking at how much weight pregnant women gained.

The Blur: The big database had errors in almost everything: the women's starting weight, their BMI, whether they smoked, and even how long the pregnancy lasted. In fact, for the women whose records were checked, the "weight gain" numbers were wrong 100% of the time in the big database!
The Gold Standard: A small group of nurses manually checked 700 charts. They found the true numbers.
The Dilemma: If you only use the 700 checked charts, you have accurate data but not enough people to be sure of the trends. If you use the 10,000 un-checked charts, you have plenty of data, but it's full of errors that could trick you into thinking, for example, that smoking causes weight gain when it actually causes weight loss.

2. The Solution: "Generalized Raking" (The Smart Balancer)

The authors used a technique called Generalized Raking. Think of this like a smart scale or a balancing act.

The Old Way (Inverse Probability Weighting): Imagine you have a scale. You put the 700 checked people on it. To make them represent the whole city, you put heavy weights on them. But these weights are just based on "how likely they were to be picked." It's a bit clumsy.
The New Way (Generalized Raking): This method is like a smart scale that learns. It looks at the 700 checked people and the 10,000 un-checked people. It asks: "Do the 700 people look like the 10,000 in terms of age, race, and other traits?"
- If the 700 people are too young compared to the whole city, the scale automatically adjusts the weights to "pull" the average age up.
- It does this by using the "blurry" data from the big map as a guide. Even though the big map has errors, it still holds some truth. The method uses the big map to fine-tune the weights of the small, perfect group so they perfectly mirror the whole city.

3. The Tool: "Cumulative Probability Models" (The Flexible Ruler)

Usually, when scientists analyze data, they try to find the "average" (the mean). But averages are fragile; one weird outlier (like a woman who gained 50 lbs in a week) can ruin the average.

The authors used Cumulative Probability Models (CPMs).

The Analogy: Instead of measuring the exact height of a person, imagine a flexible ruler that measures where a person stands relative to everyone else.
How it works: It doesn't care about the exact number; it cares about the rank. "Is this woman in the top 10% of weight gainers? The top 50%?"
Why it's great: This method is robust. It's like a rubber band; if you pull on it with a weird outlier, it stretches but doesn't snap. It handles messy, skewed data (like pregnancy weight gain, which isn't a perfect bell curve) much better than traditional math.

4. The Magic Trick: Combining Them

The paper's big innovation is teaching the Smart Scale (Raking) how to work with the Flexible Ruler (CPM).

The Challenge: Usually, you can't easily use the "Smart Scale" with the "Flexible Ruler" because the math gets too complicated when you have thousands of unique data points.
The Fix: The authors figured out a shortcut. Instead of trying to balance every single detail, they used the "influence" of the data points (a statistical concept that measures how much a single person changes the result) to guide the balancing.
The Result: They created a method that takes the errors out of the big dataset by using the small, perfect dataset to "calibrate" the weights.

5. What They Found (The Results)

When they applied this to the pregnancy weight study:

Correction: The "blurry" data suggested that women with private insurance gained more weight. The "Smart Scale" corrected this, showing there was actually no strong link.
Surprise: The "blurry" data suggested smoking was linked to more weight gain. The corrected method showed the opposite: smoking was linked to less weight gain.
Efficiency: The new method was much more precise than just using the small group of 700 people. It was like getting a high-definition photo of the whole city without actually taking 10,000 photos.

Summary

The paper says: "Don't throw away your big, messy database just because it has errors. Don't rely only on your small, perfect database because it's too small. Instead, use a 'Smart Balancing' technique (Generalized Raking) combined with a 'Flexible Ranking' tool (CPM) to get the best of both worlds."

This allows researchers to get accurate, unbiased answers about health trends even when their data is imperfect, saving time and money while avoiding misleading conclusions.

Technical Summary: Addressing Errors in Multiple Variables Using Generalized Raking and Cumulative Probability Models

Problem Statement
Routinely collected biomedical data, such as electronic health records (EHR), are increasingly used for research but are frequently prone to errors across multiple variables (both outcomes and covariates). Naive analysis of such data can lead to biased findings. While validating a subsample of records (two-phase or double-sampling) can reduce bias, researchers face a trade-off: relying solely on the small validated subsample reduces statistical efficiency, while using the full error-prone cohort introduces bias. Existing methods for measurement error, such as regression calibration or SIMEX, often struggle with errors in many variables. Conversely, multiple imputation can be biased if imputation models are misspecified. The authors address the need for a robust, efficient method to combine error-prone phase-1 data with validated phase-2 data, specifically for continuous or mixed-type outcomes that may be skewed or contain outliers.

Methodology
The paper proposes a novel combination of Generalized Raking (GR) and Cumulative Probability Models (CPMs) within a two-phase sampling framework.

Cumulative Probability Models (CPMs): The authors utilize CPMs, which are semiparametric, rank-based models suitable for continuous, ordinal, or mixed outcomes. CPMs assume a linear model on a latent variable derived from an unspecified monotonic transformation of the response. This approach is robust to outliers and skewness and allows for the estimation of the entire conditional distribution (including quantiles and exceedance probabilities) rather than just the mean. The model is fitted using ordinal regression software by treating unique continuous values as ordinal categories.
Generalized Raking (GR) for CPMs: To incorporate the full cohort's error-prone data while correcting for bias using the validated subsample, the authors adapt GR (survey calibration).
- Auxiliary Variables: The optimal auxiliary variables for calibrating sampling weights are the expected values of the influence functions for the parameters of interest, conditional on the phase-1 (error-prone) data.
- Implementation: Since the true influence functions depend on unknown parameters, the authors approximate them using "naive influence functions" derived from fitting a CPM to the error-prone phase-1 data.
- Calibration: The sampling weights for the phase-2 subsample are adjusted (raked) so that the weighted sum of these naive influence functions in the subsample matches the known total in the phase-1 cohort.
- Efficiency Consideration: Due to the high dimensionality of the intercept parameters ( $\alpha$ ) in CPMs with continuous outcomes, the authors calibrate weights using only the influence functions for the regression coefficients ( $\beta$ ).
Inference: The variance of the resulting GR estimator is estimated via Taylor-series linearization, accounting for both the full-cohort Fisher information and the phase-2 sampling variance.

Key Contributions

Novel Integration: This is the first application of generalized raking to CPMs fitted to continuous data. Previous applications of raking to ordinal models were limited to settings with a small number of categories.
Handling Continuous Responses: The authors demonstrate how to handle the non-trivial task of calibrating survey weights using estimated influence functions from CPMs with continuous responses, where the intercept function is non-parametric and high-dimensional.
Robustness and Efficiency: The method provides a way to obtain robust, rank-based estimates that are efficient by leveraging the large volume of error-prone EHR data without requiring strong parametric assumptions about the error structure or the outcome transformation.

Results

Motivating Example (Maternal Weight Gain): Applied to a study of 10,132 women (phase-1) with a validated subsample of 726 (phase-2), the GR-CPM method corrected biases present in naive analyses.
- Findings: Naive analyses suggested tobacco use and private insurance were associated with greater weight gain, whereas GR estimates suggested no association or the opposite direction (with wider confidence intervals).
- Efficiency: The GR estimators were generally more efficient than Inverse Probability Weighted (IPW) estimators alone. For the BMI coefficient, the standard error of the GR estimator was 0.67 times that of the IPW estimator. However, for variables not present in phase-1 data (e.g., pregnancy length), no efficiency gain was observed, as expected.
- Interpretability: The method successfully derived interpretable quantities, such as the probability of unhealthy weight gain by race and the non-linear relationship between pre-pregnancy BMI and weight gain.
Simulations:
- The GR-CPM estimator (M6) was unbiased and more efficient than the IPW-CPM estimator (M5) across 1,000 replicates.
- Efficiency gains were most pronounced when the correlation between error-prone and true measures was high.
- Coverage probabilities for 95% confidence intervals were close to nominal levels, indicating well-behaved variance estimators.
- The method performed robustly even when the transformation of the outcome was unknown (a realistic scenario), unlike linear regression models that require correct transformation specification to remain unbiased.

Significance and Claims
The authors claim that combining Generalized Raking with CPMs offers a "robust and efficient" solution for analyzing two-phase sampling data with errors in multiple variables.

Efficiency "for Free": The paper asserts that GR estimators improve efficiency over IPW estimators without making additional assumptions beyond those required for IPW. The authors note that GR estimators are asymptotically guaranteed not to be less efficient than their IPW counterparts.
Practical Utility: The method allows researchers to utilize the full scale of EHR data while correcting for measurement error, addressing a critical need in biomedical research where high-quality validation is costly but necessary.
Limitations and Future Directions: The authors modestly note that while they calibrated on $\beta$ influence functions, calibrating on $\alpha$ influence functions (intercepts) could theoretically improve efficiency for quantile-based estimands in lower-dimensional settings (e.g., ordinal outcomes). However, for continuous outcomes, the high dimensionality of $\alpha$ makes this computationally unstable. They suggest that future work could explore categorizing continuous outcomes to facilitate such calibration, though initial simulations showed only minor gains.