📊 statistics

Examining the Efficacy of Coarsened Exact Matching as an Alternative to Propensity Score Matching

This paper argues that Coarsened Exact Matching (CEM) is not a superior alternative to Propensity Score Matching (PSM) because it suffers from residual confounding due to its inexact nature, fails to outperform PSM in reducing imbalance when accounting for random variation, and becomes unstable and inefficient in high-dimensional settings, whereas PSM remains more robust to model misspecification.

Original authors: Fei Wan

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fei Wan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out if a new medicine works. The gold standard is a Randomized Clinical Trial (RCT), where you flip a coin to decide who gets the medicine and who gets a placebo. Because it's random, the two groups are perfectly balanced: the people who got the medicine are, on average, just as healthy, old, and wealthy as those who didn't.

But in the real world, we often can't flip coins (it's too expensive or unethical). So, we look at observational data (like hospital records). Here, the "medicine group" might be sicker or older than the "control group" to begin with. This is called confounding. To fix this, statisticians use "matching" techniques to pair up sick people from the medicine group with similar sick people from the control group, trying to recreate that perfect coin-flip balance.

For a long time, the standard tool for this was Propensity Score Matching (PSM). Recently, a new tool called Coarsened Exact Matching (CEM) became very popular. Many researchers claimed CEM was the "superior" version of PSM.

This paper, written by Fei Wan, argues that CEM is actually not the superhero everyone thinks it is. In fact, for many situations, PSM is the better, more reliable tool.

Here is the breakdown of why, using simple analogies:

1. The "Fuzzy Photo" vs. The "Blurry Snapshot"

The Claim: CEM is often called "Exact Matching," but it isn't exact. It's actually "Fuzzy Matching."

The Analogy: Imagine you are trying to match people based on their height.
- PSM is like taking a high-resolution photo of everyone's height and finding the person who is closest in height to your target. Even if they aren't exactly the same height, the math ensures that if you look at the whole group, the average height is perfectly balanced. Any small differences are just random noise (like a slight blur in a photo) that cancels itself out when you have enough people.
- CEM is like putting everyone into buckets: "Short," "Medium," and "Tall." You then match people who are in the same bucket.
- The Problem: Inside the "Medium" bucket, you might have someone who is 5'6" and someone who is 5'10". They are in the same bucket, but they are very different. This creates a systematic error (residual confounding) that doesn't go away, no matter how many people you add to the study. It's like trying to compare apples and oranges just because they are both in the "Fruit" basket.

2. The "Tightrope" of Complexity

The Claim: CEM falls apart when you have too many variables (the "Curse of Dimensionality").

The Analogy: Imagine you are trying to find a perfect twin for a person based on 10 different traits (height, weight, eye color, shoe size, favorite pizza, etc.).
- CEM tries to put everyone into boxes based on all 10 traits at once. As you add more traits, the boxes get so specific that most of them end up empty. You lose so much data (people who don't fit in any box) that your results become unstable and unreliable. It's like trying to find a needle in a haystack, but the haystack keeps shrinking until there's nothing left.
- PSM doesn't try to match on all 10 traits directly. Instead, it calculates a single "score" (a summary of all 10 traits) and matches people based on that score. This avoids the empty boxes problem and keeps the data stable, even when you have many variables.

3. The "Guessing Game" (Model Dependence)

The Claim: CEM forces you to make perfect guesses about the data later on, while PSM is more forgiving.

The Analogy:
- CEM is like a puzzle where you force the pieces together by sanding them down (coarsening) so they fit. Because you sanded them, the pieces don't fit perfectly anymore. To get the right picture, you now have to use a very complex, perfect formula to "fix" the gaps. If your formula is even slightly wrong, the whole picture is ruined.
- PSM is like a puzzle where the pieces fit naturally. Even if you use a simple, slightly imperfect formula to look at the result, the picture still looks mostly correct because the pieces were balanced to begin with. PSM is "robust," meaning it doesn't break as easily when you make a small mistake in your math.

4. The "Ruler" Problem

The Claim: Previous studies said PSM was bad at balancing groups because they used the wrong measuring stick.

The Analogy: Some researchers claimed PSM was failing because they used a ruler that only measured the distance between people, ignoring the direction.
- The author argues that in PSM, if one person is slightly taller and another is slightly shorter, these random differences cancel each other out.
- The old measuring tools (like Mahalanobis distance) treated these random ups and downs as "errors" that were getting worse.
- The author proposes a new way to measure (Standardized Mean Differences) that accounts for these random ups and downs. When you use this new ruler, PSM looks much better at balancing groups than CEM.

The Bottom Line

The paper concludes that CEM is not a magic bullet.

If you have a small number of variables, CEM might work okay, but it still has hidden flaws.
If you have many variables (which is common in real-world data), CEM causes you to lose too much data and introduces systematic errors that are hard to fix.
PSM remains the more robust, reliable choice because it handles randomness better, requires less perfect math to get a good result, and doesn't throw away as much data.

The author warns that many doctors and researchers have been using CEM thinking it's the "exact" solution, but it's actually an "inexact" shortcut that can lead to biased results if not handled with extreme care.

Technical Summary: Examining the Efficacy of Coarsened Exact Matching as an Alternative to Propensity Score Matching

1. Problem Statement

Coarsened Exact Matching (CEM) is frequently promoted in comparative effectiveness research as a superior alternative to Propensity Score Matching (PSM) for addressing covariate imbalance, model dependence, and statistical bias. Proponents argue that CEM improves upon PSM by reducing the "PSM paradox," where PSM is claimed to increase imbalance and model dependence as it approaches exact matching through the pruning of poorly matched pairs.

However, this paper argues that the recommendation to replace PSM with CEM is uncertain and potentially flawed due to three primary issues:

Mischaracterization of Exactness: CEM is often incorrectly labeled as "exact matching." In reality, it relies on coarsened variables (binned continuous or grouped categorical data), not original variables. This inherent inexactness introduces systematic residual confounding that does not vanish with increased sample size.
Misinterpretation of Imbalance Metrics: Prior studies claiming CEM outperforms PSM in reducing imbalance often utilize metrics like the Mahalanobis distance. These metrics fail to distinguish between systematic imbalance and random variation (chance imbalance). In PSM, covariate differences between matched pairs with identical propensity scores arise by chance and tend to cancel out as sample size increases; however, CEM's imbalance is structural and persistent.
Model Dependence and Dimensionality: Because CEM leaves residual confounding, unbiased estimation requires accurate post-matching regression modeling of the outcome-confounder relationship. This increases vulnerability to model misspecification. Furthermore, CEM suffers from the "curse of dimensionality" when matching on many covariates, leading to substantial data loss unless aggressive coarsening is used, which in turn exacerbates residual confounding.

2. Methodology

Theoretical Framework and Assumptions

The study operates under standard causal inference assumptions: Stable Unit Treatment Value Assumption (SUTVA), Conditional Ignorability (Unconfoundedness), Positivity, and a linear (or interaction) outcome model.

PSM: Utilizes the propensity score $e(X) = P(W=1|X)$ , which possesses the balancing score property ( $W \perp\!\!\perp X | e(X)$ ) and strong ignorability. The paper posits that in an exactly matched PSM design, chance imbalances in covariates do not introduce bias, and unbiased estimation can be achieved via simple difference-in-means without outcome modeling.
CEM: Involves a "coarsening" step where continuous variables are binned and categorical variables are grouped to create a lower-dimensional vector $C$ . Matching occurs on exact values of $C$ . Crucially, $C$ is not a sufficient statistic for $X$ regarding treatment assignment, meaning $W \not\perp\!\!\perp X | C$ . This results in systematic within-stratum imbalance ( $\delta_c \neq 0$ ) that persists regardless of sample size.

Proposed Metrics

The paper critiques existing imbalance metrics (e.g., Mahalanobis distance, $L_1$ metrics) for ignoring the direction of chance imbalances, which prevents them from averaging to zero in simulation studies.

New Metric: The authors propose a multivariate imbalance metric based on the Standardized Mean Difference (SMD). This metric sums the absolute values of averaged SMDs across simulated samples, specifically designed to capture systematic imbalance (which deviates from zero) versus random chance imbalance (which averages to zero).

Simulation Design

The study employs a comprehensive simulation study comparing CEM (using "Auto" coarsening via Sturges' rule and "G3" coarsening into three groups) against PSM (nearest-neighbor matching with a 0.2 caliper).

Scenarios:
- Homogeneous vs. Heterogeneous Treatment Effects: Evaluating performance under both constant and varying treatment effects.
- Model Specification: Testing performance under correctly specified and misspecified outcome models (including non-linearities and interactions).
- Dimensionality: Varying the number of confounders ( $p=2, 5, 7$ ) to assess the curse of dimensionality.
- Treatment Prevalence: Examining low (~~10%) vs. high (~~30%) treatment rates to assess bias in estimating the Average Treatment Effect on the Treated (PATT).
Analysis: The study evaluates bias, variance, and Mean Squared Error (MSE) using both unadjusted analyses (difference in means) and adjusted analyses (regression with linear terms).

3. Key Contributions and Results

1. Systematic vs. Random Imbalance

Finding: In PSM, covariate imbalance between matched pairs is random and converges to zero as the number of pairs increases. In CEM, imbalance is systematic and driven by the coarsening process; it does not diminish with larger sample sizes.
Result: Unadjusted analyses (simple mean differences) are substantially less biased under PSM than under CEM. CEM requires post-matching regression adjustment to mitigate residual confounding, whereas PSM does not strictly require it for unbiased estimation under the balancing score property.

2. Model Dependence and Misspecification

Finding: CEM's reliance on coarsened variables means that unbiased estimation is contingent on correctly modeling the relationship between the outcome and the original confounders.
Result: When the outcome model is misspecified, CEM yields significantly higher bias than PSM. The paper provides a theoretical proposition (Proposition 1) demonstrating that in an exactly matched PSM design, a class of misspecified linear models (omitting non-linear or interaction terms) can still yield asymptotically unbiased ATT estimates. In contrast, CEM remains highly sensitive to model misspecification due to its structural residual confounding.

3. The Curse of Dimensionality and Coarsening Trade-offs

Finding: As the number of covariates increases, CEM with fine coarsening (e.g., "Auto") suffers from severe data loss (sparsity bias), leading to unstable estimates.
Result: Increasing the level of coarsening (e.g., "G3") mitigates data loss but introduces greater residual confounding and model dependence. Consequently, in high-dimensional settings, CEM performs worse than PSM in terms of bias, variance, and MSE, particularly when the outcome model is misspecified.

4. Heterogeneous Treatment Effects

Finding: The patterns observed for homogeneous effects persist under heterogeneous treatment effects.
Result: PSM consistently demonstrates lower bias and variance compared to CEM, especially when treatment prevalence is high and the outcome model is misspecified. Analytic estimators in PSM outperform model-based estimators in CEM under misspecification.

4. Significance and Claims

The paper concludes that CEM is not a viable alternative to PSM when matching on a large number of covariates.

Correction of Misconceptions: The study challenges the prevailing view in clinical literature that CEM mimics a fully blocked randomized experiment or is superior to PSM in reducing imbalance and model dependence. It clarifies that CEM is an inexact matching method that introduces structural bias.
Methodological Guidance: The authors argue that claims of CEM's superiority often stem from the use of inappropriate imbalance metrics (like Mahalanobis distance) that fail to account for chance variation. When evaluated with appropriate SMD-based metrics, PSM demonstrates superior covariate balance.
Robustness: PSM is shown to be more robust to model misspecification. While CEM reduces model dependence relative to unmatched data, it remains more dependent on correct outcome modeling than PSM.
Limitations: The paper acknowledges that PSM is not a panacea; it cannot address unmeasured confounding and relies on the correct specification of the propensity score model. However, within the scope of observed confounders, PSM offers a more robust framework for reducing bias and model dependence than CEM, particularly in high-dimensional settings.

The study advocates for a re-evaluation of CEM's role in observational research, suggesting that its theoretical advantages are often negated by practical limitations related to residual confounding and the curse of dimensionality.