📊 statistics

A Weighting Framework for Clusters as Confounders in Observational Studies

This paper proposes a unified weighting framework for clustered observational studies that clarifies the limitations of standard inverse propensity score weighting in achieving local balance and introduces two novel methods—hierarchical balancing weights and Mundlak balancing weights—that simultaneously control for both global and local imbalance under distinct assumptions.

Original authors: Eli Ben-Michael, Avi Feller, Luke Keele

Published 2026-02-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Eli Ben-Michael, Avi Feller, Luke Keele

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out if a new type of fertilizer makes plants grow taller. You have a garden full of plants, but they aren't all in the same spot. Some are in Greenhouse A, some in Greenhouse B, and some in Greenhouse C.

In a perfect world, you could just compare the treated plants to the untreated plants. But in the real world, Greenhouse A might be sunnier, Greenhouse B might have better soil, and Greenhouse C might be drafty. These "greenhouse effects" are clusters. If you don't account for them, you might think the fertilizer worked, when really, the plants just grew because they were in the sunny greenhouse.

This paper is about how to fix this problem when you are studying things like students in schools or patients in hospitals. The authors, Eli Ben-Michael, Avi Feller, and Luke Keele, propose a new way to "weight" the data to make a fair comparison.

Here is the breakdown of their ideas using simple analogies:

The Two Types of Imbalance

The authors say there are two ways your data can be "unfair" or "imbalanced":

Global Imbalance (The "Average" Problem): This is when the treated plants in all greenhouses are, on average, different from the untreated plants. Maybe the treated plants are generally bigger to begin with, regardless of which greenhouse they are in.
Local Imbalance (The "Roommate" Problem): This is when, inside a specific greenhouse, the treated plants are different from the untreated ones. For example, in Greenhouse A, the treated plants might be on the sunny side, while the untreated ones are in the shade.

The Old Way (The "Random Intercept" Method):
For a long time, researchers used a standard method (called Random Intercept IPW) that mostly fixed the Global Imbalance. It was like saying, "Okay, on average, the treated plants look similar to the untreated plants across the whole garden."

The Flaw: It ignored the Local Imbalance. It didn't care that inside Greenhouse A, the treated plants were getting all the sun. It was like comparing apples to oranges just because the average weight of the apples in the whole orchard matched the oranges.

The New Framework: A Unified Solution

The authors developed a new "Weighting Framework" that tries to fix both problems at once. They propose two main tools to do this:

Tool 1: Hierarchical Balancing Weights (The "Strict Roommate" Approach)

This method is very strict. It says, "We don't just want the averages to match; we want the treated and untreated plants to look identical inside every single greenhouse."

How it works: It uses a mathematical optimization (like a super-advanced calculator) to assign weights to every plant so that the treated and untreated groups look exactly alike within their specific greenhouse.
The Catch: It's very picky. If a greenhouse is tiny (like a small school with only 5 students) or if everyone in a greenhouse got the treatment (no control group), this method has to throw that greenhouse out of the study. This reduces the amount of data you have to work with.

Tool 2: Mundlak Balancing Weights (The "Smart Summary" Approach)

This is the authors' new invention. Instead of treating every greenhouse as a unique, mysterious entity, this method says, "Let's summarize what makes each greenhouse special."

The Analogy: Imagine instead of looking at every single plant in Greenhouse A, you just look at the average height of plants in Greenhouse A and the percentage of plants that got fertilizer there. You create a "summary card" for the greenhouse.
How it works: The method balances the treated and untreated plants based on these "summary cards" rather than the specific greenhouse ID.
The Benefit: Because it uses summaries, it doesn't have to throw out tiny greenhouses. It can handle small groups that the "Strict Roommate" method would reject.
The Catch: It relies on a stronger assumption. It assumes that these "summary cards" capture everything important about the greenhouse. If there is some secret factor in Greenhouse A that the summary missed, the results could be wrong.

What the Experiments Showed

The authors tested these methods in two real-world scenarios:

Education (Small Clusters): They looked at students in schools. Many schools were small.
- Result: The old method (Global only) gave a very different answer than the new methods. The new methods (which fixed local imbalance) agreed with each other and suggested the effect of the program was smaller than the old method claimed. The "Smart Summary" method worked well even with small schools.
Health (Large Clusters): They looked at patients in hospitals. These hospitals were huge.
- Result: Again, the old method gave a misleading result (suggesting surgery was harmful). The new methods, which fixed the local imbalance, suggested surgery had no real effect (neither good nor bad). Since the hospitals were big, the "Strict Roommate" method worked fine, but the "Smart Summary" method gave similar results, giving researchers confidence.

The Bottom Line

The paper argues that researchers should stop using the old "Global only" method because it often misses hidden biases inside specific groups.

If you have big groups (like large hospitals), use the Hierarchical method. It's safer because it doesn't rely on strong assumptions.
If you have small groups (like small schools) or groups where everyone got the treatment, use the Mundlak method. It saves your data from being thrown away, but you have to trust that your "summary cards" tell the whole story.

In short: To get the truth, you need to balance the data not just across the whole garden, but inside every single greenhouse.

Technical Summary: A Weighting Framework for Clusters as Confounders in Observational Studies

Problem Statement

In observational causal inference, researchers often face the challenge of confounding when units are nested within groups (clusters), such as students in schools or patients in hospitals. Standard approaches typically adjust for individual-level covariates, assuming strong ignorability holds conditional on these covariates. However, when cluster membership influences both treatment assignment and outcomes, this assumption may be violated due to unmeasured cluster-level context.

The core problem addressed is how to appropriately adjust for cluster-level confounding using weighting methods. Existing literature relies heavily on model-based Inverse Propensity Score Weighting (IPW) with random effects (specifically random intercepts). The authors argue that the current standard—random-intercept IPW—targets only global balance (differences between treated and control units across the entire sample) and constant level shifts across clusters. Crucially, it imposes no constraints on local balance (differences between treated and control units within specific clusters). When local imbalances exist, random-intercept IPW may fail to reduce bias, particularly in settings with small clusters or sparse data. Furthermore, methods that attempt to achieve both global and local balance often face computational infeasibility or require dropping small clusters, thereby altering the target estimand.

Methodology

The paper develops a unified weighting framework that decomposes the bias of weighting estimators into two distinct components: global balance and local balance.

1. Bias Decomposition Framework

The authors derive a bias decomposition for the Average Treatment Effect on the Treated (ATT). They show that the bias in estimating the counterfactual outcome $\mu_0$ consists of:

Global Balance: The difference in the common component of the outcome model between treated and control units across the entire sample.
Local Balance: The difference in the interaction component (specific to each cluster) between treated and control units within each cluster.
Normalization: A term related to whether weights sum to the number of treated units within each cluster.

This decomposition clarifies that achieving the overall ATT requires controlling both global and local imbalances, though the relative importance depends on the heterogeneity of the outcome process across clusters.

2. Review of Existing Methods (Adjusting for Cluster Membership)

The paper categorizes methods that condition on cluster membership ( $G$ ) under Assumption 2 (Strong Ignorability given $X$ and $G$ ):

Random-Intercept IPW (RI-IPW): The current standard. It includes cluster-specific intercepts but no interactions between covariates and clusters. It targets global balance and normalization but fails to control local balance.
Stratified Analysis: Targets only local balance by estimating propensity scores separately within each cluster. This is often infeasible due to complete separation or small sample sizes and ignores global balance.
Random Coefficients IPW: Targets both global and local balance by allowing covariate effects to vary by cluster. This is often computationally infeasible and suffers from convergence issues.
Hierarchical Balancing Weights: A constrained optimization approach that directly minimizes local imbalance while enforcing exact global balance. It targets both forms of balance but requires dropping clusters with no treatment variation (violating overlap), which shifts the estimand.

3. Proposed Methods (Adjusting for Cluster-Level Sufficient Statistics)

To address the sparsity issues in small clusters, the authors propose methods based on Assumption 3 (Exponential Family Cluster Ignorability). This assumption posits that treatment is ignorable given individual covariates and cluster-level sufficient statistics ( $\bar{S}_G$ ), rather than cluster membership itself. This allows clusters to be treated as "anonymous" if they share similar sufficient statistics.

Generalized Mundlak Estimation: An approach that adjusts for cluster-level sufficient statistics (e.g., means of covariates and treatment proportions within the cluster) instead of fixed effects.
Mundlak Balancing Weights: A novel estimator that uses constrained optimization to balance:
1. Individual-level covariates (Global Individual Balance).
2. Cluster-level sufficient statistics (Global Cluster Balance).
3. Interactions between individual covariates and cluster-level sufficient statistics (Local Balance).

This approach reduces the dimensionality of the local balance constraints from $d \times K$ (covariates $\times$ clusters) to the dimension of the sufficient statistics, making it feasible to retain small clusters without dropping them. The authors outline two variants: one enforcing global balance on sufficient statistics and another enforcing an "average-to-one" constraint within clusters.

Key Contributions

Unified Framework: The paper provides a formal decomposition of weighting bias into global and local components, clarifying exactly what different estimation methods control.
Critique of RI-IPW: The authors demonstrate that the widely used random-intercept IPW method fails to control local imbalance, potentially leading to biased estimates in the presence of cluster-level confounding.
Hierarchical Balancing Weights: They formalize a method that directly targets both global and local balance via optimization, highlighting the trade-off between balance and effective sample size (ESS).
Mundlak Balancing Weights: They introduce a new estimator that leverages the Generalized Mundlak approach to achieve local balance in small clusters by conditioning on sufficient statistics rather than fixed effects. This avoids dropping clusters with no treatment variation.
Assumption Trade-offs: The paper explicitly contrasts the assumptions required:
- Assumption 2 (Weaker): Non-parametric regarding cluster effects; valid for Hierarchical Balancing Weights but struggles with small clusters.
- Assumption 3 (Stronger): Requires an exponential family structure for the joint distribution of covariates and treatment; valid for Mundlak methods and allows for small clusters but risks bias if the sufficient statistics are misspecified.

Results

Simulation Study

Using a data generating process with unobserved cluster-level confounding:

RI-IPW: Performed poorly, exhibiting high bias comparable to or worse than methods that ignored clusters entirely. It failed to reduce bias from cluster-level confounding.
Hierarchical Balancing Weights: Effectively reduced bias when cluster confounding was present but incurred a cost (bias) when clusters were irrelevant.
Mundlak Balancing Weights: Performed well across all scenarios, remaining nearly unbiased whether or not cluster-level confounding was present. They avoided the penalty of irrelevant clusters that affected other cluster-based methods.

Empirical Applications

The authors applied the methods to two distinct datasets:

Education (Small Clusters): A study of pre-school programs on test scores (937 schools, small cluster sizes).
- RI-IPW produced a large treatment effect (0.25 SD) with substantial covariate imbalance.
- Hierarchical and Mundlak balancing weights produced similar, smaller estimates (0.07 SD) with excellent balance.
- The agreement between the two methods suggested the stronger Assumption 3 was not grossly violated.
- Hierarchical weights required dropping clusters with no treatment variation, while Mundlak weights retained them.
Health Services (Large Clusters): A study of emergency surgery for HPB disorders (540 hospitals, large cluster sizes).
- RI-IPW again produced an outlier estimate with poor balance.
- Hierarchical and Mundlak weights yielded similar, near-zero estimates, suggesting no benefit of surgery once hospital context was accounted for.
- In this setting with large clusters, the authors note that Hierarchical Balancing Weights are preferable as they avoid the stronger exponential family assumption while still being computationally feasible.

Significance and Claims

The paper claims that researchers should generally prefer balancing weights approaches over traditional model-based IPW with random intercepts, particularly when treatment assignment varies within clusters.

When to use Hierarchical Balancing Weights: When cluster sizes are large enough to permit direct conditioning on cluster membership, this method is preferred because it relies on the weaker, non-parametric Assumption 2.
When to use Mundlak Balancing Weights: When cluster sizes are small or overlap violations exist within clusters, the Mundlak approach is attractive as it can retain these clusters. However, this comes with the caveat that researchers must believe the exponential family assumption (Assumption 3) holds and that the chosen sufficient statistics fully capture cluster-level confounding.
Practical Recommendation: The authors suggest comparing estimates across multiple methods (no adjustment, RI-IPW, Hierarchical, and Mundlak). Agreement between Hierarchical and Mundlak estimates provides reassurance that the stronger Assumption 3 is not grossly violated. Disagreement may indicate misspecification of sufficient statistics or failure to achieve local balance.

The paper concludes that while the Mundlak approach offers a solution for small clusters, the choice between methods ultimately depends on the plausibility of the exponential family assumption and the feasibility of conditioning directly on cluster membership.