📊 statistics

Multilevel Regression Discontinuity Models with Latent Variables

This paper extends the single-level regression discontinuity framework with latent variables to multilevel contexts, proposing models for both hierarchical and multisite designs that account for clustered data structures and demonstrate the accurate recovery of average treatment effects, including extrapolated estimates, through Monte Carlo simulations.

Original authors: Monica Morell, Youngjin Han, Muwon Kwon, Youjin Sung, Yang Liu, Ji Seung Yang

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Monica Morell, Youngjin Han, Muwon Kwon, Youjin Sung, Yang Liu, Ji Seung Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a school principal trying to figure out if a new tutoring program actually helps students. You decide to use a "cutoff rule": if a student scores below 60 on a math test, they get the tutoring (Treatment); if they score 60 or above, they don't (Control).

This is called a Regression Discontinuity (RD) design. It's a clever way to guess cause-and-effect without randomly assigning students, by assuming that students who score 59 are basically the same as those who score 61, except one got help and the other didn't.

However, this paper introduces a few major problems with the old way of doing this and offers a shiny new "Multilevel Latent" solution. Here is the breakdown using simple analogies.

1. The Problem: The "Noisy" Test Score

In the real world, a single test score isn't a perfect measure of a student's true ability. It's like trying to guess the temperature of a room by looking at a thermometer that is slightly broken or shaking in the wind.

The Old Way: Researchers used the raw test score (the "Noisy Thermometer") as the deciding factor.
The New Way: The authors say, "Let's stop guessing based on the noisy number. Let's try to estimate the student's True Ability (the actual temperature) behind the noise." They call this the Latent Variable.

2. The Problem: The "Nested" Mess (Multilevel Data)

Schools aren't just a pile of individual students; they are organized. Students are in classrooms, and classrooms are in schools. Students in the same school often share similar environments (good teachers, bad funding, etc.).

The Old Way: Traditional models treated every student as an independent island, ignoring that they live in the same "neighborhood" (school). This is like trying to understand traffic by looking at one car at a time, ignoring that they are all stuck in the same jam.
The New Way: This paper builds a model that respects the hierarchy. It understands that students are nested inside schools, and schools are nested inside districts.

3. The Solution: Two New "Blueprints"

The authors created two specific models to handle different scenarios, like having two different keys for two different locks:

Key A: The "Whole School" Key (Hierarchical RD)
- Scenario: The rule applies to the whole school. If the average score of a school is low, the entire school gets the tutoring.
- Analogy: Imagine a neighborhood where if the average income drops below a certain line, the whole neighborhood gets a new park. You can't give the park to just one house; it's all or nothing for the block.
- The Model: This model looks at the school's average "True Ability" to decide who gets the park, while acknowledging that individual students still vary within that school.
Key B: The "Individual" Key (Multisite RD)
- Scenario: The rule applies to individuals, but they are still grouped in schools. If Student A scores low, Student A gets tutoring, even if their classmate Student B doesn't.
- Analogy: Imagine a city-wide program where anyone with a low credit score gets a loan, but everyone is still part of a specific family unit (school) that influences their credit.
- The Model: This model tracks individual "True Ability" but remembers that students in the same family (school) are related.

4. The Superpower: Seeing Beyond the Cutoff

This is the coolest part of the paper.

The Old Limitation: In traditional RD, you can only measure the effect of the tutoring right at the cutoff line (e.g., exactly at score 60). It's like trying to judge how good a chef is only by tasting the one dish they made exactly at 6:00 PM. You can't say anything about the food they made at 5:59 PM or 6:01 PM.
The New Superpower: Because the new model accounts for the "True Ability" behind the noisy test scores, it can extrapolate.
- Analogy: Because we know the "True Temperature" of the room, we can predict how the heating system works not just at 60 degrees, but at 50 degrees or 70 degrees too.
- Result: We can now estimate if the tutoring helps students who scored 40, or 50, or 80, not just those right on the edge.

5. The "Heterogeneity" (Everyone is Different)

The old models assumed the tutoring helped everyone the exact same amount.

The New Insight: The new model admits that the tutoring might help a struggling student (low True Ability) a lot, but help a high-achiever (high True Ability) very little.
Analogy: Think of a raincoat. It works great for a drizzle (low ability), but if you are in a hurricane (high ability), the raincoat might not matter as much, or maybe you need a different kind of coat. The model can now map out exactly who benefits most.

Summary

This paper is like upgrading from a black-and-white, flat map of education research to a 3D, high-definition GPS.

It fixes the "blurry lens" of noisy test scores by looking at the True Ability underneath.
It respects the hierarchy of schools and classrooms so the data doesn't get confused.
It allows researchers to predict outcomes far away from the cutoff line, not just right next to it.
It tells us who the program helps most, rather than just giving an average number.

The authors ran thousands of computer simulations (like a video game test) to prove that their new GPS works accurately, provided you have enough schools in your dataset. This opens the door for much smarter, more nuanced educational policies in the future.

1. Problem Statement

Regression Discontinuity (RD) designs are widely used in education research to estimate causal treatment effects without randomization. However, two significant limitations hinder their application in educational contexts:

Multilevel Data Structure: Educational data is inherently hierarchical (students nested within classrooms/schools). Standard RD models often violate the independence assumption required by traditional parametric procedures, leading to biased estimates.
Measurement Error in the Running Variable: RD designs rely on a "running variable" (RV) to assign treatment. In education, this is often an observed test score (Observed Running Variable, ORV) which serves as a noisy proxy for an underlying latent construct (Latent Running Variable, LRV), such as true academic ability. Traditional RD models treat the ORV as the true score, ignoring measurement error.

While a previous framework by Morell et al. (2025) successfully integrated latent variables into single-level RD, it could not handle the multilevel structures common in education. This study aims to extend the latent RD framework to Multilevel contexts.

2. Methodology

The authors propose a unified framework integrating Multilevel RD (MLRD) with Latent Variable (LV) modeling. They address two distinct design scenarios:

A. Model Specifications

The framework utilizes a Measurement Model (to link observed indicators to the LRV) and a Structural Model (to link the LRV to the outcome).

Measurement Model:
- Assumes the LRV ( $\theta_{ij}$ ) consists of a cluster-level component ( $\theta_j$ ) and an individual-level deviation ( $\delta_{ij}$ ).
- Uses a Two-Parameter Logistic (2PL) Item Response Theory (IRT) model to relate $K$ observed indicators (e.g., test items) to the LRV.
- Accounts for measurement error, acknowledging that the ORV is an imperfect proxy for the LRV.
Structural Models:
- Hierarchical RD (HRD): Treatment is assigned at the cluster level (e.g., schools). The treatment indicator $T_j$ $T_{j}$ is determined by the cluster's average ORV score relative to a cutoff.
  - Random Effects: Random intercepts for clusters.
  - Key Feature: Estimates cluster-level Average Treatment Effects (ATE).
- Multisite RD (MRD): Treatment is assigned at the individual level within clusters (e.g., students in sampled schools). The treatment indicator $T_{ij}$ $T_{ij}$ is determined by the individual's ORV score.
  - Random Effects: Random intercepts and random slopes for the treatment effect.
  - Key Feature: Estimates individual-level ATEs while accounting for clustering.

B. Estimation Algorithm

The paper employs the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm for parameter estimation.

Why MH-RM? It is a stochastic approximation algorithm that avoids high-dimensional numerical integration, making it computationally efficient for models with multiple latent variables and random effects.
Process:
1. Stochastic Imputation: Draws missing data (latent variables and random effects) from their posterior predictive distribution.
2. Stochastic Approximation: Estimates the gradient of the observed data log-likelihood using Monte Carlo samples.
3. Robbins-Monro Update: Recursively updates parameters using a decaying gain sequence to converge to the Maximum Likelihood Estimator (MLE).
4. Standard Errors: Calculated using the Louis formula based on Monte Carlo imputations.

3. Key Contributions

The study offers several theoretical and practical advancements:

Extrapolation of ATE: Unlike conventional RD, which only identifies effects at the cutoff, the latent framework allows for the extrapolation of ATEs to ORV scores away from the cutoff. Because measurement error implies a positive probability of treatment assignment at any LRV value, the model can estimate treatment effects across the entire domain of the LRV.
Quantification of Heterogeneity: The model quantifies Heterogeneous Treatment Effects (HTE). It calculates the variance of the treatment effect conditional on the ORV, allowing researchers to understand how treatment effects vary among individuals or clusters with the same observed score.
Dual-Level Inference: The framework provides distinct inference paths for HRD (cluster-level effects) and MRD (individual-level effects), accommodating different policy implementation scenarios.
Integration of IRT and RD: It formally bridges Item Response Theory (for handling noisy test scores) with causal inference (RD), correcting for the bias introduced by using observed test scores as the forcing variable.

4. Simulation Results

A Monte Carlo simulation study was conducted with 500 replications to validate the model.

Conditions: Varied cluster sizes ( $J=100, 300, 500$ ), cluster sizes per group ( $I_j=30$ ), and effect sizes.
Parameter Recovery:
- Both HRD and MRD models showed low bias and acceptable Root Mean Squared Error (RMSE) for structural parameters.
- Cluster Size Impact: Parameter recovery improved significantly with larger numbers of clusters ( $J$ ). The MRD model generally performed better than HRD due to the larger number of units contributing to individual-level estimates.
ATE Estimation:
- The models accurately recovered ATEs at the cutoff and extrapolated them to values away from the cutoff (e.g., $c \pm 1$ for HRD, $c \pm 2$ for MRD).
- Efficiency: While bias remained low, RMSE increased as the evaluation point moved further from the cutoff, indicating reduced precision in extrapolation.
- Coverage: 95% confidence intervals maintained accurate coverage rates (falling within the Monte Carlo confidence band), particularly when $J \geq 300$ .
Heterogeneity: The models successfully quantified the variance of treatment effects, demonstrating that treatment effects are not uniform even among subjects with the same ORV.

5. Significance and Implications

Broader Applicability in Education: This framework removes the constraint that RD studies must use aggregate, error-free data. It allows researchers to use standard test scores (which are inherently noisy) as running variables while rigorously accounting for measurement error and data hierarchy.
Policy Relevance: By enabling the extrapolation of treatment effects, policymakers can evaluate how an intervention might perform for students slightly above or below the eligibility cutoff, supporting decisions on adjusting cutoff scores or expanding programs.
Methodological Rigor: The study provides a robust statistical solution for a common real-world problem: nested data with measurement error. It moves beyond the "local" average treatment effect (LATE) at the cutoff to provide a more comprehensive view of program effectiveness.

Limitations & Future Directions:
The authors note that the current models assume unidimensional latent structures and balanced cluster sizes. Future work should explore multidimensional measurement models, unbalanced designs, and the inclusion of latent covariates. Additionally, extrapolation too far from the cutoff requires caution due to decreasing probabilities of treatment assignment and potential lack of practical relevance.