Causal Effects in Matching Mechanisms with Strategically Reported Preferences

This paper proposes a robust identification approach to derive sharp bounds on the causal effects of school assignment despite strategic preference misreporting, demonstrating significant heterogeneity in graduation outcomes using data from Chile's university matching system.

The Gist

Imagine you are a high school student trying to get into your dream university. You have a list of schools you love, ranked from "My Heart's Desire" to "I'd settle for this." You submit this list to a central computer system that assigns you to a school based on your test scores and the school's cutoff line.

This sounds fair, right? But here's the catch: You know the rules, and you know the cutoffs. If you know your score is just barely enough for your dream school, you might be tempted to lie. You might skip your dream school on your list to avoid the risk of getting rejected and ending up with nothing, or you might rank a "safety" school higher than you actually prefer just to guarantee a spot.

This is the problem the paper tackles: How do we measure the true value of going to a specific school if the students are playing a game of chess with the admissions office?

The Core Problem: The "Lie" in the Data

The authors, Bertanha, Luflade, and Mourifíe, are like detectives trying to solve a crime. The "crime" is that the data we have (the lists students submitted) doesn't match the "truth" (what students actually wanted).

• The Old Way: Previous researchers tried to measure the effect of going to School A vs. School B by looking at students who barely made the cutoff. They assumed students told the truth.
• The Flaw: If students are lying strategically, comparing students just above and just below the cutoff is like comparing apples and oranges. The students just above might have lied to get in, while the students just below might have been honest but unlucky. The comparison is broken.

The Solution: The "Two-Step Detective" Approach

The authors propose a clever two-step method to fix this, which they call a "Control Mapping Approach." Think of it like this:

Step 1: The "Possibility Box" (Partial Identification)

Instead of trying to guess exactly what a student's true #1 choice was (which is impossible because they might have lied), the researchers build a "Possibility Box" for every student.

• The Metaphor: Imagine you see a student submit a list: School A, School B, School C. You don't know if they truly love A, or if they just put A first to be safe.
• The Logic: The researchers use math and game theory to say: "Okay, given the rules of the game and the fact that students are rational, their true top choice must be inside this specific box of possibilities."
• The Result: They don't know the exact truth, but they shrink the universe of possibilities. They know the true preference is somewhere in this box, and they can make the box smaller by making reasonable assumptions about how students behave (e.g., "If a student lists 3 schools, they probably truly like those 3 more than the ones they didn't list").

Step 2: The "Worst-Case Scenario" (Bounding)

Now that they have a "Possibility Box" for every student, they run their analysis. But since they don't have a single truth, they calculate Bounds.

• The Metaphor: Imagine you are trying to guess the average height of a group of people, but you only know that everyone is between 5 feet and 6 feet tall. You can't give an exact number, but you can say, "The average is definitely between 5'0" and 6'0"."
• The Application: The researchers calculate the best-case and worst-case scenarios for the effect of going to a specific school.
  • Worst Case: Maybe the students who got into the school were the ones who lied the most and would have failed anyway.
  • Best Case: Maybe they were the most motivated students who truly wanted to be there.
• The Payoff: If even in the "worst-case" scenario, the school still looks good, then you have a very strong result. If the "best" and "worst" cases are close together, you have a very precise answer.

The Real-World Test: Chile's University System

To prove their method works, they used data from Chile.

• The Setting: Chile has a massive system where 80,000 students apply to over 1,000 university programs.
• The Game: Students can only list up to 8 choices, but there are 1,000 options. This forces them to play the game strategically.
• The Evidence: The authors found that students are indeed lying. They saw students changing their lists abruptly right around the cutoff scores, proving they knew the cutoffs in advance and were gaming the system.

What Did They Find?

Using their "Possibility Box" method, they discovered some fascinating things that the old methods would have missed:

1. Preferences Matter: Students who truly wanted a specific major (like Medicine) were more likely to graduate from it, even if they were assigned to a different school. This suggests that "wanting" to be there is just as important as "being good" at it.
2. The "Safety" Trap: If you force a student who loves Medicine into a "safety" school (like a general science program), their chances of graduating drop significantly.
3. The Danger of Assuming Truth: When they compared their results to the "old way" (assuming everyone told the truth), they found that the old method was often wrong. In many cases, the old method said a school was great, but the new method showed it was actually a bad fit for those specific students.

The Big Takeaway

This paper is a toolkit for policymakers and researchers. It says: "Don't trust the data at face value when people are playing a game."

Instead of trying to find the one single "truth" (which is impossible when people lie), we should build a range of possibilities. By doing so, we can still make smart decisions about which schools work best, even when students are trying to outsmart the system. It's like navigating a foggy road: you might not see the exact destination, but you can still see the boundaries of the road and drive safely within them.

Technical Summary

Here is a detailed technical summary of the paper "Causal Effects in Matching Mechanisms with Strategically Reported Preferences" by Bertanha, Luflade, and Mourifié.

1. Problem Statement

The paper addresses a fundamental challenge in the causal evaluation of school and university assignment mechanisms: strategic misreporting of preferences.

• Context: Many centralized assignment mechanisms (e.g., Deferred Acceptance, DA) are used to allocate students to schools. While these mechanisms often rely on cutoffs (scores) to create quasi-experimental variation (Regression Discontinuity, RD), real-world mechanisms frequently incentivize students to misreport their true preferences due to constraints (e.g., limits on the number of schools they can rank) or costs.
• The Identification Gap: Standard RD strategies in matching literature (e.g., Kirkeboen et al., 2016; Abdulkadiroglu et al., 2022) typically control for reported local preferences to identify causal effects. However, when students act strategically, reported preferences ($P$) diverge from true preferences ($Q$).
  • Controlling for $P$ fails to isolate the causal effect conditional on true preferences ($Q$), which is necessary for valid counterfactual policy analysis.
  • Strategic behavior can be "discontinuous" (students change reporting behavior abruptly at cutoffs), potentially violating the continuity assumptions required for standard RD identification.
• Goal: The authors aim to derive sharp bounds on the causal effects of school assignment on future outcomes (e.g., graduation) that are robust to strategic reporting, specifically targeting parameters conditional on true local preferences.

2. Methodology

The authors propose a novel two-step "Control Mapping" approach. Unlike standard control function approaches that point-identify the control variable in the first step, this method partially identifies the control variable (true local preferences) and uses set theory to derive bounds.

Step 1: Partial Identification of Local Preferences

The first step constructs a set of possible true local preferences ($\mathcal{Q}_j$) for each student based on observed data (reported preferences $P$, scores $S$, and assignment $\mu$) and behavioral assumptions.

• Model Assumptions:
  • Cutoff Characterization: The matching outcome is determined by a student's best feasible school based on true preferences and admission cutoffs (stability).
  • Weak Partial Order (WPO): Students submit a subset of their acceptable schools ranked according to their true preferences. This is a rational strategy in constrained DA mechanisms.
  • Strong Partial Order (SPO): If a student submits fewer than the maximum allowed number of schools ($K$), they reveal their full list of acceptable schools.
  • Uniformly More Accessible Schools (UMAS): If school $e$ is uniformly more accessible than $d$ (everyone qualifying for $d$ qualifies for $e$), and a student lists $d$ but not $e$, the student must truly prefer $d$ over $e$.
• Outcome: For each student near a cutoff, the researcher constructs a random set $\mathcal{Q}_j$ that contains the student's true local preference pair $(j, k)$ with probability one. The size of this set depends on the strength of the behavioral assumptions (WPO yields larger sets; SPO + UMAS yields smaller sets).

Step 2: Partial Identification of Causal Effects

The second step uses the constructed sets $\mathcal{Q}_j$ to bound the average treatment effect (ATE).

• The Problem: The subpopulation of students near a cutoff with reported local preference $(j, k)$ contains a mix of students with true local preference $(j, k)$ and those with other true preferences. The proportion of the former is unknown.
• Solution:
  • The authors treat the true local preference as a "corrupted" variable.
  • They derive sharp bounds on the proportion of students in the subpopulation who actually have true local preference $(j, k)$ using the distribution of the random sets $\mathcal{Q}_j$ and the continuity of the score distribution.
  • Using these proportions (denoted $\delta^+$ and $\delta^-$), they apply results from Horowitz and Manski (1995) and Molinari (2020) to bound the conditional expectations $E[Y(j) | Q_j=(j,k), S_j=c_j]$ and $E[Y(k) | Q_j=(j,k), S_j=c_j]$.
  • The difference between these bounds yields the sharp bounds on the causal effect of assignment from $k$ to $j$.

3. Key Contributions

1. Novel Identification Strategy: The paper is the first to prove RD identification of returns to school assignment in matching mechanisms with strategically reported preferences. It introduces the "Control Mapping" approach, where the control variable is partially identified rather than point-identified.
2. Robustness to Strategic Behavior: The methodology explicitly accounts for the possibility that reported preferences are discontinuous functions of scores (a phenomenon observed in the data), which would otherwise invalidate standard RD designs controlling for reported preferences.
3. Sharp Bounds via Behavioral Assumptions: The authors provide a menu of behavioral assumptions (WPO, SPO, UMAS, Field constraints) that allow researchers to trade off between the strength of the assumption and the precision (width) of the bounds.
4. Theoretical Unification: The work bridges the literature on preference identification (Agarwal & Somaini, 2018) and causal inference in matching (Kirkeboen et al., 2016), showing how to combine them to estimate counterfactual policy effects.

4. Empirical Results (Chilean Data)

The authors apply their method to Chilean university assignment data (2004–2010), where students are constrained to rank at most 8 out of 1,000+ programs.

• Evidence of Strategic Behavior:
  • Cutoffs for selective programs (e.g., Medicine) are highly predictable ex-ante.
  • Application behavior changes discontinuously at cutoffs: students with scores just above a cutoff are significantly less likely to list a program as their first choice compared to those just below, suggesting they strategically drop "hope" schools when they know they won't get in, or vice versa.
• Heterogeneity in Graduation Outcomes:
  • Preferences Matter: Even controlling for test scores, students with different true local preferences (e.g., preferring Medicine vs. Math) have significantly different graduation probabilities when assigned to a common alternative (e.g., a general "Bachillerato" program). This suggests preferences are correlated with unobserved traits like effort or ability.
  • Treatment Effects: Assigning students to Medicine at PUC Santiago increases graduation rates compared to their next-best alternatives. However, the magnitude of this effect varies significantly depending on what the next-best alternative is (e.g., the effect is smaller if the alternative is Science at PUC vs. Medicine at another university).
• Impact of Assumptions:
  • Under the weakest assumption (WPO), bounds are wide, often failing to identify the sign of the treatment effect.
  • Under stronger assumptions (SPO + UMAS + Field constraints), bounds narrow significantly, often collapsing to point identification for a subset of pairs.
• Bias in Standard Methods:
  • Standard RD estimates (controlling for reported preferences) fall outside the authors' identified bounds in 12% to 25% of cases. This indicates that ignoring strategic behavior leads to severely biased estimates of the causal effect conditional on true preferences.

5. Significance

• Policy Relevance: The paper provides a rigorous framework for evaluating education policies (e.g., changing admission cutoffs or capacity) in environments where agents act strategically. It demonstrates that ignoring strategic behavior can lead to incorrect conclusions about the effectiveness of assignments.
• Methodological Advancement: By utilizing random set theory and partial identification, the authors offer a practical solution to a pervasive problem in matching markets. The approach is flexible, allowing researchers to incorporate context-specific assumptions to tighten bounds.
• Empirical Insight: The findings highlight that student preferences are a critical determinant of educational success, independent of test scores, and that these preferences are often correlated with unobserved factors that standard selection models fail to capture.

In summary, this paper provides a robust toolkit for causal inference in matching markets, proving that even in the presence of complex strategic behavior, meaningful bounds on causal effects can be derived, provided one moves beyond the assumption of truthful reporting.