Incentive-Aware Synthetic Control: Accurate Counterfactual Estimation via Incentivized Exploration

1. Problem Statement

The paper addresses a critical limitation in Synthetic Control Methods (SCMs) used for counterfactual inference in panel data (e.g., estimating the impact of marketing campaigns or policy changes).

• The Core Issue: Standard SCMs rely on the Unit Overlap Assumption. This assumption posits that the potential outcomes of a treated unit under the control condition can be expressed as a linear (or convex) combination of the outcomes of "donor" units (units that remained under control).
• The Failure Mode: In many real-world internet economy settings, units self-select their interventions based on private preferences. If subpopulations with different preferences self-select into different interventions (e.g., heavy users choose a subscription plan while light users choose pay-as-you-go), the resulting groups may have fundamentally different latent structures. Consequently, the treated group's outcomes cannot be linearly reconstructed from the control group's outcomes, violating the overlap assumption and rendering standard SCM estimates invalid.
• The Constraint: The platform (principal) cannot force units to take specific interventions (no forced compliance); it can only provide recommendations. The challenge is to design a recommendation policy that incentivizes units to explore interventions they would not normally choose, thereby satisfying the overlap assumption without violating individual rationality.

2. Methodology

The authors propose a framework combining Information Design and Online Learning (specifically, Incentivized Exploration in Multi-Armed Bandits) to solve this problem.

A. Theoretical Framework

• Latent Factor Model: Outcomes are generated via a latent factor model: $E[y_{i,t}^{(d)}] = \langle u_t^{(d)}, v_i \rangle$, where $u_t^{(d)}$ is a time/intervention-dependent factor and $v_i$ is a unit-specific latent factor.
• Bayesian Incentive Compatibility (BIC): The recommendation policy must be BIC. This means that for any unit $i$, following the recommendation $d$ must maximize their expected utility given their prior beliefs and the information conveyed by the recommendation.
• Necessity of Overlap: The paper proves (Theorem 3.1) that without the Unit Overlap Assumption, no algorithm can produce consistent counterfactual estimates, even with infinite data, if units self-select based on non-overlapping latent subspaces.

B. The Proposed Algorithm: Incentivized Exploration

The authors introduce Algorithm 1, which operates in two phases to satisfy the overlap assumption for a specific target group (e.g., Type 1 units who prefer treatment but need to be incentivized to try control).

1. Exploration Phase (Warm-up):
   • The principal observes the first $N_0$ units without making recommendations.
   • Units self-select based on their priors. This phase ensures that enough data is collected to learn the latent structure of the "preferred" intervention (e.g., treatment for Type 1 units), satisfying the overlap assumption for that specific intervention.
2. Incentivized Phase (Batched Recommendations):
   • The remaining units are processed in batches of size $L$.
   • Exploit vs. Explore: Within each batch, the principal randomly selects one unit to be an "explore" unit (recommended to take the non-preferred intervention, e.g., control) and the rest are "exploit" units (recommended to take their preferred intervention).
   • Hidden Exploration: Units do not know if they are in the explore or exploit group.
   • Mechanism:
     • If a unit receives a recommendation for their preferred intervention, they know it is likely an "exploit" recommendation and follow it.
     • If a unit receives a recommendation for the non-preferred intervention (e.g., control for a Type 1 unit), they are uncertain. However, because the probability of being an "explore" unit is low ($1/L$), and the principal has better information (from the history $H_i$) showing that the non-preferred intervention is actually beneficial for this specific unit type, the unit is incentivized to follow the recommendation to maximize expected utility.
   • Result: By setting $L$ sufficiently large, the principal can force a sufficient number of Type 1 units to take the control intervention. This creates a "synthetic control" group that overlaps with the treated group, satisfying the Unit Overlap Assumption.

C. Extension to Synthetic Interventions

The method is extended (Algorithm 2) to handle $k > 2$ interventions. The algorithm generalizes the "exploit" logic to ensure that for every sub-type of unit, the overlap assumption holds for all interventions, not just control.

D. Hypothesis Testing

The paper provides two statistical tests to determine if the Unit Overlap Assumption holds for a given dataset a priori:

1. Non-asymptotic Test: Uses Principal Component Regression (PCR) on the first half of the pre-intervention period to predict the second half. Large prediction errors indicate a violation of overlap.
2. Asymptotic Test: Leverages the convergence properties of Synthetic Interventions (SI) estimators to construct a test statistic that follows a standard normal distribution under the null hypothesis of overlap.

3. Key Contributions

1. Identification of the Overlap Gap: The paper formally identifies that self-selection based on heterogeneous preferences breaks the standard overlap assumption in SCMs, making traditional counterfactual estimation impossible.
2. Incentive-Aware SCM: It is the first work to integrate incentivized exploration into the synthetic control framework. It demonstrates how a principal can use information design (recommendations) to manipulate the data distribution to satisfy the necessary statistical assumptions.
3. Theoretical Guarantees:
   • Proves that the proposed algorithm satisfies the BIC property.
   • Proves that with sufficient data ($N_0$ and batch size $L$), the Unit Overlap Assumption is satisfied with high probability for all units and interventions.
   • Establishes that valid finite-sample counterfactual estimates can be obtained using off-the-shelf SCMs after running the algorithm.
4. Diagnostic Tools: Introduces practical hypothesis tests to verify whether the overlap assumption holds before applying SCMs.

4. Results

• Numerical Simulations:
  • The authors simulate a streaming service scenario with two plans (subscription vs. pay-as-you-go) and two user types.
  • Performance: When the Unit Overlap Assumption is violated (due to self-selection), standard SCMs fail (high estimation error). The proposed Incentive-Aware SCM significantly reduces estimation error, converging to the true counterfactual as the number of units increases.
  • Robustness: The algorithm remains effective even when units have varying "strength of beliefs" (gaps in prior utility) and different latent factor dimensions.
  • Non-Compliance: The method shows robustness even when a fraction of units ignore recommendations and act on their priors, though error increases with non-compliance rates.
• Hypothesis Test Validation: The asymptotic hypothesis test correctly identifies non-overlapping datasets with a False Positive Rate (FPR) of $\approx 5\%$ (matching the theoretical significance level) and a True Positive Rate (TPR) of $\approx 95\%$.

5. Significance

• Bridging Causal Inference and Mechanism Design: This work bridges the gap between causal inference (SCMs) and mechanism design (incentivized exploration). It moves beyond the assumption that data is passively collected to a setting where the data collection process is actively managed to ensure statistical validity.
• Practical Applicability: In the internet economy (ride-sharing, advertising, e-commerce), platforms often cannot force users into control groups. This paper provides a rigorous, theoretically grounded method for these platforms to run valid A/B tests or policy evaluations without forced compliance.
• Relaxing Assumptions: By showing how to induce the overlap assumption rather than assuming it holds, the paper expands the applicability of SCMs to domains with high heterogeneity and strategic user behavior.

In summary, the paper solves the "chicken-and-egg" problem of synthetic control: you need overlap to estimate counterfactuals, but self-selection prevents overlap. The solution is to use smart recommendations to create the necessary overlap through incentivized exploration.