Technical Summary: Counterfactual Forecasting for Panel Data

Problem Statement

The paper addresses the challenge of forecasting counterfactual outcomes in panel data settings characterized by missing entries and temporally dependent latent factors. This scenario is prevalent in causal inference, particularly when prospective decision-making requires estimating unobserved potential outcomes (e.g., what would happen to a unit under a treatment not yet assigned) ahead of time.

Existing methods for causal inference, such as Synthetic Controls, Difference-in-Differences, and Matrix Completion, typically rely on low-rank assumptions to model shared variation across units and time. However, these approaches often treat latent factors as static or i.i.d., failing to exploit the temporal dynamics (e.g., autoregressive structures) inherent in many real-world processes. Furthermore, many existing forecasting methods assume limited missingness or require fully observed pre-treatment panels, rendering them unsuitable for counterfactual estimation where large portions of the data matrix are unobserved by design.

Methodology: FOCUS

The authors propose FOCUS (FOrecasting Counterfactuals Under Stochastic dynamics), a method designed to extend traditional matrix completion by incorporating time series dynamics of the latent factors.

Core Algorithm

FOCUS operates in two primary steps:

1. Factor Estimation: The method restricts the observed panel under a specific treatment (or control) condition, treating the unobserved entries as missing. It employs a Principal Component Analysis (PCA) algorithm (specifically the consistent estimator by Xiong & Pelger, 2023) to estimate the latent factors ($\hat{F}_t$) and loadings ($\hat{\Lambda}_i$) from the partially observed data. This step accommodates general observation patterns, including missing completely at random (MCAR) and staggered adoption designs.
2. Dynamic Forecasting: Once the factors are estimated, FOCUS fits a time series model (specifically a Vector Autoregressive model of order 1, VAR(1)) to the estimated factors $\{\hat{F}_t\}$. The coefficient matrix $\hat{A}$ is estimated via Ordinary Least Squares (OLS). The forecast for the mean potential outcome at horizon $h$ for unit $i$ is then constructed as a plug-in estimator:
   $$\hat{\theta}_{i, T:T+h} = \hat{\Lambda}_i^\top \hat{A}^h \hat{F}_T$$
   This approach effectively combines time series smoothing/filtering techniques with matrix completion.

Theoretical Framework

The paper establishes theoretical guarantees under standard regularity conditions:

• Factor Model: The latent factors follow a stationary VAR(1) process ($F_t = A F_{t-1} + \eta_t$) with $\rho(A) < 1$.
• Noise and Loadings: Idiosyncratic noise and factor loadings are assumed to be independent with bounded moments.
• Observation Mechanism: The observation matrix $W$ is independent of the factors and noise, with conditions ensuring sufficient overlap (e.g., $|Q_{s,t}| \geq Nq$).

Key Theoretical Results:

• Error Bounds: The authors derive an error bound for the forecast estimator $\hat{\theta}_{i, T:T+h}$, showing it converges at a rate of $O_P(\delta_{NT}^{-1})$ plus a term dependent on the forecast horizon $h$ and the spectral radius of $A$.
• Asymptotic Normality: Under additional regularity conditions on the observation pattern, the estimator is shown to be $\min\{\sqrt{N}, \sqrt{T}\}$-consistent and asymptotically normal.
• Confidence Intervals: The paper provides a framework for constructing asymptotically valid confidence intervals for the counterfactual forecasts by estimating the asymptotic variance, which comprises components from factor estimation uncertainty and forecasting uncertainty.

Key Contributions

1. First Entry-by-Entry Out-of-Sample Forecasting: To the authors' knowledge, FOCUS is the first work to address entry-by-entry forecasting of counterfactuals out-of-sample in the presence of missing data and stochastic factor dynamics.
2. Integration of Dynamics and Missing Data: Unlike previous matrix completion methods that ignore temporal dynamics or dynamic factor models that assume limited missingness, FOCUS explicitly handles wide ranges of observation patterns while leveraging stochastic temporal dependencies (e.g., autoregressive components).
3. Theoretical Extension: The work extends the theoretical results of Xiong & Pelger (2023) from i.i.d. latent factors to temporally dependent factors, verifying that dependent factor processes satisfy general Central Limit Theorem (CLT) assumptions.
4. Scalability: The algorithm is computationally efficient, with runtime scaling favorably as the time dimension $T$ increases, making it suitable for large-$T$ settings.

Empirical Results

The authors evaluate FOCUS against two benchmarks: mSSA (Multivariate Singular Spectrum Analysis) and SyNBEATS (a neural network-based synthetic control method).

• Simulation Studies: Across four data generating processes (DGPs) involving AR(1), ARMA, and VARMA processes with various observation patterns (MCAR, simultaneous adoption), FOCUS consistently achieves lower Mean Squared Forecast Error (MSFE) than both benchmarks. Notably, FOCUS demonstrates faster decay rates of error as $T$ increases. It also outperforms benchmarks under model misspecification (e.g., when true factors follow ARMA processes but the model assumes VAR(1)).
• Computational Efficiency: FOCUS runs faster than mSSA and substantially faster than SyNBEATS.
• Real-World Application (HeartSteps): The method is applied to the HeartSteps V1 mobile health study, which involves 37 users receiving walking prompts. The study highlights that step counts exhibit negative correlations between consecutive decision slots. By capturing these temporal patterns in the latent factors, FOCUS provides more accurate forecasts of future step counts under intervention compared to mSSA, with the performance gap widening as the training horizon $T$ increases.

Significance and Claims

The paper claims that FOCUS provides a scalable and theoretically grounded framework for prospective counterfactual evaluation. Its primary significance lies in bridging the gap between causal inference (handling missing counterfactuals) and time series forecasting (handling temporal dynamics).

The authors emphasize that while many existing methods focus on post-treatment estimation or assume fully observed pre-treatment data, FOCUS is uniquely equipped to forecast future outcomes under hypothetical interventions when data is incomplete. The method is particularly effective when latent factors exhibit autoregressive dynamics, a common feature in behavioral and economic data. The paper concludes by noting limitations, such as the reliance on stationarity, and suggests future directions including non-stationary settings (e.g., state-space models) and doubly robust estimators.