Technical Summary: Performative Learning Theory

1. Problem Statement

Machine learning systems have evolved from passive analyzers of data to active shapers of the world. In many domains, the deployment of predictive models influences the very outcomes they aim to forecast, a phenomenon known as performative predictions (PP). While prior work (e.g., Perdomo et al., 2020; Brown et al., 2022) has formalized PP and studied the stability and optimality of models under population-level feedback loops, a critical gap remains: generalization under performativity.

Specifically, existing literature does not adequately address how well models generalize from a finite training sample to a population when:

1. The sample itself reacts to the model's predictions (e.g., users in a beta test changing behavior based on app recommendations).
2. The population reacts to the predictions.
3. Both react, potentially in conflicting ways.

The central question is whether and how well models trained on performative samples can generalize to a performative population, and what the theoretical bounds on this generalization error are.

2. Methodology and Framework

The authors introduce Performative Learning Theory (PLT), embedding performative predictions into statistical learning theory. The framework generalizes the standard setting by considering the evolution of both the sample distribution ($\hat{d}_t$) and the population distribution ($d_t$) over time $t$.

Key Definitions and Assumptions

• Stateful Transition: The distribution evolution is governed by an unknown transition map $Tr$, where $d_t = Tr(d_{t-1}, \theta_t)$. This generalizes stateless models where $d_t$ depends only on $\theta_t$.
• Risk Minimization: The study considers both Repeated Risk Minimization (RRM) on the population and Repeated Empirical Risk Minimization (RERM) on the sample.
• Conditions: The analysis relies on standard assumptions from the PP literature:
  • Strong Convexity: The loss function $\ell$ is $\gamma$-strongly convex in parameters $\theta$.
  • Wasserstein Sensitivity: The transition map $Tr$ is $(\varepsilon, p)$-jointly sensitive, meaning the Wasserstein distance between distributions shifts linearly with the shift in parameters and previous distributions.
  • Regularity: The loss is Lipschitz in data and continuously differentiable in parameters.

Theoretical Approach

The authors derive generalization bounds by analyzing the Wasserstein distance between distributions. They utilize:

• Convergence Bounds: Leveraging Fournier & Guillin (2015) to bound the distance between the initial sample and the true population.
• In-Sample Shift Bounds: Deriving bounds on how much the sample distribution shifts due to performative effects within the training loop.
• Covering Numbers: Using covering number entropy integrals (Kolmogorov & Tikhomirov, 1959) rather than Rademacher complexities to handle the dynamic nature of the distribution support.
• Duality in Wasserstein Space: Casting self-negating (population) and self-fulfilling (sample) predictions as min-max and min-min risk functionals, respectively, allowing the use of empirical process theory for locally distributionally robust and favorable learning.

3. Key Contributions and Results

The paper addresses four specific research questions (RQ1–RQ4) regarding generalization under different performativity scenarios:

RQ1: Generalization Under Sample Performativity

• Scenario: A model is retrained on a sample that reacts to predictions, but the population does not (or is not yet observed).
• Result (Theorem 3.7): The authors prove an upper bound on the excess risk $R(d_0, \hat{\theta}_T) - \inf R(d_0, \theta)$.
• Key Insight: The bound depends on the performative response rate ($m/n$, the fraction of sample units changing behavior). If $m = o(n)$, the bound vanishes as $n \to \infty$. However, if the sample reacts strongly, the bound grows, indicating a trade-off: manipulating the sample to improve outcomes degrades the ability to learn generalizable properties from that sample.

RQ2: Generalization Under Full Performativity

• Scenario: The model is trained on a sample, then deployed to a population that reacts to the predictions.
• Result (Theorem 3.10 & 3.13): Bounds are derived for the performative excess risk (error on the shifted population) and the generalization gap (difference between risk on the shifted population and the shifted sample).
• Structural Insight: In the worst case, the sample creates an "empirical echo chamber" by self-fulfilling predictions (deceptively confirming the model), while the population self-negates them (acting in the opposite way). This creates a fundamental gap between training performance and out-of-sample performance.
• Corollary 3.11: A practical insight suggests that the tightest generalization guarantees are obtained by using the initial fit ($\hat{\theta}_0$) on out-of-sample data while estimating the population shift from observed sample shifts, rather than retraining repeatedly on the distorted sample.

RQ3: Cumulative Performative Excess Risk

• Scenario: A model is trained on a sample, then rolled out to a population where it is retrained repeatedly on the shifting population.
• Result (Theorem 3.16): The paper provides bounds on the cumulative error over time. The analysis shows that the accumulated error is driven by the initial sample-to-population shift and the subsequent population dynamics.

RQ4: Inferential Gap

• Scenario: Comparing the empirical risk minimizer (trained on samples) to the theoretical risk minimizer (trained on the population).
• Status: The paper notes that this is partially answered by recent work (Li et al., 2025b) for asymptotic cases ($n \to \infty$) and identifies finite-sample analysis for stateful performativity as an open direction.

4. Empirical Illustration

The authors illustrate their bounds using real administrative data from the German Federal Employment Agency (1975–2017) regarding job seeker profiling.

• Setup: A logistic regression model predicts long-term unemployment risk. High-risk individuals are assigned to job training, which (if effective) reduces their unemployment risk, creating a performative effect.
• Findings: The bounds successfully quantify the generalization gap. As the fraction of job seekers assigned to training ($\xi$) increases, the performative term in the bound grows, demonstrating the change-learn trade-off: the more the model changes the population (by assigning more trainings), the less reliable its predictions become for the general population.

5. Significance and Claims

The paper claims to initiate the study of learnability under performativity. Its primary contributions are:

1. Theoretical Foundation: It establishes the first set of generalization bounds for performative predictions that account for finite samples and the distinction between sample and population reactions.
2. Fundamental Trade-off: It formalizes a trade-off between intervention (changing the world via predictions) and inference (learning from the resulting data). The more a model influences its environment, the harder it becomes to generalize from the resulting data.
3. Echo Chamber Mechanism: It identifies a specific mechanism where samples can "deceive" models through self-fulfilling prophecies while populations "negate" them, leading to inflated generalization gaps.
4. Practical Guidance: It offers a counter-intuitive but theoretically grounded recommendation: when facing performativity, relying on the initial model fit combined with estimates of the shift (derived from sample reactions) may yield tighter guarantees than continuous retraining on distorted data.

The authors maintain a modest tone, acknowledging that their bounds rely on specific regularity conditions (strong convexity, Lipschitz continuity) and that the transition map $Tr$ is treated as a black box. They position this work as a foundational step toward understanding the limitations of predictive systems in high-stakes domains like employment services, finance, and criminal justice, where predictions actively shape outcomes.