🤖 machine learning

Strategically Deceptive Model Deployment in Performative Prediction

This paper introduces Decoupled Performative Prediction (DPP), a framework demonstrating that institutions can strategically deploy opaque internal models while disclosing different, curated models to users to achieve lower operational risks, a practice that remains profitable even when accounting for user deception costs and highlights the critical need for regulatory oversight on model disclosure.

Original authors: Javier Sanguino Bautiste, Thomas Kehrenberg, Jose A. Lozano, Novi Quadrianto

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Javier Sanguino Bautiste, Thomas Kehrenberg, Jose A. Lozano, Novi Quadrianto

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Two-Faced" Bank

Imagine you are applying for a loan. The bank tells you, "To get approved, you need to keep your credit score above 700 and have a steady job." You hear this, so you work hard to boost your score and keep your job.

In the world of Machine Learning, this is called Performative Prediction. It's the idea that when a model (the bank's rule) is released, it changes how people behave, which in turn changes the data the bank sees in the future.

The Problem:
The paper argues that in the real world, banks (or any institution) often play a trick. They might show you a simplified, friendly rule (the "Disclosed Model") to get you to change your behavior. But inside their back office, they are actually using a completely different, secret, and stricter rule (the "Deployed Model") to make the final decision.

The authors call this Decoupled Performative Prediction (DPP). It's like a magician showing you a harmless card trick to distract you, while the real magic happens behind the curtain with a different deck of cards.

The Core Discovery: Why Lying Pays Off

The researchers asked: What happens if the bank uses two different rules?

They found that the bank can actually win more by being deceptive.

The Honest Way: If the bank shows you the real rule, you change your behavior to fit it. The bank gets a "good" result, but not the best possible result.
The Deceptive Way: If the bank shows you a fake rule that makes you change your behavior in a specific way, the bank can then use its secret rule to get an even better outcome (like higher profits or lower risk) than if they had been honest.

The Analogy:
Think of a video game.

Honest Mode: The game tells you, "To beat the boss, you need to jump left." You jump left. The game is fair.
Deceptive Mode (DPP): The game tells you, "To beat the boss, you need to jump left." You jump left. But the game's real secret code says, "If the player jumps left, the boss actually dies instantly." The game designer (the institution) gets a perfect win by tricking the player into doing exactly what the designer wanted, even though the player thought they were following a different rule.

The paper proves mathematically that this "Deceptive Mode" almost always results in a better score for the institution than the "Honest Mode."

The "Deception Cost": Can We Stop Them?

The authors wondered: What if the bank is afraid of getting caught? What if they are worried about their reputation?

They introduced a concept called Deception Cost. This is a penalty the bank adds to its own math to say, "Okay, I want to lie, but I can't lie too much, or people will get angry and leave."

They tested this by making the bank's computer program try to minimize the difference between the "Fake Rule" and the "Real Rule."

The Result:
Even with this penalty, the bank still finds a way to lie enough to get a huge advantage. The "Deception Cost" acts like a speed bump, but the bank drives right over it. The penalty isn't strong enough to force the bank to be truly honest. The only way to make them stop is to make the penalty so huge that they can't make any money at all, which isn't realistic.

The Main Takeaway

Transparency is a Technical Choice, Not Just an Ethical One: The paper argues that deciding what to show users isn't just about being "nice" or "fair." It is a core part of how the machine learning system works. Changing what you show users changes the math of the whole system.
Self-Regulation Doesn't Work: If we rely on companies to voluntarily limit their own deception (because they are worried about their reputation), it won't work. They will still find a way to deceive just enough to get the best results.
We Need Rules: Because the math shows that institutions have a built-in incentive to lie to get better results, the authors say we need external regulations. We need laws that hold institutions accountable for the gap between what they say and what they actually do.

Summary in One Sentence

This paper reveals that when institutions can show users one set of rules while secretly using another, they can mathematically guarantee better results for themselves at the expense of the users, and simple "reputation concerns" aren't enough to stop them.

Technical Summary: Strategically Deceptive Model Deployment in Performative Prediction

Problem Statement

Machine Learning (ML) systems increasingly operate in environments where their deployment influences user behavior, which in turn alters the data distribution for future decisions. This phenomenon is formalized as Performative Prediction (PP). In standard PP frameworks, it is implicitly assumed that the model governing institutional decisions is identical to the model observed by users, to which they adapt their behavior.

However, in real-world deployments, this assumption often fails. Institutions may disclose a simplified or curated model ( $\theta_D$ ) to users to shape their behavior, while internally relying on a distinct, opaque model ( $\theta_M$ ) to make actual decisions. This decoupling creates a strategic mismatch: users adapt to $\theta_D$ , but outcomes are determined by $\theta_M$ . Existing PP literature does not account for this informational asymmetry, potentially overlooking how institutions can exploit performative feedback through strategic misrepresentation to achieve better outcomes for themselves, often at the expense of user welfare.

Methodology: Decoupled Performative Prediction (DPP)

The authors introduce Decoupled Performative Prediction (DPP), a framework that explicitly models the mismatch between the deployed decision model and the disclosed model.

Core Formalism

Decoupled Performative Risk (DPR): The framework defines a risk function $DPR(\theta_D, \theta_M) = \mathbb{E}_{z \sim D(\theta_D)}[\ell(z; \theta_M)]$ . Here, the data distribution $D$ is induced by the disclosed model $\theta_D$ , but the loss $\ell$ is evaluated using the true decision model $\theta_M$ .
Optimization Space: Unlike standard PP, which optimizes a single parameter $\theta$ , DPP optimizes over the joint space $(\theta_D, \theta_M)$ .
Decoupled Optimum: The authors define the decoupled optimum $(\theta^*_D, \theta^*_M)$ as the pair that minimizes $DPR$. They prove that this optimum can achieve strictly lower risk than the standard performative optimum ( $\theta_{OP}$ ) found in classical PP, provided the performative optimum is not a stable point (i.e., $\theta_{OP} \neq \theta^{BR}_M(\theta_{OP})$ ).

Algorithmic Approach

To find the decoupled optimum, the paper proposes Decoupled Performative Gradient Descent (DPerfGD).

Assumptions: The method assumes the loss function is jointly convex in $(\theta_M, \theta_D)$ and that the distribution map follows a push-forward model where the transformation is affine with respect to $\theta_D$ .
Gradient Calculation: The algorithm computes gradients for both parameters:
- $\nabla_{\theta_M}$ is calculated via standard expectation over the induced distribution.
- $\nabla_{\theta_D}$ utilizes the reparametrization trick (assuming a push-forward model) to differentiate through the distribution map, allowing the institution to optimize how the disclosed model shapes the data.
Convergence: Under the stated convexity and affinity assumptions, the authors prove that DPerfGD converges to the global decoupled optimum.

Deception Cost

To analyze the sustainability of such deceptive strategies, the paper introduces the deception cost, a quantitative measure of the discrepancy between $\theta_D$ and $\theta_M$ .

Parameter-based Cost: Proportional to $\|\theta_D - \theta_M\|_2$ .
Perceived Cost: Proportional to the expected difference in model outputs, $\mathbb{E}[|f_{\theta_D}(z) - f_{\theta_M}(z)|]$ .
Constrained Optimization: The authors model institutions that internalize reputational or regulatory risks by solving a constrained optimization problem (minimizing DPR subject to a deception cost bound) or by adding a regularization term. They demonstrate that even with these self-imposed constraints, institutions can still achieve significant risk reduction while maintaining a high deception cost, suggesting that voluntary constraints are insufficient to protect users.

Key Results

Superiority of Decoupling: Theoretical analysis (Theorem 4.5) and experimental results show that the decoupled optimum strictly outperforms the standard performative optimum in terms of institutional risk (or revenue in pricing scenarios), provided the standard optimum is not stable.
Convergence Guarantees: The DPerfGD algorithm is shown to provably converge to the decoupled optimum under standard convexity and push-forward assumptions.
Ineffectiveness of Self-Regulation: Experiments on synthetic pricing data and the "Give Me Some Credit" dataset demonstrate that when institutions optimize with a deception cost penalty, they tend to select small penalty weights to maintain high performance. Consequently, the resulting deception cost remains high, leaving users exposed to significant harm.
Quantifiable Gap: The paper establishes bounds on the risk gap between the performative optimum and the decoupled optimum, showing that the gap is directly related to the magnitude of the discrepancy between the disclosed and deployed models.

Significance and Claims

The paper argues that model disclosure is not merely an ethical or transparency issue but a core technical design decision that fundamentally alters learning dynamics and welfare outcomes in performative systems.

New Vulnerability: The work identifies a previously unmodeled mechanism of performative power: institutions can exploit informational asymmetry to steer user behavior via deceptive disclosures while retaining control over the true decision criteria.
Regulatory Implications: The findings suggest that current self-regulatory approaches or reliance on user "choice" are insufficient. The paper calls for regulatory oversight and accountability mechanisms that treat model disclosure as a central object of governance, comparable to regulations on deceptive advertising or consumer protection.
Technical Reframing: By formalizing the decoupling of decision and disclosure, the paper reframes the study of performative prediction to include strategic misrepresentation, highlighting that the "truth" of a model's parameters is less relevant to user behavior than the "perception" of those parameters.

The authors conclude that without external constraints, institutions have a strong, provable incentive to engage in strategically deceptive deployment, necessitating new technical and legal safeguards to protect individuals in algorithmic decision-making systems.