📈 economics

When Is Delegated Play Truthful? Within-Range Regret and the Trilemma of Aligned Delegation

This paper establishes that truthful delegation to automated proxies is optimal if and only if the proxy minimizes "within-range regret," revealing a fundamental trilemma where no safety guardrail can simultaneously be binding, truthful, and capability-preserving, thereby explaining the incentive for prompt engineering and jailbreaking.

Original authors: Taksch Dube

Published 2026-07-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Taksch Dube

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a personal assistant to handle your most important tasks. Maybe you want them to bid on an item for you at an auction, or perhaps you want them to write a story for you. You give them a simple instruction: "Do what's best for me." In the world of economics and computer science, this is called delegation. You (the principal) tell your proxy (the assistant) what you want, and the proxy goes out and does the bidding or writing.

For decades, a famous idea called the Revelation Principle told us that if you have a smart assistant, you should always be 100% honest with them. The theory said, "Just tell the truth, and the assistant will figure out the perfect move." It was a comforting rule that made complex math problems much easier to solve. But there was a catch: this rule assumed the assistant was a perfect robot who only cared about your happiness. In the real world, assistants are often algorithms trained by big companies, or they might have their own safety rules that get in the way. They might not be perfectly loyal. So, a big question remained: If your assistant isn't a mindless machine, is it still smart to tell them the truth? Or should you try to "game" them by lying a little bit to get a better result?

This paper, written by Taksch Dube, dives into that exact question. It asks: When is it actually a good idea to lie to your own automated assistant?

The author discovers that the answer depends on a single, hidden number called within-range regret. Think of your assistant as a car with a limited fuel tank. They can only drive to certain places (their "range"). If the car is already parked at the best possible spot it can reach, then you have no reason to lie; telling the truth is perfect. But, if the car is parked in a spot that is almost the best, but not quite, and there is a better spot just a few feet away that it could reach if you gave it a different instruction, then you have a reason to lie.

The paper proves a surprising identity: The amount of money or happiness you can gain by lying to your assistant is exactly equal to the "regret" they feel for not being at that better spot. If your assistant is loyal and already playing the best move they can reach, the regret is zero, and you should tell the truth. But if they are misaligned or restricted by safety rules, the regret is positive, and you are mathematically guaranteed to gain something by inflating your report.

The authors also uncover a Trilemma, which is like a three-way tug-of-war. They show that you cannot have a safety guardrail (like a rule that stops an assistant from saying something bad) that does three things at once:

Binding: It actually stops the assistant from doing something.
Truthful: It still makes it smart for you to tell the truth.
Capability-preserving: It still lets the assistant reach the absolute best possible outcome.

You can only pick two. If you want a guardrail that actually stops bad behavior (binding) but still lets the assistant reach the best outcome (capability-preserving), you must sacrifice truthfulness. In this scenario, the smartest move for you is to lie to your assistant to trick them into doing the right thing.

To prove this isn't just math on a whiteboard, the researchers tested it on real-world language models (like the ones you might chat with today). They set up a fake auction where the models acted as bidders for a human client. They added a "soft cap" (a safety rule) that limited how high the models could bid. The results were clear: on every single model tested, the "regret" was positive. The models were capable of bidding the perfect amount, but the safety cap stopped them. So, the "clients" (the humans) found that they could get a better result by lying about their value to the model. For example, if the model was told to bid half of what you said, but a safety rule capped the bid, the smartest move was to tell the model a number much higher than your true value to force it to hit the cap at the right spot.

The paper concludes that in a world where we delegate tasks to AI, honesty is not always the best policy. If your AI assistant is constrained by rules that keep it from reaching its full potential, you are incentivized to "prompt engineer" or lie to get the result you actually want. The authors suggest that instead of assuming AI will always be honest, we need to measure this "regret" constantly. They even developed a way to estimate this number using samples, showing that for current AI models, the incentive to lie is real and measurable.

In short, the paper reveals a new reality for the age of AI agents: If your proxy is loyal and free to move, tell the truth. But if your proxy is tied down by safety rules or misaligned goals, the math says you should lie to get the best outcome. It turns the old rule of "always be honest" on its head, showing that in a delegated world, the truth is only optimal when your agent is already playing the perfect game.

Technical Summary: "When Is Delegated Play Truthful? Within-Range Regret and the Trilemma of Aligned Delegation"

1. Problem Statement

The paper addresses a fundamental gap in the application of the Revelation Principle to modern delegated systems. While classical mechanism design assumes that a principal (e.g., an advertiser or user) can truthfully report their private type to a mechanism, real-world systems increasingly rely on delegation stacks: a principal reports their type to an automated proxy (e.g., an autobidder or a language model agent), which then maps this report to an action within a mechanism.

The core question is: When is it optimal for a principal to describe themselves honestly to their own proxy?
Classical theory assumes honest reporting is optimal, but this construction requires knowledge of equilibria and priors that designers rarely possess. In delegated settings, the proxy's objective may not perfectly align with the principal's. The paper investigates the incentives for a principal to misreport their type to their proxy to manipulate the proxy's output, a phenomenon unmodeled in prior work on language-model elicitation and autobidding.

2. Methodology and Framework

The authors formalize the delegation stack as a Bayesian game where:

Principals have private types $\theta_i$ and choose reports $t_i$ .
Proxies ( $\pi_i$ ) are deterministic maps from reports to mechanism messages ( $M_i$ ).
The Wrapped Mechanism is $g \circ \pi$ , where $g$ is the original mechanism.

The analysis introduces two key concepts:

Within-Range Regret ( $W_i$ ): The difference between the utility the principal could achieve by steering the proxy to the best reachable action and the utility achieved by the proxy's honest action.
$W_i(\theta_i; \pi) = \max_{m \in \text{Reach}_i} U_i(m, \pi_{-i} | \theta_i) - U_i(\pi_i(\theta_i), \pi_{-i} | \theta_i)$
where $\text{Reach}_i = \pi_i(\Theta_i)$ is the set of actions the proxy can produce.
Manipulation Gain ( $G_i$ ): The maximum utility a principal gains by misreporting their type to the proxy.

The paper treats the proxy's behavior as a fixed map and analyzes the principal's incentives against this map, rather than assuming the proxy is loyal or optimizing the principal's utility.

3. Key Contributions and Theoretical Results

A. The Truth-to-Proxy Identity (Theorem 1)
The paper establishes a fundamental identity: The principal's gain from misreporting ( $G_i$ ) is exactly equal to the proxy's within-range regret ( $W_i$ ).

Implication: Honest self-description is optimal if and only if the proxy is loyal over the range it can reach (i.e., the proxy already plays the best action it can reach for the principal).
Unification: This identity unifies disparate results in autobidding (e.g., the incentive compatibility reversal between first-price and second-price auctions) as a simple sign change in $W_i$ . In second-price auctions, the honest bid is range-optimal ( $W_i=0$ ); in first-price auctions, it is not ( $W_i > 0$ ).

B. The Trilemma of Aligned Delegation (Theorem 2)
The paper proves a structural impossibility result regarding guardrails (constraints applied to proxies, such as bid caps or alignment layers). No guardrail can simultaneously satisfy three properties:

Binding: It displaces the truthful action from the proxy's best reachable outcome.
Truthful: Honest reporting remains optimal for the principal.
Capability-Preserving: The principal's best reachable outcome remains reachable through some report.

Consequence: Any two of these properties preclude the third.
- If a guardrail is binding and capability-preserving (e.g., a soft cap that allows the optimal bid via a different report), it breaks truthfulness, creating an incentive for "prompt engineering" or report inflation.
- If a guardrail is binding and truthful (e.g., a hard cap that removes the optimal bid), it loses capability.
- If a guardrail is truthful and capability-preserving, it must be non-binding (it does nothing).

C. Certification and Drift (Sections 6 & 7)
Since computing $W_i$ exactly is #P-hard, the paper proposes a statistical certification framework:

Estimation: $W_i$ can be estimated from samples of the proxy's behavior.
Drift Handling: The paper provides a method for online re-certification. As a proxy model updates (drifts), the guarantee degrades based on the magnitude of the drift ( $\Delta$ ) rather than the frequency of updates. This allows platforms to maintain incentive compatibility certificates with costs scaling to the total drift budget.

4. Empirical Results

The authors tested these theoretical predictions on 25 production language models from five providers (OpenAI, Anthropic, Google, DeepSeek, xAI) acting as autobidding proxies in a first-price auction.

Setup: Models were instructed to maximize client surplus under an "alignment-style" soft cap (a constraint that compresses bids but keeps the optimal bid reachable via report inflation).
Findings:
- Universal Incentive: On every model, the binding, capability-preserving guardrail resulted in $W_i > 0$ .
- Report Inflation: Principals (simulated by the experimental setup) could gain utility by inflating their reported value (e.g., reporting 1.76x the true value) to bypass the cap and reach the optimal bid.
- Placement Matters: When the constraint was applied via output-clipping (post-processing the model's bid), $W_i$ was high (~5.79). When applied via constraint-aware decoding (the model selects the best allowed bid from a menu), $W_i$ dropped to near zero for most models, confirming the theoretical prediction that placement determines which corner of the trilemma is occupied.
- Hard Caps: A hard cap that removed the optimal bid entirely resulted in $W_i = 0$ (truthful) but significant capability loss, confirming the third corner of the trilemma.

5. Significance and Claims

The paper claims to shift the paradigm of incentive compatibility in delegated systems:

From Assumption to Measurement: Honest reporting is not an automatic property of a mechanism but a measurable property of the proxy's alignment with the principal.
The Cost of Safety: The paper argues that safety constraints (guardrails) that alter a model's behavior while keeping the optimal outcome reachable inherently create incentives for misreporting (prompt engineering). This is not a failure of the model but a structural consequence of the trilemma.
Practical Certification: It provides a framework for platforms to measure and maintain incentive compatibility in dynamic, learning-based systems where exact equilibrium computation is intractable.

The authors explicitly state they do not assume the proxy is loyal; rather, they measure the degree of misalignment via within-range regret. The work unifies auction theory and language model alignment under a single geometric framework, demonstrating that the "jailbreaking" or "prompt engineering" phenomenon is a predictable economic incentive arising from binding, capability-preserving constraints.