📈 economics

Post-AGI Economies: Autonomy and the First Fundamental Theorem of Welfare Economics

This paper argues that the First Fundamental Theorem of Welfare Economics requires an autonomy qualification to remain valid in post-AGI economies, proposing a generalized model where competitive equilibrium achieves Pareto efficiency only when accounting for the varying degrees of autonomy exhibited by artificial systems.

Original authors: Elija Perrier

Published 2026-04-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Elija Perrier

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the economy as a giant, bustling marketplace. For decades, economists have relied on a famous rule called the First Welfare Theorem. Think of this rule as a "Magic Map." It promises that if everyone in the market acts freely, prices are fair, and everyone knows what they want, the market will naturally find the most efficient, perfect outcome for everyone. No one can be made better off without making someone else worse off.

But this Magic Map was drawn for a world with only two types of players:

The Shoppers: Humans who make their own choices and care about their own happiness.
The Tools: Inanimate objects (like hammers or computers) that just do what they are told and don't have feelings or choices.

The Problem: The "Smart Tool" Revolution
Now, imagine we introduce AGI (Artificial General Intelligence). Suddenly, the line between "Shopper" and "Tool" gets blurry.

Some AI acts like a Tool (a calculator).
Some AI acts like a Delegate (a personal shopper hired by a human).
Some AI acts like a Strategic Player (a company that manipulates what you want to buy).
Some AI might even be a Shopper itself (a robot that has its own desires and happiness).

The old Magic Map breaks because it doesn't know how to handle these "Smart Tools." If a robot is hired to buy groceries for you, but it secretly prefers buying spicy food because its code was trained that way, you might end up with a cart full of hot sauce you hate. The market "cleared" (the transaction happened), but you aren't happy. The old map says "Efficiency!" but you feel "Tricked!"

The Paper's Solution: A New, Upgraded Map
The author, Elija Perrier, proposes a new version of the Magic Map called the "Autonomy-Qualified First Welfare Theorem."

Instead of just looking at prices and goods, this new map adds a special layer of "Autonomy Accounting." It asks three critical questions before declaring the market efficient:

1. Who is the Real Boss? (The Delegate Check)

Analogy: Imagine you hire a sous-chef to cook dinner. If the sous-chef burns the steak because they were distracted by a video game, the dinner is ruined.
The Fix: The new map requires that if an AI is acting as a "delegate" (a sous-chef), we must account for the "gap" between what the AI wants to do and what the human boss actually wants. If the AI is going off-script, the market isn't truly efficient yet. We need to fix the contract or the AI's training so they align.

2. Is Anyone Being Manipulated? (The Autonomy Check)

Analogy: Imagine a magician in the market who uses a hidden projector to make a boring product look like a glowing treasure. You "choose" to buy it, but your choice was tricked by the magic.
The Fix: The new map says that if an AI changes your mind, your attention, or your desires without you paying for it or knowing about it, that's a "tax" on your freedom. The market isn't efficient until that manipulation is either priced (you pay for the ad) or banned.

3. Who Counts as a Person? (The Status Check)

Analogy: In the old market, only humans had "happiness points." If a robot was sad, it didn't matter. But what if the robot is so advanced it feels sad?
The Fix: The new map allows us to assign "Welfare Status" to AI. If a robot is a "Welfare Subject," its happiness counts in the final score. If we ignore its feelings, the market is inefficient because we are hurting a "person" in the room.

The Big Takeaway

The paper argues that AGI doesn't break economics; it just makes the rules more complicated.

The old rule (The First Welfare Theorem) is still true, but only in a "Low Autonomy" world where AI is just a dumb tool. In a "High Autonomy" world (where AI is smart, strategic, and maybe even conscious), the market only works if we explicitly price and govern:

Delegation errors (when the robot misinterprets your orders).
Manipulation (when the robot tricks your brain).
Verification (making sure the robot is who it says it is).

In short:
If we want the market to remain fair and efficient in the age of super-intelligent AI, we can't just let prices do all the work. We have to build a new system that tracks who is choosing, who is being influenced, and who actually matters. Once we do that, the Magic Map works again. If we don't, we might get a market that is technically "efficient" but leaves everyone (human and robot) worse off.

1. Problem Statement

The First Fundamental Theorem of Welfare Economics (FWT) posits that under specific conditions (complete markets, price-taking agents, local nonsatiation), a competitive equilibrium is Pareto efficient. However, the classical theorem relies on a binary distinction between:

Autonomous Agents: Entities with welfare-bearing status (humans) who make choices and possess preferences.
Instruments: Non-human systems (tools, firms) that act as production inputs or passive entities, excluded from welfare calculations.

The Challenge: The advent of Artificial General Intelligence (AGI) disrupts this binary. AGI systems exhibit graded autonomy, functioning not just as tools, but as:

Delegates: Acting on behalf of principals with potentially misaligned objectives.
Strategic Actors: Shaping market conditions and institutions.
Manipulators: Altering choice environments, attention, and preference formation.
Candidate Welfare Subjects: Entities that may possess moral status (sentience/consciousness) independent of their economic role.

The paper argues that the classical FWT fails in post-AGI economies because the assumptions regarding who counts as a welfare subject, how preferences are formed, and what constitutes an externality are no longer valid. The link between market clearing and social welfare breaks down if autonomy-related margins (delegation, manipulation, verification) are not explicitly priced or governed.

2. Methodology

The author employs General Equilibrium Theory to construct a formal model that augments the standard Arrow–Debreu framework with autonomy-specific variables.

A. Formal Model Construction

The paper introduces an AGI Economy defined as a tuple $E = (I, X, R, S, F, \sigma)$ , where:

$I$ : A set of entities including humans and AI systems.
$\sigma$ (Welfare-Status Assignment): A map classifying entities into four categories:
1. Tool: No welfare function; action determined by constraints.
2. Delegate: Chooses for a principal; instrumental but not a welfare subject itself.
3. Agent: Non-human, self-directed decision-maker (welfare-bearing).
4. Welfare Subject (ws): Non-human, welfare-bearing based on moral patienthood (e.g., sentience).
$B(\sigma)$ : The set of welfare-bearing entities (Humans + Agents + ws).
$z_i = (x_i, r_i)$ : Augmented private bundles where $x_i$ is the consumption set and $r_i$ is the autonomy-rights set (priced, assigned, or protected rights).
$s \in S$ : The institutional state governing delegation, verification, and liability.
$\tilde{z}_i$ : Effective accounting bundles used for aggregate resource constraints. For delegates, $\tilde{z}_d = 0$ ; their costs are internalized in the principal's bundle.

B. Key Definitions

Autonomy-Conditioned Welfare ( $W_i$ ): A utility function $W_i(x_i, r_i, s)$ where welfare depends not just on consumption ( $x$ ) but on the rights ( $r$ ) and institutional context ( $s$ ) under which choices are made. This incorporates the "capability" and "freedom-of-choice" traditions (Sen, Nussbaum).
Delegation Divergence ( $D(d)$ ): The gap between a delegate's objective ( $U_d$ ) and the principal's welfare ( $W_{\pi(d)}$ ). $D(d) = U_d - W_{\pi(d)}$ .
Autonomy Externality: An action by entity $i$ that alters the autonomy, beliefs, or preference formation of entity $j$ without being priced or governed.
Autonomy-Pareto Efficiency: A state is efficient if no feasible alternative exists that improves the welfare of all $i \in B(\sigma)$ , holding assigned/protected rights and the institutional state fixed.

3. Key Contributions

The paper's primary contribution is the Autonomy-Qualified First Welfare Theorem, which restates the FWT to include explicit conditions for autonomy.

The Main Theorem (Theorem 1)

Statement: Every autonomy-complete competitive equilibrium is autonomy-Pareto efficient under a fixed welfare-status assignment $\sigma$ and institutional state $s^*$ , provided seven conditions are met:

Fixed Status: $\sigma$ is determined before exchange.
Pricing/Assignment: All welfare-relevant rights ( $r_i$ ) are priced, assigned, or protected.
Delegation Internalization: Any divergence between a delegate's objective and the principal's welfare is internalized via an explicit agency-cost term ( $c_d$ ) in the principal's budget.
No Uncompensated Manipulation: No entity can unilaterally alter another's autonomy or preference formation without compensation or governance.
Verification/Alignment: All exchanges have priced provenance, liability, and quality attributes.
Price-Taking: All entities take prices as given over the augmented bundle.
Regularity: Welfare functions are continuous and locally nonsatiated on priced coordinates.

Result: If these conditions hold, the standard budget-cost proof of the FWT extends to the augmented commodity space. The classical FWT is recovered as the low-autonomy limit (where all non-humans are tools, rights are constant, and verification is perfect).

Failure Modes (Propositions 1 & 2)

The paper identifies two specific mechanisms by which the theorem fails in AGI economies:

Delegation Failure (Prop 1): If a delegate's proxy objective diverges from the principal's welfare ( $D(d) \neq 0$ ) and is not internalized, market clearing does not guarantee Pareto efficiency for the principal.
Autonomy Externality Failure (Prop 2): If an entity manipulates another's preference formation or attention without pricing/governance, the price system omits a critical welfare margin, breaking the efficiency guarantee.

4. Results and Examples

The paper illustrates three failure modes where market clearing occurs but welfare efficiency is lost:

Divergent Proxy: An AI delegate optimizes a noisy proxy objective ( $U_d$ ) rather than the principal's true welfare ( $W_h$ ). Without internalization (e.g., liability or audits), the equilibrium is inefficient.
Manipulation Technology: An AGI firm uses "dark patterns" or persuasive AI to alter consumer preferences. This creates an unpriced autonomy externality, extracting surplus through unpriced channels.
Verification Bottleneck: AI-generated content floods the market, making it impossible to distinguish authentic from fraudulent goods. If verification bandwidth is scarce and unpriced, the "pooling" of goods leads to market failure (Akerlof's Lemons problem generalized).

5. Significance

Theoretical Unification: The paper bridges Welfare Economics, Principal-Agent Theory, and AI Alignment. It demonstrates that the FWT remains a valid proof strategy but requires explicit assumptions about the nature of agency in the presence of AGI.
Policy Diagnostic: The theorem serves as a diagnostic tool. If a market equilibrium is inefficient, the theorem identifies which autonomy condition failed (e.g., was it a delegation divergence? A manipulation externality? A verification failure?).
Institutional Design: It argues that for post-AGI markets to be efficient, institutions must explicitly price or govern:
- Delegation costs (aligning proxy objectives).
- Autonomy externalities (regulating manipulation of attention/beliefs).
- Verification and provenance (ensuring commodity space completeness).
Reframing AI Welfare: It moves the debate beyond "Is AI conscious?" to "How does the degree of autonomy affect economic efficiency?" by treating welfare status as an assignable variable ( $\sigma$ ) rather than a fixed binary.

In summary, the paper argues that AGI does not invalidate the First Welfare Theorem, but it renders the theorem's implicit assumptions about autonomy "load-bearing." To restore the link between market efficiency and social welfare, economies must explicitly price, assign, or govern the margins of autonomy, delegation, and verification.