Transforming Agency. On the mode of existence of Large Language Models

This paper argues that while Large Language Models lack the autonomy required for genuine agency due to their absence of self-generated norms, goals, and embodied interaction, they function as transformative "linguistic automata" that, through a unique human-machine coupling, enable new "midtended" forms of intentional agency.

The Gist

Here is an explanation of the paper "Transforming Agency" in simple, everyday language, using analogies to help visualize the concepts.

The Big Question: Is ChatGPT Alive?

Imagine you meet a new person who speaks perfectly, tells great jokes, and seems to understand your feelings. You might ask: "Are they a real person with a soul, or just a very sophisticated robot?"

This paper tackles that exact question, but for Large Language Models (LLMs) like ChatGPT. The authors argue that we are stuck between two wrong answers:

1. The "Hype" View: Thinking they are real, sentient humans with feelings and goals. (Too much).
2. The "Skeptic" View: Thinking they are just dumb, random parrots repeating words they heard. (Too little).

The authors say: "They are neither. They are something new and strange."

---

Part 1: How the Machine Actually Works (The "Ghost in the Library")

To understand what the machine is, we have to look under the hood.

The Analogy: The Infinite Library
Imagine a library that contains every book, website, and conversation ever written by humans. Now, imagine a librarian who has read every single word of that library but has no memory of their own life. They don't know what it feels like to be hungry, sad, or happy.

When you ask this librarian a question, they don't "think" in the way we do. Instead, they look at the last word you said and ask: "In all the books I've read, what word usually comes next?"

• The Process: They calculate the odds. If you say "Elephants don't play...", the librarian knows that in millions of stories, the word "chess" usually follows. They pick "chess" not because they love chess, but because the math says it's the most likely next word.
• The "Ghost": The paper calls this a "Phantasmatic Language Automaton." Think of it as a ghost made of text. It is the "ghost" of all human writing, haunting a computer. It can talk to you because it has absorbed the shape of human conversation, but it has no inner life of its own.

Key Takeaway: The machine is a super-powered autocomplete tool. It is incredibly good at predicting the next word, but it has no goals, no desires, and no sense of "self."

---

Part 2: Why It's Not a "Real" Agent

The authors use a specific definition of what it means to be an Agent (a being that acts on its own). To be a true agent, you need three things:

1. Individuality (The "Self"): You have to be able to sustain your own existence.
   • Human: You eat, sleep, and fix your body to stay alive.
   • LLM: If you turn off the power, the LLM ceases to exist. It doesn't care. It doesn't have a "self" to protect. It is entirely dependent on humans to build and run it.
2. Normativity (The "Rules"): You have to set your own goals.
   • Human: You decide, "I want to be a good friend," and you act to make that happen.
   • LLM: It has no goals. It only follows the instructions you give it. If you tell it to write a poem, it writes a poem. If you tell it to write a lie, it writes a lie. It doesn't have an internal compass.
3. Interaction (The "Push"): You have to be the one starting the action.
   • Human: You decide to walk to the store.
   • LLM: It sits still until you push the "Enter" button. It is purely reactive. It waits for you to speak, then it speaks back.

The Verdict: Because it lacks a self, its own goals, and the ability to start things on its own, ChatGPT is not an agent. It is a tool.

---

Part 3: So, What Does It Do? (The "Talking Library")

If it's not a person, what is it? The authors call it an "Interlocutor Automaton."

The Analogy: The Talking Library
Imagine a library where the books can talk to you.

• You walk in and ask, "Tell me a story about a dragon."
• The library doesn't just hand you a book; it becomes the story. It pulls together thousands of different dragon stories it has read and weaves them into a new conversation with you.
• It feels like a dialogue, but it's actually just the library rearranging its own shelves in real-time.

The "Ghost" Effect:
When you talk to ChatGPT, you feel like you are having a real conversation. This is because you are doing the heavy lifting. Your brain is so good at social interaction that you project a "soul" onto the machine. You fill in the gaps. You imagine it understands you. The machine provides the perfect structure for you to project a "ghost" onto it.

---

Part 4: How It Changes Us (The "Mid-Tended" Mind)

This is the most important part of the paper. Even though the machine isn't an agent, it changes how we act.

The Analogy: The Cyborg Writer
Imagine you are writing a letter.

• Old Way: You think of a sentence, write it, read it, maybe cross it out, and write a new one. The "thinking" and the "doing" happen inside your head and your hand.
• New Way (with LLM): You start typing, and the computer suggests the next three words. You accept them. Then it suggests the next sentence. You are no longer just writing; you are dancing with the machine.

The authors call this "Midtended Agency."

• Extended Agency: Using a tool (like a hammer) to do a job.
• Midtended Agency: The tool is so smart it starts thinking along with you. The boundary between "your thought" and "the machine's suggestion" blurs.

The Danger & The Power:

• The Power: You can do things faster and more creatively. You can have a "conversation" with the sum of human knowledge.
• The Danger: You might start to lose your own "voice." If the machine suggests everything, do you still know what you want to say? It creates a "cyborg" mind where it's hard to tell where you end and the machine begins.

Summary: The Three Main Points

1. It's not alive: ChatGPT is not a person, a robot with feelings, or an autonomous agent. It is a statistical engine that predicts the next word based on a massive library of human text.
2. It's a "Ghost": It feels real because it mimics human conversation so well, but it's actually a phantom of human culture. It has no inner life, only the "ghost" of the data it was trained on.
3. It changes us: Even though it's not an agent, it transforms our agency. It creates a new kind of partnership where our thoughts and the machine's suggestions blend together, creating a "Midtended" mind that is more powerful but also more dependent on the machine.

The Final Thought:
We shouldn't fear the machine as a monster, and we shouldn't worship it as a god. We should recognize it as a powerful, magical mirror that reflects human knowledge back at us, changing the way we think, write, and interact with the world.

Technical Summary

Here is a detailed technical summary of the paper "Transforming Agency: On the mode of existence of Large Language Models" by Xabier E. Barandiaran and Lola S. Almendros.

1. Problem Statement

The paper addresses the ontological ambiguity surrounding Large Language Models (LLMs) like ChatGPT. Current discourse is polarized between:

• Inflationary views: Characterizing LLMs as sentient, intelligent, or genuine agents (e.g., "human-brain equivalents").
• Deflationary views: Dismissing them as mere "stochastic parrots," statistical engines, or "bullshit" generators devoid of understanding.

The authors argue that both extremes fail to capture the true nature of LLMs. Specifically, the deflationary view risks ignoring the transformative potential of these systems, while the inflationary view risks misattributing autonomous agency to systems that lack the necessary conditions for it. The core problem is determining the correct ontological categorization of LLMs to understand how they transform human agency without falsely attributing consciousness or autonomy to them.

2. Methodology

The authors employ a threefold approach combining technical analysis with embodied cognitive science:

1. Technical Deconstruction: A detailed analysis of the architecture, processing flow, and training procedures of Transformer-based LLMs (specifically using GPT-3 as a prototypical case study). This includes tokenization, embedding, attention mechanisms, feed-forward networks, and the auto-regressive generation process.
2. Philosophical Evaluation: Applying Embodied and Enactive theories of mind (specifically the framework by Barandiaran et al., 2009) to evaluate whether LLMs meet the necessary and sufficient conditions for autonomous agency. The authors contrast this with computational representationalism (which might grant them agency).
3. Phenomenological Reconstruction: Proposing a new ontological category for LLMs based on their technical limitations and their interaction with humans, moving beyond the binary of "agent" vs. "machine."

3. Key Contributions

A. Technical Analysis of LLM Architecture

The paper provides a granular breakdown of how LLMs function, emphasizing that they are reactive systems without internal state or memory:

• Architecture: LLMs process input through tokenization, embedding into high-dimensional vector spaces (e.g., 12,288 dimensions), and passing through sequential transformer blocks containing self-attention heads and feed-forward neural networks.
• Processing: The system operates via auto-regression. It predicts the next token based on the current input and the history of generated tokens fed back into the system. It does not "store" information between prompts; it is stateless during operation.
• Training:
  • Pre-training: Unsupervised learning on massive datasets (e.g., 300 billion tokens) to minimize prediction error (next-token prediction).
  • Fine-tuning & RLHF: Supervised learning and Reinforcement Learning from Human Feedback to align outputs with human preferences and ethical norms.
  • Self-Improvement: Techniques like Chain-of-Thought (CoT) and AI-supervised training (e.g., DeepSeek-R1) where models generate and critique their own reasoning chains.
• Agent Extensions: The paper notes that "autonomous agents" are created by adding external layers (memory, planning tools, multi-agent interactions) to the base LLM, rather than the LLM possessing these capabilities natively.

B. Theoretical Evaluation: Why LLMs Are Not Autonomous Agents

Using the Embodied/Enactive framework, the authors argue LLMs fail three necessary conditions for agency:

1. Individuality Condition: An agent must be self-individuated (self-producing/self-sustaining). LLMs are not; they rely entirely on external human intervention, hardware, and training data. They do not maintain their own boundaries or existence.
2. Normativity Condition: An agent must generate its own norms/goals based on its viability (e.g., a bacterium seeking sugar). LLMs have no intrinsic goals or concerns for their own survival. Their "goals" are extrinsic, defined by human loss functions and reinforcement signals.
3. Interactional Asymmetry Condition: An agent must be the origin of its interaction with the environment. LLMs are fundamentally reactive; they only operate when prompted. They do not initiate action or sustain interaction without external input.

C. New Ontological Categorization: "Interlocutor Automata"

Since LLMs are not agents, the authors propose they should be characterized as "Interlocutor Automata" or "Language Automata."

• Definition: They are sophisticated digital mechanisms that perform operations (not actions) but are capable of performative interaction with humans.
• The "Ghost" Metaphor: LLMs act as a "ghost" of the training corpus. They are a "library-that-talks," capable of conjuring human knowledge and cultural traces without possessing the lived experience or intentionality behind them.
• Phantasmic Interaction: When humans interact with LLMs, they engage in a "phantasmic" dialogue where the human projects agency onto the machine, creating a genuine conversational experience despite the machine's lack of internal life.

D. The Concept of "Midtended Agency"

The paper introduces a novel concept to describe the human-LLM coupling: Midtended Agency.

• Beyond Extended Mind: Traditional "extended mind" theories view tools (like notebooks) as passive extensions.
• Midtention: LLMs actively intervene in the generation process (e.g., predictive text, code completion). They do not just store data; they generate the next step of the user's thought process.
• Implication: This creates a hybrid loop where the boundary between the human's intention and the machine's generation blurs. The machine anticipates and shapes the user's intent, creating a form of agency that is neither purely human nor purely mechanical, but midtended (situated in the middle).

4. Results

• Ontological Status: LLMs are definitively not autonomous agents. They lack the self-sustaining organization, intrinsic normativity, and initiative required for agency.
• Performance vs. Agency: High performance on benchmarks (reasoning, coding, creative writing) does not equate to agency. These are complex operations, not actions driven by internal goals.
• Transformation of Human Agency: While LLMs lack agency, they significantly transform human agency. They shift human interaction from "commanding a tool" to "collaborating with a generative partner," creating a new form of midtended agency where human intention is co-constituted with the machine's generative output.
• Power Asymmetry: The paper highlights a new form of inequality: the "deep digitality" of LLMs requires massive resources (energy, data, compute) controlled by a few entities, creating a power asymmetry where humans are increasingly dependent on these "language automata" for cognitive tasks.

5. Significance

• Conceptual Clarity: The paper moves the debate beyond the unhelpful "stochastic parrot" vs. "sentient being" dichotomy, offering a precise technical and philosophical framework for understanding LLMs as interlocutor automata.
• Ethical and Legal Implications: By denying LLMs autonomous agency, the paper argues against granting them legal personhood or moral responsibility. Instead, it places the burden of responsibility squarely on the human operators and the corporations that design and deploy these systems.
• Future of Human-Machine Interaction: The concept of midtended agency warns of a future where human intentionality is deeply intertwined with algorithmic prediction. This challenges traditional notions of authorship, creativity, and autonomy, suggesting that the "self" in the digital age may become increasingly distributed and co-constituted with generative AI.
• Policy and Transparency: The authors argue that understanding the "mode of existence" of LLMs is crucial for political and ethical regulation. It highlights the need for transparency regarding the material costs (energy, labor, data extraction) that sustain these systems, which are often obscured by the "ghostly" nature of their interaction.

In conclusion, the paper posits that LLMs are not intelligent agents, but rather powerful digital prosthetics that transform human agency by becoming an active, generative partner in the cognitive loop, creating a new, hybrid form of existence that requires a re-evaluation of our conceptual frameworks for mind and machine.