💻 computer science

Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering

This paper presents a unified systems-level framework arguing that the advancement of practical LLM agents increasingly depends on externalizing cognitive burdens into memory, skills, and protocols coordinated by harness engineering, rather than solely on improving model weights.

Original authors: Chenyu Zhou, Huacan Chai, Wenteng Chen, Zihan Guo, Rong Shan, Yuanyi Song, Tianyi Xu, Yingxuan Yang, Aofan Yu, Weiming Zhang, Congming Zheng, Jiachen Zhu, Zeyu Zheng, Zhuosheng Zhang, Xingyu Lou, Chan

Published 2026-04-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Chenyu Zhou, Huacan Chai, Wenteng Chen, Zihan Guo, Rong Shan, Yuanyi Song, Tianyi Xu, Yingxuan Yang, Aofan Yu, Weiming Zhang, Congming Zheng, Jiachen Zhu, Zeyu Zheng, Zhuosheng Zhang, Xingyu Lou, Changwang Zhang, Zhihui Fu, Jun Wang, Weiwen Liu, Jianghao Lin, Weinan Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a brilliant, incredibly fast, but slightly forgetful and impulsive genius (the LLM) how to run a complex business.

If you just sit the genius down and say, "Go do this," they might get the general idea but will likely forget details halfway through, make up rules as they go, or get confused when they need to use a specific tool.

This paper argues that the secret to making AI agents (like the genius) actually useful isn't just making the genius smarter. It's about building a super-organized office around them. The authors call this process "Externalization."

Here is the simple breakdown of how this works, using a creative analogy: The "Brilliant Intern" vs. The "Perfect Office."

1. The Core Idea: Don't Rely on Memory, Rely on the Office

In the past, we tried to make the AI smarter by feeding it more data during training (changing its "weights"). It's like trying to stuff a human brain with every fact in the universe. It's expensive, slow to update, and the brain still forgets things.

The New Approach: Instead of stuffing the brain, we build an office where the brain can look things up, follow checklists, and talk to others using a standard language. We take the hard stuff out of the brain and put it into the environment.

2. The Four Pillars of the "Perfect Office"

The paper identifies four main things we move out of the AI's brain and into the office:

A. Memory: The "Filing Cabinet" (Externalized State)

The Problem: The AI is like a genius who has a 5-minute memory span. If you ask it to work on a project for three days, it forgets what it did on Day 1 by Day 3.
The Solution: We give it a Filing Cabinet.
- Instead of trying to remember everything, the AI writes notes to the cabinet.
- When it needs to know what happened yesterday, it doesn't "recall" from its brain; it retrieves the file.
- Analogy: It's the difference between trying to memorize a phone book (hard, error-prone) vs. just looking up the number in your contacts (easy, reliable).

B. Skills: The "Recipe Book" (Externalized Expertise)

The Problem: If you ask the AI to "Build a website," it has to invent the steps from scratch every time. Sometimes it forgets a step, or does them in the wrong order.
The Solution: We give it a Recipe Book.
- Instead of improvising, the AI opens a book called "How to Build a Website." It follows the steps: 1. Get domain, 2. Design layout, 3. Write code.
- If the AI makes a mistake, we don't retrain the whole genius; we just update the recipe book.
- Analogy: It's the difference between a chef trying to invent a cake recipe from memory vs. following a trusted, tested recipe card.

C. Protocols: The "Company Handbook" (Externalized Interaction)

The Problem: When the AI talks to other computers or tools, it might speak "robot" in a weird way that the other computer doesn't understand. It's like trying to order coffee by shouting random words instead of using the menu.
The Solution: We give it a Company Handbook (Protocols).
- This handbook says: "When you talk to the database, you must use this specific sentence structure."
- It stops the AI from guessing and forces it to use a standard, safe language.
- Analogy: It's the difference between a chaotic shouting match and a formal meeting where everyone follows an agenda and uses specific terms.

D. The Harness: The "Office Manager" (The Unifier)

The Problem: You can have a filing cabinet, a recipe book, and a handbook, but if no one manages them, the office is a mess. The AI might grab the wrong file or ignore the rules.
The Solution: The Harness is the Office Manager.
- The Manager decides which file to pull, which recipe to open, and when to check the handbook.
- The Manager also acts as a security guard (stopping the AI from deleting files) and a quality control inspector (checking if the work is done right).
- Analogy: The AI is the engine, but the Harness is the car's chassis, steering wheel, and brakes. Without the car, the engine is just a loud, dangerous noise.

3. Why This Matters (The "Aha!" Moment)

The paper argues that we are moving from an era of "Bigger Brains" to an era of "Better Offices."

Old Way: "Let's make the AI so smart it never needs to look anything up." (This is getting harder and more expensive).
New Way: "Let's build a system where the AI doesn't need to remember everything because the system remembers for it."

The Magic Transformation:

Recall (trying to remember) becomes Recognition (finding the right file).
Improvisation (making it up as you go) becomes Composition (putting together pre-made blocks).
Chaos (random chatting) becomes Contracts (structured agreements).

4. The Future: A Self-Improving Office

The paper looks ahead to a future where the "Office Manager" (the Harness) gets smarter too.

Self-Evolving: If the AI keeps failing at a specific task, the Office Manager might automatically rewrite the Recipe Book or add a new rule to the Handbook without a human needing to step in.
Shared Infrastructure: Imagine a "Public Library" of skills and memories that all AI agents can use. If one AI learns how to fix a bug, it writes the recipe into the library, and every other AI can use it immediately.

Summary

This paper says that the future of AI isn't just about building a smarter robot. It's about building a better workspace for the robot. By moving memory, skills, and rules out of the robot's head and into a structured, managed environment, we make AI more reliable, safer, and actually useful for real-world jobs.

In short: Don't just train the brain; build the library, the rulebook, and the manager around it.

Based on the paper "Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering," here is a detailed technical summary.

1. Problem Statement

The paper addresses a fundamental limitation in the current paradigm of Large Language Model (LLM) agents: the reliance on parametric capabilities (model weights) and ephemeral context (prompts) to solve complex, long-horizon tasks.

The Mismatch: LLMs are excellent at flexible synthesis and reasoning over provided information but struggle with stable long-term memory, procedural repeatability, and governed interaction with external systems.
The Consequence: When agents attempt tasks requiring continuity across sessions, consistent multi-step workflows, or coordination with tools, they suffer from "amnesia" (loss of state), "variance" (inconsistent execution), and "coordination brittleness" (ad-hoc prompting failures).
The Gap: Existing literature often surveys specific components (e.g., RAG, tool learning) in isolation. There is a lack of a unified theoretical framework explaining why these components are converging and how they collectively transform the nature of agent intelligence.

2. Methodology and Theoretical Framework

The authors employ a systems-level review grounded in the theory of Cognitive Artifacts (Donald Norman) and Distributed Cognition.

Core Thesis: The paper argues that the evolution of LLM agents follows a logic of Externalization. Just as human history moved from internal memory to writing, printing, and digital computation to offload cognitive burdens, LLM agents are shifting cognitive burdens from the model's internal weights to explicit, persistent, and inspectable external structures.
Representational Transformation: The methodology analyzes how externalization transforms the task the model faces:
- Recall $\to$ Recognition: (Memory) The model no longer needs to regenerate history from weights; it recognizes and retrieves curated slices.
- Improvisation $\to$ Composition: (Skills) The model no longer invents workflows from scratch; it composes behavior from pre-validated components.
- Ad-hoc $\to$ Structured: (Protocols) Communication shifts from free-form prompting to governed, machine-readable contracts.
Analytical Approach: The paper traces the historical progression from Weights (parametric knowledge) $\to$ Context (prompt engineering/RAG) $\to$ Harness (infrastructure). It deconstructs the "Harness" into three externalization dimensions and one unifying engineering layer.

3. Key Contributions

A. The Three Dimensions of Externalization

The paper defines and analyzes three distinct but coupled forms of externalization:

Externalized State (Memory):
- Function: Externalizes temporal continuity.
- Content: Distinguishes between Working Context (live state), Episodic Experience (past trajectories), Semantic Knowledge (domain facts), and Personalized Memory (user preferences).
- Architecture: Evolves from Monolithic Context $\to$ Retrieval Stores $\to$ Hierarchical Orchestration $\to$ Adaptive Memory Systems (using RL/MoE for dynamic routing).
- Key Insight: Memory transforms the task from "remembering" to "recognizing relevant history," solving the continuity problem.
Externalized Expertise (Skills):
- Function: Externalizes procedural know-how.
- Content: Encodes Operational Procedures (step skeletons), Decision Heuristics (branching rules), and Normative Constraints (safety/governance).
- Lifecycle: Moves from atomic primitives $\to$ large-scale selection $\to$ packaged expertise. Skills are specified, discovered, progressively disclosed, bound to tools, and composed.
- Key Insight: Skills transform the task from "generating a workflow" to "selecting and following a reusable procedure," reducing variance.
Externalized Interaction (Protocols):
- Function: Externalizes interaction structure.
- Content: Defines contracts for Invocation Grammar (schemas), Lifecycle Semantics (state machines), Permission Boundaries, and Discovery Metadata.
- Types: Covers Agent-Tool (e.g., MCP), Agent-Agent (e.g., A2A), and Agent-User protocols.
- Key Insight: Protocols transform the task from "negotiating interfaces" to "operating within explicit contracts," ensuring interoperability and safety.

B. Harness Engineering (The Unifying Layer)

The paper introduces Harness Engineering as the critical runtime environment that coordinates the three dimensions. It is not a fourth module but the "cognitive environment" containing:

Control Flow: Agent loops, termination limits, and recursion bounds.
Sandboxing: Execution isolation and security boundaries.
Human Oversight: Approval gates and intervention points.
Observability: Structured logging and feedback loops for self-correction.
Context Budget Management: Dynamic allocation of token limits between memory, skills, and protocols.

C. Cross-Cutting Analysis & Trade-offs

Module Interaction: The paper maps the feedback loops (e.g., Memory $\to$ Skill distillation; Protocol $\to$ Memory assimilation).
Parametric vs. Externalized Trade-off: It proposes a framework for deciding what stays in the model (stable, fast, generic) vs. what is externalized (fast-changing, reusable, governable).
Input/Output Perspective: The harness reorganizes the model boundary into Contextual Input (Memory), Instructional Input (Skills), and Action Schema (Protocols).

4. Results and Findings

While a review paper, the authors synthesize findings from the literature to establish several critical conclusions:

Convergence: Contemporary agent systems (e.g., OpenAI Codex, Anthropic Claude Code, AutoGen) are independently converging on the same six harness dimensions (Loop, Sandbox, Oversight, Observability, Config, Context), suggesting these are structural requirements for reliable agency.
Reliability Source: The largest gains in agent reliability in 2024–2026 come not from larger base models, but from better external infrastructure (memory, skills, protocols).
Failure Modes: Failures are often due to representational design flaws (e.g., stale memory, unsafe skill composition, protocol mismatches) rather than model incompetence.
Embodied Extension: The externalization logic applies to robotics, where the "Cerebrum" (LLM) handles planning/memory and the "Cerebellum" (VLA) handles low-latency execution, mirroring the digital agent architecture.

5. Significance and Future Directions

Paradigm Shift: The paper redefines "Agent Engineering" as "Harness Engineering." The focus shifts from training better models to designing better cognitive environments.
Self-Evolving Harnesses: Future agents may autonomously revise their own memory policies, skill libraries, and orchestration logic, moving from static scaffolding to adaptive systems.
Shared Infrastructure: The field is moving from private agent scaffolding to shared, institutionalized infrastructure (shared memory, public skill registries, standard protocols), raising new governance and security challenges.
Evaluation: Current benchmarks are insufficient. The paper calls for new metrics focusing on transferability (does the harness work with different models?), maintainability, and context efficiency.

Conclusion:
The paper posits that the future of LLM agents lies in the co-evolution of models and external infrastructure. Reliable agency is no longer a property of the model alone but an emergent property of a system where cognitive burdens are systematically externalized into memory, skills, and protocols, all governed by a robust harness. This represents a fundamental shift in AI design from "bigger brains" to "better organized cognitive systems."