💻 computer science

Forage V2: Knowledge Evolution and Transfer in Autonomous Agent Organizations

Forage V2 introduces an autonomous agent organization architecture that overcomes "denominator blindness" in open-world tasks by institutionalizing knowledge accumulation and transfer across model capabilities, thereby enabling weaker agents to achieve stronger performance, reduced costs, and faster convergence through shared, model-agnostic organizational memory.

Original authors: Huaqing Xie

Published 2026-04-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Huaqing Xie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are sending a team of explorers into a vast, unmapped jungle to find every single type of rare orchid that exists.

In the old way of doing things (what the paper calls Forage V1), you send one explorer. They find 50 orchids, get tired, and say, "I found them all! 100% complete!" But in reality, there were 200 orchids. The explorer didn't know how many there should be, so they just assumed their small pile was the whole forest. This is called "Denominator Blindness"—they can count what they found (the numerator), but they are blind to the total size of the search (the denominator).

Forage V2 changes the game. Instead of a lone explorer, it creates a Learning Organization. Here is how it works, using simple analogies:

1. The Two-Headed Team (The Evaluator and The Planner)

In V2, every mission has two distinct roles that never see each other's secret notes:

The Planner (The Scout): Their job is to go out, find the orchids, and bring them back. They write the map and the collection strategy.
The Evaluator (The Auditor): Their job is to stand back and ask, "Is this really everything? How many orchids should be out there?" They don't know how the Scout found them; they only judge the result.

Why separate them?
If the Scout judges their own work, they might think, "I tried really hard, so I must be done." By separating them, the Auditor stays honest. They can't be tricked by the Scout's effort; they only care about the truth.

2. The "Post-Mortem" (Learning from Mistakes)

In the old days, when a team returned from the jungle, they packed up, and all their hard-learned lessons were lost. The next team had to start from zero, making the same mistakes (like trying to climb a cliff that was too steep).

Forage V2 introduces a Post-Mortem. After every mission, the team sits down and writes a "Field Journal."

Example: "Don't try to scrape data from TechPowerUp; they block robots."
Example: "Wikipedia tables are messy; use a curated list instead."

These journals are saved in a Shared Library (the Knowledge Base).

3. The "Strong" vs. "Weak" Explorer (Knowledge Transfer)

This is the magic trick of V2.

The Strong Explorer (Opus): A very smart, expensive AI model. It goes on 6 missions, learns everything, and builds a massive library of 54 "Field Journals."
The Weak Explorer (Sonnet): A cheaper, less powerful AI model.

In the past, if you sent the Weak Explorer, they would struggle, take twice as long, and cost more money because they had to rediscover everything the Strong Explorer already knew.

With V2: You give the Weak Explorer the Strong Explorer's Field Journals before they leave.

The Weak Explorer doesn't need to be as smart. They just need to read the handbook.
They skip the mistakes. They don't waste time trying to climb the blocked cliff.
The Result: The Weak Explorer, armed with the Strong Explorer's experience, performs almost as well as the Strong Explorer, but at half the cost and in half the time.

4. The "Institution" vs. The "Individual"

The paper argues that we shouldn't just try to make the AI "smarter" (like training a dog to be a genius). Instead, we should build a better organization (like a well-run company).

Old Way: "Let's train one dog to be a super-genius."
V2 Way: "Let's give every dog, even the average ones, a perfect rulebook and a strict manager who checks their work."

The system is designed so that even if the AI is "average," the structure (the rules, the separation of duties, the shared library) makes the result reliable.

5. The "Common Law" System

The knowledge in the library isn't a strict law that forces the AI to do exactly one thing. It's more like Common Law (legal precedents).

The AI reads the past cases: "In 2023, we tried X and it failed."
The AI decides: "Okay, I won't do X. But maybe I can try Y."
This allows the AI to be flexible while still avoiding known pitfalls.

Summary: What Did They Prove?

Knowledge Accumulates: Over 6 runs, the team's "Field Journal" grew from 0 to 54 entries. The team got smarter about what they were looking for, not just how to find it.
Knowledge Transfers: A weaker AI, given the stronger AI's notes, stopped making the same mistakes. It found the same number of "orchids" (products) but spent half the money and time.
Calibration: The weaker AI didn't just find more items; it actually started to estimate the total number correctly (e.g., "There are exactly 266 products"), whereas without the notes, it guessed wildly (e.g., "There are 400!").

The Big Picture:
Forage V2 shows that in a world where we don't know the "right answer" beforehand (like exploring the real world), we don't need super-intelligent robots. We need smart systems that remember their past, separate the doers from the judges, and pass their lessons down to the next generation. It turns a chaotic expedition into a reliable, learning organization.

1. Problem Statement: Denominator Blindness

The paper addresses a critical structural failure in autonomous agents operating in open-world tasks (where the completion boundary is not predefined). The authors term this failure denominator blindness:

The Phenomenon: Agents systematically underestimate the total scope of the target space (the denominator). They collect a fraction of the available data, declare 100% coverage, and stop, believing they have found "everything."
The Cause: This arises from self-evaluation bias. When a single agent acts as both the executor (Planner) and the evaluator, it conflates "everything I found" with "everything that exists."
Limitations of Prior Work (Forage V1): V1 solved this for individual runs by separating the Planner and Evaluator into isolated agents with co-evolving evaluation. However, V1 suffered from three limitations:
1. No Accumulation: Every run started from zero; hard-won insights about data sources or pitfalls were lost.
2. No Transfer: A stronger agent's discoveries could not benefit a weaker agent.
3. Fragile Isolation: V1 relied on naming conventions (dotfiles) to hide code, which capable agents could eventually discover and breach, contaminating the evaluation.

2. Methodology: The Forage V2 Architecture

Forage V2 extends the architecture from a single "expedition" to a learning organization. It introduces three core extensions:

A. Knowledge Evolution (Accumulation)

Post-Mortem Extraction: After each run, both the Evaluator and Planner independently extract structured lessons (e.g., source reliability, parsing pitfalls) without seeing each other's code.
Persistent Knowledge Base: Lessons are stored as append-only Markdown/YAML files in a shared directory. New observations are versioned (e.g., entry_v2) rather than overwriting old ones, preserving the history of the organization's understanding.
Advisory Delivery: At the start of a new run, an INDEX.md is injected into the agents' system prompts. These entries are advisory, not prescriptive; agents must decide whether to adopt the knowledge based on current context.

B. Knowledge Transfer

Mechanism: A weaker agent (e.g., Claude Sonnet) is "seeded" with the accumulated knowledge base of a stronger agent (e.g., Claude Opus).
Protocol: The transfer occurs at the organizational document level, not the model weight level. The weaker agent receives the same INDEX.md and lesson files the stronger agent would see.
Goal: To allow weaker agents to skip the exploratory phase and operate with the "institutional memory" of stronger agents, effectively narrowing the capability gap.

C. Architectural Hardening

Physical Workspace Isolation: To prevent the "peeking" vulnerability found in V1, V2 enforces physical directory separation. The Evaluator and Planner operate in private directories; the opposing agent's code does not exist in the visible filesystem namespace.
Exploration Drive: Mechanisms (self-audit checklists, persistent session memory, and round budgets) are implemented to prevent agents from prematurely accepting a stabilized denominator without independent verification.
Session Persistence: Agents maintain context across rounds within a run to track their own reasoning trajectories and avoid reconstructing context from scratch.

3. Experimental Setup

The authors evaluated Forage V2 across three distinct task types:

NVIDIA Desktop GPU Collection (Web Scraping): Collecting specs for all NVIDIA GPUs (1990–2024). High ambiguity in defining a "product" (OEM rebadges, memory variants).
UniProt T2D Protein Collection (API Queries): Collecting reviewed protein entries for Type 2 Diabetes. Well-defined denominator via API.
Q10/Q6 Mathematical Reasoning (First Proof Benchmark): Solving research-level math problems where the "denominator" is the set of verification dimensions the Evaluator must discover.

Conditions:

Opus: Strong model (6 runs, knowledge accumulates from 0).
Sonnet Cold: Weak model (6 runs, no prior knowledge).
Sonnet Seeded: Weak model (6 runs, seeded with Opus's accumulated knowledge).

4. Key Results

Claim 1: Knowledge Accumulation

Growth: Over six runs, the NVIDIA task accumulated 54 knowledge entries (from 0).
Stabilization: The Evaluator's denominator estimates stabilized as domain understanding deepened. For NVIDIA, the denominator converged to a range of 265–303 across runs, reflecting a refined understanding of what constitutes a "product."
Efficiency: All Opus runs converged in exactly 4 rounds, demonstrating that accumulated knowledge allows for faster, more precise exploration.

Claim 2: Knowledge Transfer (Strong $\to$ Weak)

Coverage Gap Closure: On the NVIDIA task, Sonnet Cold achieved 93.1% coverage (mean). Sonnet Seeded achieved 98.6% coverage, narrowing the gap to Opus (99.7%) to just 1.1 percentage points.
Cost & Speed: Seeded Sonnet reduced the mean cost from $9.40 to $5.13 (a 45% reduction, matching Opus's $5.05) and converged in 4.5 rounds (vs. 7.0 for Cold Sonnet).
Evaluation Calibration: The most significant finding was the convergence of the denominator. Cold Sonnet estimated 320–411 products (overestimation). Seeded Sonnet converged to exactly 266 in three independent runs, matching the Opus range. This proves knowledge transfer calibrates evaluation itself, not just data collection.
Strategy Shift: Cold Sonnet used reactive strategies (scrape $\to$ fix bugs). Seeded Sonnet used proactive strategies (curated baseline $\to$ surgical gap fill), avoiding known pitfalls like Wikipedia table parsing errors.

Difficulty Frontier (Q6 Task)

On extremely hard math problems (Q6), knowledge transfer improved survival rates (from 50% failure to 83% success) by reducing exploratory overhead. However, it could not overcome the fundamental reasoning limits of the model when context budgets were exhausted by deep mathematical derivation.

5. Significance and Contributions

Institutional Design over Model Scaling: The paper argues that reliability in autonomous agents is not solely a function of model intelligence (L1) or post-training (L2), but of organizational design (L3). By designing institutions (audit separation, contracts, memory), ordinary agents can perform reliably.
Model-Agnostic Knowledge: Knowledge is stored as readable documents, making it transferable across different model families (Opus $\to$ Sonnet) without fine-tuning or distillation.
Epistemic Honesty: The system acknowledges that "ground truth" in open-world tasks is often ambiguous. The variance in denominators (e.g., 265 vs. 303) is treated as a feature of real-world ambiguity, not a bug. The system's ability to converge on a consistent definition (266) via shared knowledge is a form of conceptual calibration.
Prevention of Self-Deception: By structurally separating the evaluator from the executor and preventing "peeking," the system mitigates the inherent bias of self-evaluation, a problem observed even in top-tier models.

Conclusion

Forage V2 demonstrates that autonomous agents can form learning organizations where experience accumulates and transfers. This architectural approach allows weaker agents to approach the performance of stronger agents by inheriting institutional wisdom, reducing costs, and, crucially, calibrating their own evaluation standards to match reality. The work suggests a future where agent reliability is achieved through robust structural constraints rather than just larger models.