💻 computer science

AgentCo-op: Retrieval-Based Synthesis of Interoperable Multi-Agent Workflows

AgentCo-op is a retrieval-based framework that synthesizes interoperable multi-agent workflows by composing reusable tools and agents through typed artifact handoffs and local self-repair, enabling effective collaborative discovery in open-ended scientific domains while outperforming baselines on various benchmarks with reduced costs.

Original authors: Shuaike Shen, Wenduo Cheng, Shike Wang, Mingqian Ma, Jian Ma

Published 2026-05-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuaike Shen, Wenduo Cheng, Shike Wang, Mingqian Ma, Jian Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a complex machine, like a high-tech coffee maker that also sorts your mail and writes your daily journal. In the past, to make this happen with AI, researchers would try to "search" for the perfect design. They would generate thousands of different blueprints, test them, and hope one works. But this is like trying to find a specific needle in a haystack by building a new needle factory every time you look. It's slow, expensive, and often fails when the task is something new and messy, like real-world science, where there is no "correct answer key" to grade the work.

AGENTCO-OP is a new way to build these AI teams. Instead of searching for a new design from scratch, it acts like a master contractor who walks into a warehouse full of pre-built, specialized tools and workers, picks the right ones, and snaps them together into a working machine.

Here is how it works, using simple analogies:

1. The "Warehouse" vs. The "Search"

Most current AI systems try to search for the best workflow. Imagine trying to bake a cake by randomly mixing ingredients and tasting them until you get it right.
AGENTCO-OP uses retrieval. Imagine you are a chef who needs to bake a cake. Instead of inventing new recipes, you go to a well-organized warehouse (a library of skills and tools). You grab a pre-made "mixing skill," a "baking tool," and a "frosting agent." You don't reinvent the wheel; you just assemble the parts you need.

2. The "Typed Handoff" (The Secure Package)

When you pass a note to a friend, you might write it on a napkin. If the next person needs a specific form, the napkin might get lost or misunderstood.
In AGENTCO-OP, every time one AI agent passes a result to another, it puts the data in a strictly labeled, secure box (called a "typed artifact").

Example: If Agent A finds a list of genes, it doesn't just shout the list to Agent B. It puts the list in a box labeled "Gene List: 53 items, verified." Agent B opens the box, checks the label, and knows exactly what to do. This prevents the agents from talking past each other or getting confused by messy data.

3. The "Local Repair" (The Quick Fix)

Sometimes, even with the best plan, a machine breaks.

Old Way: If a part fails, the whole factory shuts down, and the engineers go back to the drawing board to design a brand-new machine from scratch.
AGENTCO-OP Way: If a specific part fails, a "Reviewer" (a smart supervisor) looks at the error and performs a local repair. It's like a mechanic who sees a loose bolt on the engine and tightens just that bolt, rather than rebuilding the whole car. The system fixes only the broken piece and keeps going.

4. The "Docker" (The Universal Adapter)

Scientists often have their own tools built in different environments (like one tool works on a Mac, another on a Linux server). Usually, they can't talk to each other.
AGENTCO-OP wraps these different tools in Docker containers. Think of this as putting every tool in a universal shipping container. No matter what the tool is or where it was built, once it's in the container, it can be plugged into the workflow seamlessly. This allows a "Gene Agent" and a "Tissue Agent" (who were built by different people at different times) to work together perfectly.

What Did They Actually Do?

The paper tested this system in two main ways:

A. The "Real World" Science Tests (Open-World)
They didn't use standard test questions. They used real, messy scientific problems:

The Heart Study: They connected two separate AI agents (one that looks at heart tissue, one that analyzes genes) to figure out why certain heart cells behave differently. The system built a workflow to pass data between them, and they successfully identified a specific biological pattern.
The Cell Study: They combined two different analysis methods (looking at RNA and looking at DNA accessibility) to find better markers for cell types. The system ran both analyses in parallel, compared the results, and produced a report that was more accurate than using either method alone.
The "Reuse" Test: They took a workflow that another AI had already designed and "fixed" it up using AGENTCO-OP's method, making it even better.

B. The Standard Tests (Benchmarks)
They also tested the system on standard math, coding, and question-answering tests (like the ones used to grade AI models).

The Result: AGENTCO-OP scored the highest on four out of six of these tests.
The Cost: It was also much cheaper to run. While other methods spent a lot of money "searching" for the right answer, AGENTCO-OP just assembled the right tools and fixed small errors as they happened, saving a significant amount of computer time and money.

The Bottom Line

The paper claims that AGENTCO-OP is a smarter way to build AI teams for complex tasks. Instead of trying to "search" for a perfect solution from scratch, it retrieves existing tools, snaps them together with strict rules, and fixes small errors on the fly. This makes it better at handling real-world science where there are no answer keys, and it saves money compared to older methods.

Technical Summary: AGENTCO-OP

Problem Statement

Designing interoperable multi-agent workflows is particularly challenging in open-ended scientific domains. Unlike standard benchmarks (e.g., QA, coding, math) that offer curated training sets, reliable scalar evaluation metrics, and standardized interfaces, scientific tasks often lack these resources. In fields like genomics, tasks are open-ended, ground-truth answers are frequently ambiguous or non-existent, and evaluation relies on heterogeneous intermediate evidence (e.g., statistical significance, biological plausibility) rather than a single scalar reward. Furthermore, existing scientific agents and tools are often developed independently with incompatible environments, interfaces, and provenance states. Current automated workflow design methods (e.g., ADAS, AFlow, AgentSquare) rely on searching over candidate topologies and optimizing against scalar feedback, a strategy that becomes impractical when representative benchmarks and reliable reward signals are unavailable.

Methodology: AGENTCO-OP

The authors propose AGENTCO-OP, a retrieval-based synthesis framework that reframes automated multi-agent workflow design as a composition problem rather than a global search problem. The system operates through five main stages:

Planning and Retrieval: Given a task specification $x = (g, c, r, \Omega)$ (goal, context, constraints, and resources), the system formulates a retrieval plan. It gathers relevant artifacts from curated libraries or user-provided repositories, including reference documentation, procedural agent skills, callable tools, and metadata from external GitHub repositories.
Synthesis: AGENTCO-OP synthesizes an initial executable workflow as a directed graph $G = (V, E)$ $G = (V, E)$ .
- Graph Representation: Nodes represent agents with assigned roles, which can be external agents wrapped in Docker containers or end-to-end methods. Edges represent the flow of information and intermediate artifacts.
- Node Grounding: Each node is grounded with specific skills and tools retrieved from the library, mapped via a function $\phi$ .
- Interface Alignment: The system enforces typed artifact handoffs. Communication between agents is governed by a protocol $\Pi$ where artifacts (e.g., validated marker tables, structured JSON) are passed through standardized schemas, ensuring compatibility between heterogeneous components.
- Containerization: External repositories and methods are wrapped in Docker containers (following Repo2Run) to resolve environment dependencies and allow them to function as graph nodes.
Execution: The synthesized workflow is executed. During this phase, the system monitors heterogeneous evidence, including execution traces, validation checks, tool errors, and cost signals.
Evidence-Guided Local Repair: If execution evidence indicates failure or uncertainty, a "Reviewer" triggers bounded local repair. Instead of restarting the global synthesis or search, the system consults a set of repair policies to revise only the implicated nodes, attached skills, tools, or communication edges. This process is bounded by a repair budget or maximum rounds.
Composition with Search-Based Workflows: AGENTCO-OP can also ingest an existing agent graph produced by a search-based method (e.g., AFlow) as a structural prior. It then refines this graph by grounding nodes with retrieved skills and applying local repair, demonstrating that synthesis and search are complementary.

Key Contributions

Retrieval-Based Synthesis Formulation: The paper formulates automated workflow design as a retrieval-based synthesis problem for settings where scalar rewards are weak or unavailable. It instantiates this in AGENTCO-OP, which dynamically composes resources, skills, tools, and external agents into executable workflows via typed artifact handoffs and bounded local repair.
Coordination of Independent Scientific Agents: The framework demonstrates the ability to coordinate independently developed scientific agents and tool repositories in open-world genomics tasks. Given only task specifications and GitHub links, it automatically synthesizes interoperable workflows without redesigning the underlying repositories.
Complementarity of Synthesis and Search: The authors show that synthesis and search are complementary by importing a workflow searched by AFlow and improving it through retrieval grounding and local repair.
Performance and Efficiency: On six standard benchmarks (QA, math, code), AGENTCO-OP achieves the best performance on four benchmarks and the best average score under a unified backbone setting, while consistently reducing per-task token costs compared to multi-agent baselines.

Experimental Results

The paper evaluates AGENTCO-OP in two regimes:

1. Open-World Scientific Case Studies:

Spatial Transcriptomics Collaboration: AGENTCO-OP coordinates TissueAgent (spatial transcriptomics) and GeneAgent (gene-set interpretation) to analyze a developing human heart MERFISH dataset. The system successfully synthesizes a serial workflow where TissueAgent identifies 53 upregulated marker genes, which are validated and passed as a typed artifact to GeneAgent for interpretation. The agents conclude that AVN/AV ring fibroblasts represent a specialized developmental state.
Cross-Modality Marker Discovery: The system composes Seurat (RNA-seq) and Signac (ATAC-seq) workflows in parallel on PBMC multiome data. It integrates results from both modalities, evaluating marker quality against CellMarker 2.0 and PanglaoDB. The combined approach yields higher macro precision and recall than either modality alone.
Workflow Reuse: By importing an AFlow-searched graph for the MBPP benchmark and refining it, the system achieves a pass@1 score of 87.5, outperforming both AFlow alone (78.2) and AGENTCO-OP built from scratch (87.1).

2. Standard Benchmarks:

Evaluated on HotpotQA, DROP, HumanEval, MBPP, GSM8K, and MATH using GPT-4o-mini.
AGENTCO-OP achieves the highest scores on four of the six benchmarks and the highest average score.
Cost Analysis: AGENTCO-OP significantly reduces test-time token costs compared to discussion-based methods (e.g., LLM-Debate, ReConcile) and search-based methods (e.g., AFlow), which incur high costs during training/search or repeated inter-agent communication.

Significance and Claims

The paper claims that retrieval-based synthesis extends automated agentic workflow design beyond benchmark-optimized agent graphs to open-world workflows built from existing agents, tools, and typed artifacts.

Scalability to Open-Ended Tasks: By avoiding reliance on global topology search and scalar rewards, AGENTCO-OP addresses the bottleneck in scientific domains where tasks lack curated benchmarks.
Interoperability: The framework solves the interoperability challenge by treating skills as typed, testable units and enforcing interface alignment through Docker wrapping and typed artifacts, allowing independently developed systems to collaborate.
Efficiency: The synthesis-first approach, combined with bounded local repair, avoids the expensive trial-and-error of global search and the redundant communication of debate-based systems, leading to lower costs and faster adaptation to specific problem instances.
Scientific Utility: The work suggests that progress in scientific agents depends not just on building stronger individual agents, but on developing methods to organize, compose, and adapt heterogeneous agents into reliable collaborative systems. AGENTCO-OP provides a practical path toward a reusable ecosystem where existing expertise can be composed and verified to address complex cross-domain discovery challenges.

The authors acknowledge limitations, including the current focus on single-domain case studies, dependence on the quality of available resources, and the fact that local repair does not guarantee global optimality. However, they posit that this paradigm offers a robust alternative to search-based design for real-world scientific applications.