🤖 AI

ROMA: Recursive Open Meta-Agent Framework for Long-Horizon Multi-Agent Systems

The paper introduces ROMA, a recursive, modular multi-agent framework that overcomes the limitations of current systems in long-horizon tasks through dependency-aware task decomposition, structured aggregation, and a GEPA+ prompt optimization strategy, achieving state-of-the-art performance on reasoning and long-form generation benchmarks.

Original authors: Salaheddin Alzu'bi, Baran Nama, Arda Kaz, Anushri Eswaran, Weiyuan Chen, Sarvesh Khetan, Rishab Bala, Tu Vu, Sewoong Oh

Published 2026-02-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Salaheddin Alzu'bi, Baran Nama, Arda Kaz, Anushri Eswaran, Weiyuan Chen, Sarvesh Khetan, Rishab Bala, Tu Vu, Sewoong Oh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to write a massive, 500-page novel, or solve a complex mystery that involves checking hundreds of different websites. If you ask a single AI (like a smart chatbot) to do this all at once, it often gets overwhelmed. It might forget the beginning of the story by the time it reaches the end, get confused by conflicting information, or just give up because the task is too big for its "brain" (context window).

ROMA is a new framework designed to fix this. Think of it not as a single super-brain, but as a highly organized construction company for AI tasks.

Here is how it works, using simple analogies:

1. The Problem: The "One-Person Band" vs. The Orchestra

Current AI systems often try to do everything in one long line of thought. It's like asking one person to be the architect, the bricklayer, the electrician, and the interior designer all at once, while trying to remember every single detail of a 50-story building. They get tired, make mistakes, and lose track of the plan.

ROMA changes this by building a hierarchical team. Instead of one giant brain, it uses a recursive (repeating) structure where big problems are broken down into tiny, manageable pieces.

2. The Four Key Roles (The Construction Crew)

ROMA organizes the work into four specific roles. Every time a task comes in, it passes through this cycle:

The Atomizer (The Foreman):
- What it does: It looks at a big task and asks, "Can we do this in one go, or do we need to break it down?"
- Analogy: Imagine you order a custom house. The Foreman looks at the blueprint. If you just need a single brick, he hands it to a worker. But if you need a whole house, he says, "No, we need to break this into foundations, framing, plumbing, and roofing."
The Planner (The Architect):
- What it does: If the task is too big, the Planner draws a map. It figures out the order of operations and who depends on whom.
- Analogy: The Architect draws the blueprints. They know you can't paint the walls before you build the frame. They create a schedule where different teams can work simultaneously (parallel) on different parts of the house without getting in each other's way.
The Executors (The Workers):
- What it does: These are the actual doers. They tackle the small, "atomic" (indivisible) tasks.
- Analogy: These are the specialized workers. One team lays the bricks, another wires the electricity, and another paints. Because the tasks are small, they don't get confused or forget what they are doing.
The Aggregator (The Project Manager):
- What it does: Once the workers finish their small tasks, the Aggregator collects their work, checks it, summarizes it, and passes the "summary" up to the next level.
- Analogy: Instead of the CEO reading every single brick report, the Project Manager takes the reports from the framing team, the plumbing team, and the electrical team, summarizes them into a "Level 1 Complete" report, and hands that up to the main boss. This keeps the main boss from getting overwhelmed by too much detail.

3. The Magic Trick: "Recursive" and "Compression"

The word Recursive just means the process repeats itself.

The Project Manager (Aggregator) might get a report that is still too big. So, that report becomes a new task for a new Foreman, who breaks it down again, and so on.
Compression: This is the secret sauce. As the work moves up the chain, the Aggregators don't pass up everything. They pass up summarized, verified results.
- Analogy: Imagine reading a 1,000-page book. Instead of trying to remember every sentence, you write a one-page summary of each chapter. By the time you get to the end, you have a 10-page summary of the whole book, not the 1,000 pages. This prevents the AI from "forgetting" the beginning of the task (a problem known as "context rot").

4. GEPA+: The "Tuning Knob"

Even with a great team, sometimes the instructions (prompts) given to the workers aren't perfect.

GEPA+ is like a smart coach that watches the team work.
It tries out different ways of giving instructions to the Foreman, Architect, and Workers. It tests them, sees which ones work best, and then combines the best ideas into a new, better set of instructions.
Analogy: It's like a chef tasting a soup and adjusting the salt, pepper, and herbs. GEPA+ does this automatically and quickly, finding the perfect recipe for the specific task without needing to retrain the whole AI from scratch.

Why Does This Matter?

The paper shows that this "Construction Company" approach works incredibly well:

Better Reasoning: When the AI has to find facts on the internet that contradict each other, ROMA breaks the search into small pieces, checks them individually, and then synthesizes the truth. It beat other top AI research agents by a wide margin.
Better Writing: When writing long stories, ROMA plans the plot and characters first, then writes chapter by chapter, ensuring the story stays consistent from start to finish. It allowed an open-source model to match the quality of the most expensive, closed-source models.

The Bottom Line

ROMA proves that you don't need a single, god-like AI to solve hard problems. Instead, you need a structured system that breaks big problems into small ones, assigns them to specialized workers, and carefully summarizes the results as they go back up the chain. It's the difference between a chaotic crowd trying to build a house and a well-oiled construction crew with a clear plan.

1. Problem Statement

Current agentic frameworks struggle with long-horizon tasks (complex, multi-step workflows like deep research or long-form writing). The paper identifies three primary failure modes in existing systems:

Brittleness in Sequential Orchestration: As reasoning depth increases, linear chains of thought or simple sequential workflows become fragile.
Context Window Limits: Accumulating intermediate reasoning, tool outputs, and artifacts leads to "context rot," where performance degrades as the context grows, eventually exceeding model limits.
Opacity and Debuggability: Execution traces are often unstructured or hidden (especially in closed-source systems), making it difficult to localize failures, attribute errors to specific planning or retrieval steps, or debug complex branching executions.
Lack of Standardization: Most systems are built ad-hoc with hard-coded control flows, preventing transferability across domains and making it difficult to mix heterogeneous models or tools effectively.

2. Methodology: The ROMA Framework

The authors introduce ROMA (Recursive Open Meta-Agents), a domain-agnostic framework that standardizes agent construction around a fixed recursive control loop.

Core Architecture

ROMA operates on a task tree where every node executes the same four modular roles. This creates a hierarchical, transparent execution trace:

Atomizer: Determines if a task is atomic (can be solved in one pass) or non-atomic (requires decomposition).
- If Atomic: The task is passed to an Executor.
- If Non-Atomic: The task is passed to a Planner.
Planner: Decomposes non-atomic tasks into a dependency-aware subtask graph (a Directed Acyclic Graph or DAG).
- Subtasks are MECE (Mutually Exclusive and Collectively Exhaustive).
- Dependencies are explicit, allowing independent branches to execute in parallel.
Executor: Handles atomic tasks.
- Supports heterogeneous execution strategies (e.g., ReAct for search, CoT for reasoning, CodeAct for coding).
- Operates on localized context to prevent context explosion.
Aggregator: Synthesizes, verifies, and compresses the outputs of all child subtasks.
- Instead of passing raw transcripts upward, it produces a concise, verified summary (artifact) for the parent node.
- This "bottom-up" aggregation controls context growth and ensures the parent only sees relevant, distilled information.

Key Design Principles

Recursive Control Loop: The same logic applies at every level of the tree, ensuring consistency and scalability.
Heterogeneity: Different models can be assigned to different roles (e.g., a smaller, cheaper model for planning; a larger, more capable model for complex reasoning) based on cost and capability.
Artifact Management: Intermediate results are stored in an object store and referenced via typed signatures rather than embedded in prompts, reducing token usage.
Task Types: The system supports specific task types (Search, Think, Write, Code), each routed to specialized execution strategies.

3. Key Contributions

A. ROMA Framework

A unified, recursive meta-agent architecture that replaces ad-hoc orchestration with a standardized Atomizer $\to$ Planner $\to$ Executor $\to$ Aggregator loop.
Solves context growth issues through bounded aggregation (compressing results at each step) and parallel execution of independent subtasks.
Provides transparent, hierarchical execution traces, enabling precise debugging and error localization in deep reasoning chains.

B. GEPA+ (Prompt Optimization)

To adapt ROMA to specific tasks without fine-tuning, the authors introduce GEPA+, an extension of the Genetic-Pareto prompt evolution method.
Multi-Component Optimization: Unlike standard GEPA which optimizes a single prompt, GEPA+ jointly optimizes prompts for all four components (Atomizer, Planner, Executors, Aggregator).
Mechanism:
1. Diverse Proposals: Generates $K$ candidate prompt edits in parallel using diverse models.
2. Reranking: Uses learned judges and task-specific verifiers (e.g., unit tests, consistency checks) to score candidates.
3. Contract-Preserving Merge: Merges the best candidates into a single update that respects interface contracts (e.g., ensuring the Planner still outputs valid DAGs).
Efficiency: GEPA+ achieves 2–6 point accuracy gains while requiring 3–4 $\times$ fewer metric evaluations than standard GEPA.

4. Experimental Results

ROMA was evaluated across five benchmarks covering reasoning, search, and long-form generation.

Benchmark	Task Type	Key Result
SEAL-0	Reasoning over conflicting web evidence	45.9% accuracy (GLM-4.6 + ROMA). +9.9% over Kimi-Researcher (36.0%) and +14.4% over Perplexity Deep Research (31.5%).
FRAMES	Multi-hop factual reasoning	82.3% accuracy. Outperforms Kimi-Researcher (78.8%) and general reasoning models (DeepSeek-R1 < 35%).
SimpleQA	Short-form factual QA	93.9% accuracy. Matches the top closed-source system (Liner Pro Reasoning at 95.3%) and outperforms Kimi-Researcher (93.6%).
EQ-Bench	Long-form writing (8 turns)	79.8% score (DeepSeek-V3 + GEPA+). Matches leading closed-source models like Claude Sonnet 4.5 (79.8%).
AbGen	Designing scientific ablation studies	4.93/5.0 Overall score. Best performance on Importance, Soundness, and Overall metrics among all baselines.

Efficiency Findings:

On EQ-Bench, GEPA+ improved DeepSeek-V3's performance from 71.9% to 79.8%.
GEPA+ reduced optimization cost by ~73% compared to standard GEPA (150 metric calls vs. 560).

5. Significance and Impact

Scalability: ROMA demonstrates that recursive, modular architectures can scale reasoning depth effectively without succumbing to context limits or performance degradation.
Interpretability: By externalizing the control loop into explicit roles, ROMA makes agent behavior transparent, allowing for systematic debugging and failure attribution.
Model Agnosticism: The framework decouples orchestration from specific foundation models, enabling the use of heterogeneous model fleets (mixing open-source and closed-source, small and large) to optimize for cost and latency.
Practical Adaptation: GEPA+ provides a robust, fine-tuning-free method to specialize complex multi-agent systems for new domains, significantly lowering the barrier to deploying high-performance agentic workflows.

In conclusion, ROMA establishes a new standard for building long-horizon agentic systems by replacing brittle, ad-hoc pipelines with a structured, recursive, and transparent framework that balances performance, interpretability, and efficiency.