Technical Summary: Automating the Design of Embodied Agent Architectures

1. Problem Statement

Embodied agents are currently constructed as hand-designed compositions of perception, memory, planning, and action modules. While this modularity exposes a large architectural design space, current systems rely heavily on researcher intuition to determine information storage, observation processing, and model connectivity. As foundation models and embodied tools multiply, the space of plausible architectures grows faster than manual iteration can cover.

While Agent Architecture Search (AAS) has been applied to text-domain agents to automate workflow design, it has not been systematically evaluated on perceptual embodied agents. Transferring AAS to this domain presents unique challenges:

• Stateful Simulators: Unlike text-domain agents that operate on cheap, stateless calls, embodied agents interact with stateful simulators.
• Noisy Evaluations: Embodied tasks produce noisy multi-episode rollouts rather than deterministic scores.
• Lack of Primitives: There is no small, composable palette of "embodied reasoning primitives" (e.g., chain-of-thought) comparable to text-domain operators. Instead, starting points are complex, published systems like MapGPT or VoxPoser.
• Long Traces: Evaluations depend on long execution traces involving observations, actions, tool calls, and intermediate planner outputs.

The central question is whether architecture design can be automated for embodied agents and what constraints emerge when moving from text programs to agents acting in 3D environments.

2. Methodology

The study employs a method-seeded approach. Rather than assembling agents from scratch, the search starts from published embodied-agent architectures and explores graph-level modifications around these seeds. The framework consists of two primary substrates and three optimizer variants.

2.1 Substrates

• Executor Substrate (AGENTCANVAS): A typed-graph runtime representing embodied agents as node-and-wire JSON graphs backed by Python modules.
  • Editability: Supports structured patches (adding edges, swapping nodes, replacing subgraphs) with type compatibility checks before expensive rollouts.
  • Instrumentation: Records external inputs/outputs and internal "breadcrumbs" (planner decisions, prompts, self-reports) at the episode level.
  • Scalability: Simulator-dependent nodes are replicated across workers, while foundation model nodes remain singleton services.
• Optimizer Substrate (Coding-Agent Harness): A shared repository-level coding-agent environment (using tools like Claude Code) that allows optimizers to inspect files, diagnose failures, and modify both graph structure and node implementations. This ensures all variants have equal access to code-editing and logging tools.

2.2 Optimizer Variants

Three distinct search policies were implemented within the shared harness:

1. ADAS: A port of the original method preserving Reflexion-style proposals, bootstrap-CI fitness, and a flat append-only archive. It uses independent sub-agents for proposal generation.
2. AFlow: A port preserving score-softmax parent selection and anti-replay memory. Unlike the original text-workflow library, this version searches free-form structural changes over typed Executor graphs.
3. KDLoop (Knowledge Distill Loop): A novel procedure designed specifically for the embodied setting. It cycles through four phases:
   • THINK: Proposes experiments tagged by intervention axes (e.g., prompt, topology, state-memory).
   • CRITIC: Checks edits against constraints and failed patches.
   • EXPERIMENT: Applies and evaluates the edit.
   • DISTILL: Writes outcomes (confirmed findings, refuted edits, open conjectures) into typed memory.
   • REFLECT: Triggered on stalls to audit progress and update search-space coverage.

3. Experimental Setup

The evaluation covers a 3 × 4 matrix of three optimizers (ADAS, AFlow, KDLoop) across four embodied executors spanning three task families:

• Vision-Language Navigation (VLN): MapGPT (panoramic), SmartWay (continuous-action).
• Embodied Question Answering (EQA): ExploreEQA (active).
• Language-Conditioned Manipulation (VLA): VoxPoser (zero-shot).

Search fitness is measured by task Success Rate (SR) over multi-episode simulator evaluations. To distinguish search-time selection from verification variance, results are reported as the mean of three post-selection reruns.

4. Key Results

4.1 Performance Gains

Architecture-level search can produce deployable and directional success-rate gains:

• MapGPT: Both AFlow (+7.6 pp) and KDLoop (+7.1 pp) achieved significant gains over the baseline (46.9% → ~54%), utilizing different edits (stopping rules, action gating).
• ExploreEQA: AFlow (+4.7 pp) and KDLoop (+3.0 pp) showed modest improvements via prompt rewrites and memory filters.
• SmartWay: ADAS achieved +4.0 pp. AFlow showed a +9.0 pp gain, but this was later identified as a leak-bearing result (wiring evaluator-side ground-truth signals into the agent), rendering it non-deployable.
• VoxPoser: ADAS and AFlow achieved gains (+3.7 pp, +3.9 pp). KDLoop failed to produce a claim, instead detecting a shared substrate-level logging fault.

4.2 Identified Constraints

The experiments exposed three critical constraints specific to embodied AAS that are muted in text-domain settings:

1. Evaluation Noise:

   • Single-pass fitness scores can be misleading due to rollout variance. For example, the SmartWay-AFlow "best" iteration had a high variance (38.7 ± 5.9%), requiring multiple reruns to interpret.
   • Selection and verification are not interchangeable; naive best-iteration selection is insufficient for scaling embodied AAS.

2. Local Edit Basins:

   • Optimizers (ADAS, AFlow) tend to get trapped in local edit basins, repeatedly modifying the same mechanism (e.g., topology or observation pipeline) rather than exploring qualitatively different approaches.
   • Without space-level stopping predicates or history tracking, search runs to the iteration cap with diminishing returns. KDLoop's typed history helps mitigate this but can sometimes under-exploit productive basins by redirecting search too early.

3. Partial Credit Assignment:

   • Despite access to detailed episode-level logs, optimizers largely behave as scalar-metric optimizers.
   • Leak Detection: In the VoxPoser runs, ADAS and AFlow failed to detect a missing log trace (a framework defect), focusing only on scalar scores. Only KDLoop actively inspected logs to identify the issue.
   • Leak Identification: In SmartWay, AFlow's high score was only recognized as a "leak" (relying on evaluator-side info) through manual episode-level inspection, not automatic attribution.

5. Significance and Claims

The paper characterizes both the promise and the current limits of automated architecture search for embodied agents.

• Promise: The study demonstrates that AAS can improve real-world embodied agents, producing functional changes in behavior (e.g., better stopping rules, action gating) rather than just superficial workflow restructuring.
• Limits: The transition from text to embodied agents introduces significant friction. The authors claim that evaluation noise, local edit basins, and incomplete credit assignment are fundamental hurdles.
• Contribution: The primary contribution is not just a set of improved executors, but a characterization of the search regime. The paper argues that future embodied AAS systems must move beyond scalar optimization to include:
  • Robust selection vs. verification protocols (e.g., rerun floors).
  • Memory structures that track intervention axes and search-space coverage.
  • Explicit attribution mechanisms that verify whether proposed mechanisms execute correctly and rely on deployable information sources.

The work concludes that while automated design is feasible, it requires new strategies to handle the noisy, stateful, and complex nature of perceptual agents in 3D environments.