Technical Summary: MemCompiler — State-Conditioned Memory for Embodied Agents

Problem Statement

Current memory systems for embodied agents predominantly rely on an Ahead-of-time Monolithic Memory Injection (AMMI) paradigm. In this approach, retrieved memory entries are assembled into a static context at the beginning of an episode and remain fixed throughout execution. The authors argue this design suffers from attention dilution: as the agent's execution state evolves, the static memory becomes increasingly misaligned with current needs. Valuable experience is buried under irrelevant or counterproductive entries, causing lightweight policy executors to degrade below the no-memory baseline.

This issue is specific to embodied agents due to two structural properties absent in language agents:

1. State-Dependent Relevance: Memory usefulness depends on inferred subgoal progress and environmental beliefs derived from continuous visual-action streams, not just explicit queries. Static injection cannot adapt to these shifts.
2. Multimodal Nature of Experience: Embodied experience contains spatial and affordance information difficult to encode in text. Text-only memory loses critical cues, while naive visual injection amplifies attention dilution.

Existing solutions for language agents (e.g., structured graphs, progressive disclosure) assume powerful reasoning backbones and text-centric interaction, making them unsuitable for lightweight embodied policies operating over visual streams.

Methodology: MemCompiler

The paper proposes MemCompiler, a paradigm shift from static injection to State-Conditioned Memory Compilation (SCMC). Analogous to a software compiler translating source code into target instructions based on the runtime environment, MemCompiler treats task memory as "source code" and the agent's current state as "runtime context."

Core Components

1. Brief State ($b_t$): A structured, runtime context maintained by the Memory Compiler. It compresses the task goal, action history, and observations into two dimensions:
   • Task Progress State: Completed/current/pending subgoals.
   • Environment Belief: Known object locations, agent status, and environmental facts.
2. Memory Compiler ($\pi_C$): A lightweight model that reads the current state $s_t = (o_t, b_t)$ and the memory pool $M$. At each step, it dynamically decides to output one of four types:
   • Experience: Strategic guidance ($m^*_{t}$) for the Executor.
   • Brief: Updates ($\Delta b_t$) to the runtime context.
   • Hybrid: Both guidance and context updates.
   • NoAction: No output when no memory is currently applicable.
3. Soft-Mem (Dual-Channel Output): To address the multimodal gap, the compiled memory is delivered via two channels:
   • Text Channel: Interpretable strategic guidance.
   • Latent Soft Channel ($m^*_{t, soft}$): Latent soft tokens projected directly into the Executor's embedding space, carrying perceptual and spatial information (e.g., visual layouts) that text cannot express. An orthogonality constraint ensures this channel does not collapse into a redundant copy of the text.

Training Pipeline

MemCompiler is trained end-to-end in two stages:

1. Supervised Fine-Tuning (SFT): Optimizes three objectives: text generation loss, action prediction loss (using the injected soft tokens), and a soft-token loss. The soft-token loss includes a semantic alignment term and a complementarity constraint (penalizing cosine similarity between latent and text representations) to ensure the latent channel encodes distinct perceptual information.
2. Reinforcement Learning (GRPO): Uses Group Relative Policy Optimization to refine the Memory Compiler's compilation decisions based on binary task-success rewards, allowing the model to explore strategies beyond the SFT distribution.

Key Contributions

• Paradigm Shift: Introduces SCMC, replacing static injection with dynamic, step-wise compilation conditioned on a structured Brief State.
• Multimodal Memory Delivery: Proposes Soft-Mem, a dual-channel mechanism that bypasses the text bottleneck to deliver latent perceptual knowledge directly to the executor.
• State-Aware Compilation: Demonstrates that a lightweight compiler can effectively filter and format memory, enabling smaller models to perform complex tasks without the attention dilution caused by static context.

Experimental Results

The method was evaluated on AlfWorld, EmbodiedBench (EB-ALFRED, EB-Habitat), and ScienceWorld across four open-source backbones (Qwen series) and two closed-source backbones (GPT-5.2, Gemini-3-flash).

• Performance Gains: MemCompiler consistently improved performance over the no-memory baseline across all open-source executors. Gains reached +110% on EB-ALFRED and +129% on ScienceWorld.
• Robustness vs. AMMI: While AMMI baselines frequently degraded performance on smaller open-source models (e.g., -83.3% on Qwen-2.5-VL-32B), MemCompiler was the only method to improve every open-source backbone.
• Closing the Gap: MemCompiler enabled mid-size open-source models to approach or match frontier closed-source systems. For instance, Qwen-3.5-27B + MemCompiler matched Gemini-3-flash on Alf World and surpassed GPT-5.2 on EB-Habitat.
• Efficiency: By reducing executor input tokens by 60% and cutting per-step latency from 0.30s to 0.12s, MemCompiler demonstrated that state-aware compilation improves both effectiveness and efficiency.
• Ablation: Removing the Brief State or the Soft-Mem channel caused significant performance drops, confirming that both state-conditioning and latent perceptual delivery are critical.

Significance

The paper concludes that progress in memory for embodied agents depends increasingly on when and how memory is delivered, rather than solely on what is stored. By reframing memory utilization as a compilation process conditioned on the agent's evolving state, MemCompiler addresses the fundamental misalignment of static injection, allowing lightweight embodied agents to leverage accumulated experience effectively without suffering from attention dilution.