🤖 AI

Enhancing Software Engineering Through Closed-Loop Memory Optimization

This paper introduces \ours, a closed-loop framework that optimizes memory utility in software engineering agents by grounding it in validated downstream impact, thereby enabling task-agnostic evaluation and annotation-free optimization that significantly improves success rates, resolve efficiency, and computational cost.

Original authors: Xuehang Guo, Zora Zhiruo Wang, Qingyun Wang, Graham Neubig, Xingyao Wang

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xuehang Guo, Zora Zhiruo Wang, Qingyun Wang, Graham Neubig, Xingyao Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Goldfish" Software Engineer

Imagine you hire a brilliant new software engineer to fix bugs in a massive, complex codebase (a giant library of digital instructions). This engineer is incredibly smart but suffers from a severe case of goldfish memory.

Every time they start a new task:

They forget everything they learned yesterday.
They have to re-read the entire library from scratch to find where things are.
They make the exact same mistakes they made on the previous project.
They reinvent the wheel every single time.

In the world of Artificial Intelligence, these "engineers" are Large Language Models (LLMs) acting as software agents. The paper argues that while these AI agents are powerful, they are fundamentally episodic. They live only in the "now." They don't learn from their past experiences, so they keep stumbling over the same hurdles, wasting time and computing power.

The Old Solution: The "Notebook" That Doesn't Work

Researchers tried giving these AI agents a "notebook" (memory) to write things down. But the old notebooks had a major flaw: Nobody knew what was actually useful to write down.

Some agents wrote down every single keystroke (too much noise).
Some wrote down vague summaries that didn't help (too vague).
There was no rule to say, "Hey, this specific note actually helped me solve the problem later, so keep it," or "This note was useless, throw it away."

Without a way to judge if a memory is actually good, the agents just filled their notebooks with junk, making them slower and more confused.

The New Solution: MemOp (The "Smart Mentor")

The authors introduce MemOp, a system that acts like a strict, data-driven mentor. Instead of just letting the AI write whatever it wants, MemOp uses a "closed-loop" process to ensure the AI only keeps memories that prove they work.

Think of MemOp as a fitness coach for the AI's brain:

The Workout (The Task): The AI tries to fix a software bug.
The Reflection (The Memory): After the task, a special "Memory Model" (a smaller AI) looks at what happened and tries to write a summary (a memory) of what was learned.
The Test (The Crucial Step): This is the magic part. The system doesn't just guess if the memory is good. It tests it.
- It asks: "If we give this memory to the AI to solve a new problem, does it help?"
- If the memory helps the AI solve the new problem faster or better, the memory is accepted.
- If the memory doesn't help (or makes things worse), it is rejected and thrown in the trash.

This creates a Closed Loop: The system only learns from memories that have been proven to improve performance. It's like a chef who only keeps recipes that actually taste good, discarding the ones that ruin the meal.

How It Works in Practice

The paper describes a two-step training process for this "Memory Mentor":

Step 1: Learning the Basics (Supervised Fine-Tuning): The mentor learns how to write a decent summary of what happened, just like a student learning to take notes.
Step 2: Learning from Results (Reinforcement Learning): The mentor gets a score based on whether its notes actually helped the AI in the next round. If the notes helped, the mentor gets a "reward." If they didn't, the mentor gets a "penalty." Over time, the mentor learns to write only the notes that matter.

The Results: Faster, Smarter, Cheaper

The paper tested this system on real-world software engineering tasks (fixing bugs in open-source projects). Here is what happened:

Success Rate: The AI agents with MemOp fixed significantly more bugs (up to 5.25% more) than agents without it.
Efficiency: They solved problems faster, using fewer steps (up to 4.63% more efficient).
Cost: Because they didn't have to re-read everything or make the same mistakes, they used at least 9.79% less computing power.

The Takeaway

MemOp changes the game by treating memory not as a static list of facts, but as a dynamic tool that is constantly optimized.

Instead of an AI that says, "I remember I tried this before, but I don't know if it worked," MemOp gives the AI a system that says, "I remember this specific lesson because I tested it, and I know it helps me win."

It turns the AI from a forgetful goldfish into a seasoned veteran who learns from every single experience, getting smarter and more efficient with every task it tackles.

Technical Summary: Enhancing Software Engineering Through Closed-Loop Memory Optimization

Problem Statement

Large Language Model (LLM) agents have demonstrated significant capabilities in navigating complex codebases and resolving real-world software engineering (SE) issues. However, current SE agents remain fundamentally episodic. They fail to retain, refine, or reuse experiences across tasks, forcing them to reconstruct context from scratch for every new problem. This leads to the repetition of ineffective strategies, the rediscovery of codebase structures, and the recurrence of similar mistakes.

While existing approaches attempt to augment agents with memory, they suffer from a critical lack of a principled, task-agnostic definition of memory utility. Current methods rely on task-specific heuristics, complex architectures, or human expertise to determine what to remember. Without a rigorous way to quantify whether a specific memory actually improves downstream performance, it remains unclear how to distinguish beneficial memories from noise, how to optimize memory generation, or how to generalize memory mechanisms across different agents and settings.

Methodology: MemOp

To address these limitations, the authors introduce MemOp, a closed-loop framework for memory augmentation in SE agents. MemOp grounds memory utility in validated downstream impact, treating memory not as a static prompt artifact but as an optimizable component.

Core Components

Principled Memory Utility:
MemOp defines memory utility strictly through causal impact on downstream tasks. A candidate memory $M_j$ is considered useful if and only if it measurably improves the SE agent's performance on specific metrics compared to a no-memory baseline. This definition serves a dual role:
- Evaluation Benchmark: A task-agnostic standard for assessing memory quality.
- Optimization Signal: An annotation-free signal to drive memory learning.
The acceptance criterion (Eq. 1) requires that a memory must not degrade any of the $N_Q=10$ performance metrics and must improve at least one.
Closed-Loop Memory Optimization:
MemOp employs a two-stage training process to learn a memory model $M_\theta$ that distills agent experiences into reusable knowledge:
- Trajectory-Based Rejection Sampling: The system generates multiple memory candidates from SE trajectories. Each candidate is evaluated by running the SE agent with the memory versus without it. Candidates that fail the utility criterion are rejected; those that pass form the training dataset.
- Stage I (Supervised Finetuning - SFT): The memory model is trained on accepted memory samples to learn the fundamental ability to generate performance-enhancing memories from SE trajectories.
- Stage II (Reinforcement Learning - RL): The model is further optimized using a preference-based RL approach. For a given task, the model generates multiple candidates; the "ground truth" (accepted) memory is compared against rejected candidates. The reward function is directly grounded in the performance gain ( $\Delta$ ) of the generated memory, encouraging the model to produce memories that causally improve agent outcomes.
Memory Representation:
The framework utilizes a structured, markdown-based memory format (Memory.md) that integrates key knowledge, repository structures, and actionable insights, which preliminary studies found to be superior to unstructured strings or other formats.

Key Contributions

MemOp Framework: A closed-loop system that transforms memory from a hand-designed heuristic into a learnable, optimizable component grounded in empirical SE performance.
Utility-Grounded Definition: The introduction of a formal, task-agnostic definition of memory utility based on validated downstream impact, enabling rigorous evaluation and annotation-free optimization.
Two-Stage Optimization Strategy: A novel training pipeline combining Supervised Finetuning and Reinforcement Learning with Preference Optimization, where the reward signal is derived directly from the agent's performance improvement.
Comprehensive Evaluation: Systematic evaluation across single-episode (immediate reuse) and cross-episode (progressive evolution) regimes, demonstrating the framework's adaptability and robustness.

Experimental Results

The authors evaluated MemOp on the SWE-Bench-Verified benchmark using various LLM backbones (e.g., Devstral-Small, Qwen3-Coder) and memory models (e.g., Qwen2.5, DeepSeek-R1).

Performance Gains: MemOp consistently improved SE agents across both single-episode and cross-episode settings.
- Success Rate (SR): Achieved absolute gains of up to ↑5.25% in single-episode and ↑3.00% in cross-episode scenarios.
- Resolve Efficiency: Improved by up to ↑4.63%, indicating faster problem resolution.
- Localization Accuracy: Significant improvements in identifying target files and functions.
Computational Efficiency: Despite the overhead of memory generation, MemOp reduced computational costs by ≥9.79% relative to baselines, primarily by reducing the number of iterations required to resolve tasks.
Generalizability: The framework demonstrated robust performance across different SE agents, memory model backbones, and diverse repository structures.
Ablation Studies:
- Training Stages: Both SFT and RL stages independently contributed to performance, with the combination yielding superior results.
- Granularity: Episode-level memory evolution was found to be significantly more efficient and effective than action-level evolution.
- Algorithm Agnosticism: MemOp yielded consistent gains when integrated with various RL algorithms (GRPO, DAPO, GSPO).

Significance and Claims

The paper claims that MemOp addresses a fundamental gap in agentic software engineering: the lack of a principled mechanism to learn what to remember. By anchoring memory utility in empirical downstream impact, MemOp:

Eliminates the need for costly human annotation or task-specific heuristics in memory design.
Transforms memory quality from an ill-defined property into a measurable, optimizable quantity.
Enables adaptive, continually improving agents that can distill lessons from past trajectories to solve future, diverse software engineering challenges more effectively and efficiently.

The authors position MemOp not merely as a performance booster, but as a step toward building generalizable, adaptive agents where memory utility serves as a first-class design principle. They acknowledge limitations, such as the current focus on Python repositories and the dependency on the intrinsic problem-solving capabilities of the underlying SE agent, suggesting cross-benchmark validation and tighter integration with agent training as future directions.