🤖 AI

MARS: Modular Agent with Reflective Search for Automated AI Research

MARS is a novel framework for autonomous AI research that overcomes the bottlenecks of complex machine learning engineering by integrating budget-aware Monte Carlo Tree Search planning, a modular design-decompose-implement pipeline, and comparative reflective memory to achieve state-of-the-art performance on MLE-Bench.

Original authors: Jiefeng Chen, Bhavana Dalvi Mishra, Jaehyun Nam, Rui Meng, Tomas Pfister, Jinsung Yoon

Published 2026-05-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiefeng Chen, Bhavana Dalvi Mishra, Jaehyun Nam, Rui Meng, Tomas Pfister, Jinsung Yoon

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to win a high-stakes cooking competition, but you have a very strict rule: you only have 24 hours to cook, and every minute you spend chopping onions costs you money.

Most computer programs trying to solve this problem are like amateur chefs who just start chopping everything at once. They write one giant, messy recipe (a "monolithic script") that tries to do everything in one go. If they burn the soup, they throw away the whole pot and start over from scratch. They don't care how long it takes to boil the water; they just want the soup to taste good. In the real world of AI research, this is a disaster because training AI models is like boiling a giant pot of soup—it takes a lot of time and electricity.

The paper introduces MARS (Modular Agent with Reflective Search), which is like a Master Chef with a Team of Specialists and a Smart Notebook.

Here is how MARS works, broken down into three simple parts:

1. The "Budget-Aware" Search (The Smart Planner)

Imagine you are playing a game of "Guess the Best Recipe," but you have a limited budget for ingredients and time.

Old Way: You try every recipe you can think of, even if one takes 10 hours to cook. If it tastes slightly better, you pick it, even if you run out of time to cook anything else.
MARS Way: MARS uses a strategy called Budget-Aware MCTS. Think of this as a smart planner who looks at the clock. If Recipe A takes 1 hour and tastes "okay," and Recipe B takes 10 hours and tastes slightly better, MARS picks Recipe A. It knows that wasting 9 hours on a tiny improvement is a bad deal. It constantly asks, "Is this improvement worth the extra time and money?"

2. Modular Construction (The Assembly Line)

Instead of writing one giant 50-page recipe, MARS breaks the cooking process into small, separate stations.

The Old Way: One chef writes a single, confusing script. If the sauce is wrong, they have to rewrite the whole 50-page document to fix it.
MARS Way: MARS uses a "Design-Decompose-Implement" pipeline. It assigns different "agents" (specialist chefs) to specific tasks:
- Agent A handles the vegetables (data).
- Agent B handles the spices (model architecture).
- Agent C handles the cooking (training).
  If the sauce is too salty, MARS only changes the "Spice Agent's" instructions. It doesn't touch the vegetables or the meat. This makes the code easier to fix, easier to test, and much less likely to crash the whole system.

3. Comparative Reflective Memory (The "Aha!" Notebook)

This is the most magical part. In a normal kitchen, if a dish fails, you might just say, "That didn't work," and move on.

The Old Way: AI agents often forget why a dish failed. They might try the same mistake again because they didn't learn the specific cause.
MARS Way: MARS keeps a "Comparative Reflective Memory." When a new dish turns out better than the old one, MARS doesn't just say "Good job." It acts like a detective. It compares the exact difference between the old recipe and the new one.
- Example: "Ah! The new dish is better because we added a pinch of salt and cooked it for 2 minutes longer. The salt was the key, not the time."
- It writes this down as a "Lesson." Later, if it tries a completely different dish, it can say, "Hey, I learned that salt helps with this type of problem," and apply that lesson to a totally different branch of the search.
- The paper found that 63% of the lessons MARS used came from these "cross-branch transfers"—meaning it learned from one type of problem and successfully applied it to a different one. This is the "Aha!" moment.

The Result

The researchers tested MARS on MLE-Bench, which is like a giant, difficult cooking competition with 75 different challenges (from analyzing text to recognizing images).

The Competition: Other AI agents (like AIDE or AIRA) tried to solve these problems but often got stuck, wrote messy code, or wasted time on expensive experiments that didn't pay off.
The Winner: MARS won more medals (Gold, Silver, Bronze) than any other open-source system. It didn't just get lucky; it was more efficient. It found better solutions faster by not wasting time on expensive, low-value experiments and by learning from its mistakes in a structured way.

In summary: MARS is an AI researcher that doesn't just "try random things." It plans its time carefully, breaks big problems into small, manageable pieces, and keeps a detailed notebook of exactly why things worked or failed, allowing it to learn and improve much faster than its competitors.

Technical Summary: MARS (Modular Agent with Reflective Search) for Automated AI Research

1. Problem Statement

The automation of AI research faces a critical bottleneck in the execution of complex Machine Learning Engineering (MLE) tasks. Unlike general software engineering, where correctness is often binary and verification is computationally cheap, MLE is a probabilistic, resource-intensive endeavor. It requires navigating a landscape defined by:

Computationally Expensive Evaluation: Model training and data processing consume vast resources, making "trial and error" costly.
Opaque Performance Attribution: When an experiment yields better results, isolating the specific causal factors (credit assignment) is difficult.
Architectural Complexity: Research repositories require seamless interaction between data loading, model architecture, and training loops.

Current Large Language Model (LLM) agents struggle in this domain. They typically generate monolithic scripts that ignore execution costs, lack the modularity required for rigorous engineering, and fail to learn effectively from past trials due to an inability to solve the credit assignment problem.

2. Methodology: The MARS Framework

MARS (Modular Agent with Reflective Search) reformulates the research process as a search for an optimal software repository. It is built upon three core pillars designed to address the specific constraints of autonomous scientific discovery:

A. Budget-Aware Planning (Resource-Aware MCTS)

To address the high cost of evaluation, MARS employs a cost-constrained Monte Carlo Tree Search (MCTS). Unlike standard search algorithms that optimize solely for task performance, MARS integrates an Efficiency-Guided Reward Function directly into the search tree.

Mechanism: The reward function $R(v)$ balances performance gains ( $G(v)$ ) with execution latency ( $t(v)$ ), penalizing solutions that exceed time budgets or are computationally inefficient.
Action Space: The search tree utilizes three operators:
1. Drafting: Generating a new solution from scratch.
2. Improvement: Modifying valid nodes to maximize objectives.
3. Debugging: Resolving runtime errors in failed nodes.
Outcome: This allows the agent to prioritize efficient solutions (e.g., a 1-hour training run over a 4-hour run with comparable performance), optimizing the discovery rate within a fixed budget.

B. Modular Construction

MARS replaces fragile, monolithic scripting with a structured, repository-level software architecture.

Pipeline: It employs a "Design-Decompose-Implement" workflow involving specialized agents:
1. Idea Generation: Articulates a comprehensive natural language plan.
2. Module Decomposition: Parses the plan into logical, independent functional modules.
3. Component Implementation: Sequentially implements and validates modules.
Diff-Based Refinement: Code modifications are structured in a standardized diff format, enabling atomic, multi-file updates. This approach bypasses token limits, reduces context noise, enables code caching/reuse, and significantly improves testability by localizing debugging to specific files.

C. Comparative Reflective Memory

To resolve the credit assignment problem, MARS introduces a Comparative Reflective Memory mechanism that distills high-signal, causal insights from the exploration process.

Lesson Distillation: Instead of merely summarizing logs, the system performs a comparative analysis between the current solution and the best-known solution. It isolates specific algorithmic changes driving performance shifts.
Lesson Types:
- Solution Improvement Lessons: Derived from valid solutions, identifying causal changes and generalized rules.
- Debugging Lessons: Derived from failed executions, explaining failure logic and providing guidelines to preempt similar errors.
Utilization: The agent retains the most recent lessons in memory and explicitly cites them when applying them to new solutions, facilitating cross-branch transfer of insights.

3. Key Contributions

Framework Introduction: The proposal of MARS, a framework explicitly optimized for autonomous AI research by combining Budget-Aware MCTS, a modular implementation pipeline, and Comparative Reflective Memory.
State-of-the-Art Performance: Extensive evaluation on the MLE-Bench benchmark demonstrates that MARS achieves state-of-the-art performance among open-source frameworks under comparable settings, maintaining competitiveness with global leaderboard leaders.
Qualitative "Aha!" Moments: The system exhibits qualitative breakthroughs where 63% of utilized lessons originate from cross-branch transfer, demonstrating the agent's ability to effectively generalize insights across different search paths.
Open Release: The authors release prompts, generated code, and trajectories to facilitate future research.

4. Experimental Results

The evaluation was conducted on MLE-Bench, comprising 75 challenging Kaggle competitions across NLP, computer vision, and tabular data.

Performance Metrics: MARS achieved the highest Gold Medal rate (31.1%) among all reported agents in the controlled evaluation.
Competitiveness:
- Using Gemini-3-Pro-Preview, MARS achieved an Any Medal rate of 56.0% and an Above Median rate of 65.8%.
- The scaled variant, MARS+ (utilizing 2x H100 GPUs), achieved an Any Medal rate of 62.7% and a Gold Medal rate of 33.8%, outperforming strong competitors like ML-Master 2.0.
Ablation Studies:
- Modular Decomposition: Removing this component significantly degraded success, validating its role in reducing context noise and improving testability.
- Memory Mechanisms: Complete removal of memory caused a drastic performance drop. Comparative distillation was shown to be essential for isolating causal factors compared to simple empirical analysis.
- Budget-Aware MCTS: The proposed search strategy consistently outperformed Vanilla MCTS and Greedy Search, confirming that penalizing execution time improves the discovery of optimal solutions.
Cost Efficiency: A 4-hour version of MARS achieved a higher Any Medal rate (28.4%) while spending significantly less ($9.6) than 24-hour baselines, demonstrating that performance gains stem from architectural efficiency rather than merely increased resource allocation.

5. Significance and Claims

The paper claims that MARS addresses the limitations of current autonomous agents in long-horizon AI research by treating research as a rigorous, repository-level engineering challenge. By integrating resource-aware planning, modular construction, and comparative reflective memory, MARS effectively resolves the credit assignment problem while balancing exploration with computational efficiency.

The authors assert that this structured approach, which mimics the strategic foresight of human engineers, enables state-of-the-art performance in complex Machine Learning Engineering tasks. The system's ability to experience "Aha!" moments through cross-branch lesson transfer suggests a move toward agents that can not only execute code but also learn and generalize scientific insights autonomously.

The paper acknowledges limitations, noting that evaluation is currently restricted to MLE-Bench and that the computational cost of iterative search and distillation remains a barrier. Future work is proposed to extend the framework to broader scientific domains and optimize economic viability through advanced context caching and early stopping.