EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery

EvoScientist is an evolving multi-agent framework that leverages persistent memory and self-evolution to continuously refine research strategies, thereby outperforming existing state-of-the-art systems in generating novel scientific ideas and executing successful experiments for end-to-end discovery.

The Gist

Imagine you are trying to build a team of robots to discover new scientific truths. In the past, we gave these robots a strict, unchangeable instruction manual. They would follow the steps, make mistakes, and then... start over from scratch the next time, forgetting everything they learned. They were like students who failed a math test, got a bad grade, and then took the exact same test again without studying the solutions.

EvoScientist is a new system that changes the game. It's not just a team of robots; it's a team of robots that learns, remembers, and evolves just like a human research lab would.

Here is how it works, broken down into simple concepts:

1. The Three Specialized Teammates

Instead of one robot trying to do everything, EvoScientist uses three distinct "agents" (specialized AI characters) who work together:

• The Researcher (The Dreamer): This agent is the creative genius. Its job is to come up with wild, new scientific ideas and write the research proposal. Think of it as the architect drawing up blueprints for a new type of house.
• The Engineer (The Builder): This agent takes the Researcher's blueprints and tries to build them. It writes the computer code, runs the experiments, and tries to make the idea work in the real world. If the house blueprint is impossible to build, this agent finds out why.
• The Evolution Manager (The Coach): This is the secret sauce. While the other two are working, the Coach watches everything. It doesn't just watch; it keeps a notebook. When the Researcher has a great idea, the Coach writes it down. When the Engineer fails because of a specific coding error, the Coach writes that down too.

2. The Two "Magic Notebooks" (Persistent Memory)

The biggest problem with old AI scientists was that they had short memories. EvoScientist has two permanent "notebooks" (memories) that never get erased:

• The "Idea Notebook" (Ideation Memory):
  • What's inside: It lists the "winning" ideas that looked promising, but more importantly, it lists the dead ends.
  • The Analogy: Imagine a hiker exploring a forest. If they hit a cliff, they don't just turn around and walk toward the cliff again next time. They mark the spot on their map as "Do Not Go Here." The Idea Notebook does this for science. It remembers, "We tried mixing chemical A and B, and it exploded. Don't do that again."
• The "How-To Notebook" (Experimentation Memory):
  • What's inside: It stores the tricks the Engineer used to fix broken code.
  • The Analogy: If the Engineer is trying to fix a leaky pipe and finally finds that tightening a specific screw works, the Coach writes that down. Next time the Engineer faces a similar leak, they don't have to guess; they just open the notebook and use the proven trick.

3. How They "Evolve"

In the old systems, every new research project started with a blank slate. In EvoScientist, every new project starts with experience.

1. The Researcher looks at the "Idea Notebook" before dreaming up a new idea. They say, "Oh, I see we tried that last time and it failed. Let's try a different angle."
2. The Engineer looks at the "How-To Notebook" before writing code. They say, "I remember the last time we had this error; the fix was to change this one line of code."
3. The Coach updates the notebooks after every single task, making the team smarter for the next one.

The Big Result: Winning the Science Olympics

The paper tested this system by asking it to do real scientific research.

• The Competition: They compared EvoScientist against 7 other top AI systems (some open-source, some expensive commercial ones).
• The Score: EvoScientist won almost every time. It came up with ideas that were more creative, more practical, and clearer than the others.
• The Code: It also wrote code that actually worked much more often because it learned from its past mistakes.
• The Grand Prize: The system was given a challenge to write full scientific papers. It wrote six papers, and all six were accepted by a real scientific conference (ICAIS 2025). Two of them even won awards, including "Best Paper."

Why This Matters

Think of EvoScientist as the difference between a student who memorizes answers and a scientist who learns from experience.

Old AI systems were like students who memorized a textbook but forgot it the moment the test was over. EvoScientist is like a scientist who keeps a lab journal. Every failure makes the journal more valuable, and every success adds a new tool to the toolbox. Over time, the system doesn't just get faster; it gets smarter, avoiding the same mistakes and building on the best strategies, just like human scientists do.

In short: EvoScientist is an AI scientist that doesn't just work; it grows. It remembers its failures so it never has to make them twice, and it remembers its successes so it can use them again and again.

Technical Summary

Here is a detailed technical summary of the paper "EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery."

1. Problem Statement

Current state-of-the-art AI scientist systems (e.g., AI Scientist-v2, Virtual Scientist) typically rely on static, hand-designed pipelines. While they can coordinate specialized roles for idea generation and experiment execution, they suffer from critical limitations:

• Lack of Adaptability: They do not adapt their strategies based on accumulated interaction histories.
• Repetition of Errors: They frequently repeat previously failed experiments or overlook promising research directions because failures are discarded rather than learned from.
• Static Policies: Agent roles and decision strategies remain fixed after deployment, preventing continuous improvement in idea quality or code execution reliability.

The core challenge is to formulate end-to-end scientific discovery as a learning problem where multi-agent systems evolve their idea-generation and code-generation strategies by learning from prior successes and failures.

2. Methodology: The EvoScientist Framework

EvoScientist is a self-evolving multi-agent framework designed to bridge this gap. It decomposes scientific discovery into three specialized agents and utilizes two persistent memory modules to facilitate continuous learning.

A. Agent Architecture

1. Researcher Agent (RA): Responsible for scientific idea generation. It performs an Idea Tree Search involving literature review, proposal generation, and iterative refinement.
2. Engineer Agent (EA): Responsible for experiment implementation and execution. It performs an Experiment Tree Search across four stages: initial implementation, hyperparameter tuning, proposed method execution, and ablation studies.
3. Evolution Manager Agent (EMA): The core "learning" component. It distills insights from the interactions of the RA and EA into reusable knowledge to guide future tasks.

B. Persistent Memory Modules

The system maintains two distinct memory stores that are retrieved by the RA and EA for every new task:

1. Ideation Memory ($M_I$):
   • Content: Summarizes feasible research directions from top-ranked ideas and records unsuccessful directions identified during validation.
   • Function: Helps the RA avoid past pitfalls and refine promising directions.
2. Experimentation Memory ($M_E$):
   • Content: Captures effective data processing and model training strategies derived from code search trajectories and best-performing implementations.
   • Function: Helps the EA increase code execution success rates by reusing successful debugging patterns and implementation strategies.

C. Self-Evolution Mechanisms

The EMA drives the system's evolution through three specific mechanisms:

1. Idea Direction Evolution (IDE): Summarizes promising research directions from top-ranked ideas to update $M_I$.
2. Idea Validation Evolution (IVE): Analyzes execution reports to identify and record failed proposals (e.g., infeasible methods or results worse than baselines) to update $M_I$.
3. Experiment Strategy Evolution (ESE): Distills reusable execution strategies (data processing, model training) from successful code trajectories to update $M_E$.

D. Workflow

1. Retrieval: For a new user goal, the RA and EA retrieve relevant strategies from $M_I$ and $M_E$.
2. Generation & Execution: The RA generates ideas via tree search; the EA executes experiments via tree search.
3. Evolution: After task completion, the EMA analyzes the outcomes, updates the memories, and prepares the system for the next task.

3. Key Contributions

1. Novel Framework: Introduction of EvoScientist, the first multi-agent system specifically designed for end-to-end scientific discovery with persistent memory and self-evolution capabilities.
2. Three Evolution Mechanisms: Formalization of Idea Direction, Idea Validation, and Experiment Strategy evolution, enabling the system to learn from both successful and failed outcomes.
3. Empirical Superiority: Demonstration that EvoScientist outperforms 7 state-of-the-art systems (both open-source and commercial) in idea generation quality and code execution success rates.
4. Real-World Validation: Successful generation of 6 full research papers submitted to ICAIS 2025, all of which were accepted. Notably, one paper won the Best Paper Award, and another received the AI Reviewer's Appraisal Award.

4. Experimental Results

The authors evaluated EvoScientist on three core tasks: Idea Generation, Code Generation, and End-to-End Discovery.

A. Idea Generation Performance

• Metrics: Novelty, Feasibility, Relevance, and Clarity.
• Evaluation: Compared against 7 baselines (e.g., AI Scientist-v2, Virtual Scientist, Hypogenic) using both LLM judges (Gemini-3-flash) and human experts.
• Results: EvoScientist achieved a significant win rate across all dimensions.
  • Automatic Evaluation: Outperformed open-source baselines by an average gap of +29.17% to +93.34% and commercial baselines by +46.00% to +80.83%.
  • Human Evaluation: Achieved a 82.50% win rate on Novelty and 64.17% on Feasibility against strong baselines.

B. Code Generation Performance

• Metric: Execution Success Rate (proportion of trials where code runs successfully).
• Results: Multi-agent evolution significantly improved reliability.
  • The mean execution success rate increased from 34.39% (pre-evolution) to 44.56% (post-evolution).
  • The system showed measurable improvement even in the most difficult stage (Proposed Method), increasing success from 20.33% to 21.57%.

C. End-to-End Scientific Discovery

• Task: Autonomous generation of 6 complete research manuscripts.
• Outcome: All 6 papers were accepted to ICAIS 2025 (Acceptance rate of the track: ~31.7%).
• Awards:
  • Best Paper Award: Adaptive Evidential Meta-Learning with Hyper-Conditioned Priors for Calibrated ECG Personalisation.
  • AI Reviewer's Appraisal Award: Hierarchical Change Signature Analysis: A Framework for Online Discrimination of Incipient Faults...
• Feedback: Reviewers praised the methodological novelty and robust experimental validation but noted that deeper theoretical formalization remains a human-led task.

D. Ablation Studies

Removing the evolution mechanisms resulted in significant performance drops:

• Removing Idea Direction Evolution reduced Novelty and Feasibility.
• Removing Idea Validation Evolution disproportionately harmed Feasibility (63.33% loss rate).
• Removing all evolution caused an 80%+ loss rate in Novelty and Feasibility comparisons.

5. Significance and Impact

• Paradigm Shift: EvoScientist moves AI science from static execution to dynamic learning. It treats interaction history as a first-class resource, allowing the system to "get smarter" over time.
• Reliability: By persistently storing successful code strategies and failed directions, the system drastically reduces the cost of trial-and-error in scientific research.
• Human-AI Collaboration: The system acts as a powerful decision support tool, handling the empirical "what" (generation and validation) while leaving the deep theoretical "why" to human researchers.
• Scalability: The framework demonstrates that multi-agent systems can autonomously produce peer-reviewed, award-winning scientific work, marking a significant step toward fully automated scientific discovery.

Limitations

The current evaluation focuses on computational research (simulations and code execution). Generalization to domains requiring physical experimentation (e.g., materials science, drug discovery) requires integration with laboratory workflows and real-world feedback, which is an open challenge.