💬 NLP

SciDER: Scientific Data-centric End-to-end Researcher

SciDER is a novel, data-centric end-to-end system that leverages specialized collaborative agents, self-evolving memory, and critic-led feedback to autonomously process raw scientific data, generate hypotheses, and execute experiments, thereby outperforming existing general-purpose agents in scientific discovery.

Original authors: Ke Lin, Yilin Lu, Shreyas Bhat, Xuehang Guo, Junier Oliva, Qingyun Wang

Published 2026-03-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ke Lin, Yilin Lu, Shreyas Bhat, Xuehang Guo, Junier Oliva, Qingyun Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a brilliant scientist with a great idea, but you don't have time to write code, clean up messy data, or run thousands of experiments yourself. You need a partner who can do all that grunt work while you focus on the big picture.

Enter SciDER (Scientific Data-centric End-to-end Researcher). Think of SciDER not just as a chatbot, but as a super-charged, self-driving research lab that you can hire with a single click.

Here is how it works, broken down into simple concepts and analogies:

1. The Problem: The "Manual Labor" of Science

Traditionally, doing science is like building a house. You (the human) have to:

Draw the blueprints (Ideas).
Go to the junkyard to sort through piles of scrap metal and wood (Raw Data).
Write the instructions for the construction crew (Coding).
Build the house (Experiments).
Fix the mistakes when the roof leaks (Debugging).

Most current AI tools are like smart assistants who can only draw blueprints. They can suggest ideas, but when it comes to actually sorting the scrap metal (raw data) or building the house, they get stuck. They struggle with messy, real-world data that doesn't come in neat, pre-packaged boxes.

2. The Solution: A Team of Specialized Robots

SciDER is different because it acts like a full construction crew rather than just an architect. It breaks the work down into four specialized "agents" (robots) that talk to each other:

The Dreamer (Ideation Agent): This robot reads millions of scientific papers to find gaps in knowledge. It says, "Hey, what if we try this?" and writes a plan.
The Janitor & Analyst (Data Agent): This is the magic part. When you upload messy data (like a spreadsheet with typos or weird image files), this robot cleans it, organizes it, and figures out what it actually means. It turns a messy pile of scrap into a neat pile of bricks.
The Builder (Experiment Agent): Once the plan is set and the bricks are ready, this robot writes the code and runs the experiments. It's like a robot arm that builds the house.
The Inspector (Critic Agent): This robot is the boss. It checks the Dreamer's ideas, the Janitor's cleaning, and the Builder's work. If the roof is leaking, the Inspector yells, "Fix it!" and sends the Builder back to work.

3. The Secret Sauce: The "Self-Evolving Brain"

Most AI forgets what it learned yesterday. SciDER has a Self-Evolving Memory.

Imagine a student who takes notes.

Short-term memory: "I just solved this math problem."
Long-term memory: "I know how to solve this type of math problem."
Project memory: "I know exactly how this specific house was built."

SciDER saves every lesson it learns. If it struggles to clean a specific type of data today, it remembers that struggle. Next time it sees similar data, it says, "Oh, I've seen this before! I know the trick." It gets smarter the more it works, without needing a human to retrain it.

4. How It Performs: The "Olympics" of Science

The researchers tested SciDER in three different "Olympic Games":

The Idea Game: Can it come up with new, creative research ideas? Yes. It beat all other AI models, coming up with ideas that were twice as novel and feasible.
The Machine Learning Game: Can it build AI models from scratch? Yes. It won more "Gold Medals" (solved more problems) than specialized AI competitors.
The Hard Science Game: Can it solve complex physics and chemistry problems? Yes. It outperformed top-tier models like GPT-5 in solving difficult, multi-step scientific puzzles.

5. A Real-Life Example

Imagine an astronomer who knows stars but doesn't know how to code. They want to find new planets in a noisy dataset.

Old Way: The astronomer spends months learning Python, cleaning the data, and debugging code.
SciDER Way: The astronomer types: "Use this star data to find new planets."
- SciDER's Dreamer figures out the best method.
- The Janitor cleans the noisy star data.
- The Builder writes the code and runs the simulation.
- The Inspector checks the results.
- Result: In minutes, the astronomer has a working program and a report showing a 98% success rate in finding planets.

Why This Matters

SciDER is like giving every scientist a personal research assistant that never sleeps, never gets tired, and gets smarter every day. It removes the boring, technical barriers (like coding and data cleaning) so humans can focus on the creative and discovery parts of science.

It's not about replacing scientists; it's about giving them a superpower to do more science, faster, and with fewer mistakes.

1. Problem Statement

Current Large Language Model (LLM) based agents for scientific discovery face two critical limitations that prevent fully autonomous research:

Limited Adaptability to Raw Data: Most existing systems are designed for clean, public machine learning datasets. They struggle to autonomously parse, clean, and analyze diverse, raw experimental data collected from real-world scientific instruments.
The Domain Gap: General-purpose assistants often fail in specialized scientific domains (e.g., physics, biology) where abstract hypotheses must be translated into precise, executable code that adheres to specific data formats and domain constraints.
Lack of Continuous Learning: Existing frameworks often lack mechanisms to retain and utilize knowledge from previous tasks, leading to repetitive errors and an inability to evolve over time.

2. Methodology: The SciDER System

SciDER is a modular, data-centric, end-to-end system designed to automate the entire scientific research lifecycle. It is built on LangGraph and litellm, featuring a multi-agent architecture and a novel memory system.

A. Multi-Agent Architecture

The system orchestrates four specialized agents that collaborate in a closed loop:

Ideation Agent:
- Performs automated literature reviews (scraping arXiv, Semantic Scholar, PubMed).
- Generates hypotheses and experimental outlines tailored to the specific dataset.
- Includes a "LLM-as-Judge" module to evaluate proposals on Uniqueness, Innovation, Gap Addressing, and Impact.
Data Analysis Agent:
- Core Innovation: Unlike static scripts, this agent autonomously parses raw user-uploaded data (CSV, JSON, Parquet, custom images).
- It structures data, assesses quality/semantics, and generates a structured analysis report ( $R$ ) that serves as the ground truth for subsequent code generation.
Experimentation Agent:
- Coding Phase: Generates executable scripts ( $C$ ) based on the data report ( $R$ ) and hypothesis ( $I$ ). It uses external frameworks (OpenHands, Claude Code) and iteratively fixes syntax/type errors.
- Execution Phase: Runs the code, monitors for timeouts or failures (e.g., high loss, frozen screens), and provides feedback to the coding agent for revision.
Critic Agent:
- Reviews outputs from all stages for accuracy, completeness, and bias.
- Identifies logical inconsistencies and proposes improvements, triggering a self-refinement loop ( $I_{n+1} \leftarrow \text{Ideation}(F_n, \dots)$ ).

B. Self-Evolving Memory Mechanism

SciDER introduces a dynamic memory system adapted from ReasoningBank, categorized into:

Short-term Memory: Context for the current session.
Long-term Memory: Divided into Task-specific (general domain guidance) and Project-specific (results and insights from the current project).
Mechanism: Uses Retrieval-Augmented Generation (RAG) to retrieve relevant reasoning chunks via semantic vector or keyword search. New insights are summarized and integrated back into the memory bank, enabling "test-time learning" where the system improves proficiency as it accumulates experience.

C. Workflow

The system accepts text, raw data, code, or prior papers. It executes a closed-loop cycle:

Input: Dataset ( $D$ ) + Research Query ( $Q$ ).
Process: Ideation $\to$ Data Analysis $\to$ Experimentation (Code Gen + Exec) $\to$ Critique.
Output: Hypotheses, Analysis Reports, Executable Code, and Final Results.

3. Key Contributions

SciDER System: A modular Python package that automates the full research lifecycle (ideation, data analysis, experimentation) using specialized agents and a self-evolving memory mechanism.
Data-Centric Paradigm: A novel approach where code generation is strictly grounded in autonomous analysis of raw experimental data, bridging the gap between abstract reasoning and executable synthesis in interdisciplinary fields.
Empirical Superiority: Extensive evaluation demonstrating that SciDER significantly outperforms state-of-the-art (SOTA) models and general-purpose agents in generating ideas, conducting experiments, and solving domain-specific coding problems.

4. Evaluation Results

SciDER was evaluated on three benchmarks:

AI-Idea-Bench (Idea Generation):
- Metric: Idea-to-Idea Matching, Novelty, Feasibility.
- Result: SciDER achieved a Novelty score of 47.06 (Motivation) and 44.52 (Experiment), nearly doubling the performance of SOTA models like AI-Scientist (25.03) and AI-Researcher (24.92). It also showed superior feasibility (24.0).
MLE-Bench (General ML Experimentation):
- Metric: % Gold Medal and % Any Medal on 75 Kaggle competitions.
- Result: SciDER achieved a 36.40% Gold Medal rate, outperforming specialized agents like AIRA (28.64%) and AIDE (9.40%), demonstrating strong scalability through its iterative trial-critic-revision cycles.
SciCode (Domain-Specific Coding):
- Metric: Solve rate on 16 subfields (Physics, Chemistry, Biology, etc.).
- Result: SciDER achieved a 15.38% solve rate on main problems and 42.71% on sub-problems, outperforming GPT-5 (13.85% / 38.26%) and other top models. This highlights its ability to handle complex, multi-step scientific reasoning.
Human Evaluation:
- 13 domain experts rated the system's "helpfulness." SciDER received a mean score of 4.846/5.0, praised for reducing workflow friction and ensuring code executability.

5. Significance and Impact

Lowering Barriers: SciDER enables researchers with limited coding skills (e.g., the astrophysicist case study) to build high-performance ML pipelines (achieving 98% F1 score in the demo) with minimal intervention.
Scalability: The self-evolving memory allows the system to learn from past failures and successes, making it more robust over time compared to static agents.
Accessibility: Distributed as a modular Python package with a lightweight web interface, SciDER aims to be accessible to all researchers, promoting the adoption of autonomous research agents.
Ethical Awareness: The authors acknowledge risks such as hallucinations and data privacy, recommending that SciDER serve as a collaborative assistant where human experts validate all AI-generated findings.

In summary, SciDER represents a significant leap forward in autonomous scientific discovery by shifting the focus from generic code generation to data-grounded, iterative, and memory-enhanced research workflows.