🔬 materials science

AGAPI-Agents: An Open-Access Agentic AI Platform for Accelerated Materials Design on AtomGPT.org

This paper introduces AGAPI, an open-access agentic AI platform that integrates multiple large language models with scientific tools to achieve numerical precision in materials property predictions and significantly outperform tool-free models on novel, memorization-resistant datasets by leveraging external databases and simulation workflows.

Original authors: Jaehyung Lee, Justin Ely, Kent Zhang, Akshaya Ajith, Charles Rhys Campbell, Kamal Choudhary

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jaehyung Lee, Justin Ely, Kent Zhang, Akshaya Ajith, Charles Rhys Campbell, Kamal Choudhary

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a complex puzzle about how new materials work, like designing a super-strong metal or a better solar cell. In the past, you had two main options:

The "Super-Genius" Approach: You ask a very smart AI (a Large Language Model) to answer from its memory. It has read millions of books and papers, so it knows a lot. But, it might guess wrong if the specific puzzle piece is rare or if it's trying to remember a fact it learned years ago that turned out to be slightly off.
The "Toolbox" Approach: You give the AI a set of calculators, databases, and simulation machines to use. It doesn't just guess; it looks up the answer or runs a calculation to get the exact number.

This paper introduces AGAPI, a new open-access platform that acts like a smart conductor for a scientific orchestra. It connects the "Super-Genius" AI to a massive toolbox of 18 different scientific tools (like databases of material properties, simulation engines, and protein structure predictors).

Here is the breakdown of what the paper actually found, using simple analogies:

1. The Big Question: Does the Toolbox Help?

The researchers wanted to know: Does giving the AI a toolbox actually make its answers more accurate, or does it just add noise?

They tested this on materials science problems. They found that the answer depends entirely on what the AI is trying to predict and where the data comes from.

2. The "Perfect Copycat" vs. The "Flawed Library"

The team discovered something surprising about why the AI sometimes gets things wrong.

The AI is a Perfect Copycat: When the AI uses the toolbox, it doesn't "hallucinate" or make up numbers. It acts like a perfect messenger. If the toolbox says the answer is "5," the AI says "5."
The Library Has Old Books: The problem isn't the AI or the toolbox; it's the library (the database) the toolbox is pulling from.
- Analogy: Imagine the AI is a student taking a test. The student is given a textbook (the database) to look up answers. If the textbook has a typo or uses an outdated formula, the student will write down the wrong answer, even though they copied it perfectly.
- The Result: For things like "bulk modulus" (how hard a material is) and "bandgap" (how electricity flows), the AI got the numbers exactly right from the database. The reason the answer didn't match real-world experiments was that the database itself was slightly off compared to reality, not because the AI failed.

3. When the Toolbox is a Lifeline (The "New Stuff" Test)

The researchers created a special test with brand new, made-up materials and damaged materials (like a crystal with a missing piece) that the AI had never seen before in its training.

Without Tools: The AI tried to guess from memory. It failed miserably, giving answers that were way off (like guessing the weight of a car by guessing the weight of a bicycle).
With Tools: The AI used the toolbox to calculate the properties of these new materials. It got the answers almost perfectly.
The Lesson: For things the AI has never seen before, the toolbox is indispensable. The AI cannot "memorize" every possible new material, so it must use the tools to calculate the answer.

4. The "Smart Switch" (Adaptive Selection)

The paper suggests that the AI shouldn't blindly use the toolbox for every question. It needs to be smart about when to use it.

Scenario A: If you ask about a very common material (like table salt) that is well-documented in textbooks, the AI's memory might actually be more accurate than the database, because the database might use a slightly outdated calculation method.
Scenario B: If you ask about a weird, new, or broken material, the AI's memory is useless, and it must use the tools.

The authors propose a future version where the AI acts like a smart librarian: it checks, "Do I know this well enough from memory, or do I need to go to the reference section (the database) to get the right answer?"

5. The "Robot Chef" (Autonomous Workflows)

Finally, they showed the AI can do complex, multi-step tasks on its own without a human holding its hand.

Analogy: Instead of just asking "How do I bake a cake?", you can say, "Design a new type of battery, check if it's stable, simulate how it looks under a microscope, and tell me how it would react to heat."
The AI successfully did a 10-step process: it found a material, built a model of it, broke it (to test defects), fixed it, simulated how it would look under X-rays, and calculated its properties—all in one go.

Summary

AGAPI is a platform that lets AI scientists use real tools instead of just guessing.

Good News: The AI is very good at using these tools accurately.
Caveat: If the tools pull data from a database that is slightly inaccurate compared to reality, the AI will give an accurate-but-wrong answer. The error comes from the source data, not the AI.
Best Use: The system is a game-changer for new, unknown materials where the AI has no memory to rely on.

The platform is open for anyone to use, and it proves that when you combine a smart AI with a good set of scientific tools, you can accelerate the discovery of new materials.

Technical Summary: AGAPI-Agents

Problem Statement
The integration of Large Language Models (LLMs) with external scientific tools (Agentic AI) is increasingly proposed to accelerate materials discovery. However, a fundamental gap exists in understanding when tool access actually improves prediction accuracy versus when it introduces errors. Current assumptions that tool augmentation is universally beneficial are complicated by systematic errors in underlying databases (e.g., DFT functional biases), the potential for parametric knowledge in LLMs to outperform specific database entries for well-characterized materials, and the risk of error propagation in multi-step chains. Furthermore, existing frameworks often rely on commercial LLMs with non-deterministic API updates, hindering reproducibility.

Methodology
The authors present AGAPI (AtomGPT.org API), an open-access platform designed to rigorously evaluate and execute agentic workflows in materials science.

Architecture: AGAPI utilizes a three-layer architecture: (1) 50 user-facing web apps, (2) 28 LLM-callable agent tools (Python wrappers), and (3) 18 REST API endpoints. These endpoints integrate diverse resources including JARVIS-DFT, ALIGNN (graph neural networks), ALIGNN-FF and CHIPS-FF (force fields), SlaKoNet (tight-binding), XRD analysis (DiffractGPT), and protein structure prediction (ESMFold).
Agent Design: The system employs a single-agent architecture with a Planner-Executor-Summarizer loop, built on the OpenAI Agents SDK. It uses self-hosted, open-source LLMs (defaulting to GPT-OSS-20B) to ensure reproducibility via version pinning and deterministic sampling.
Evaluation Strategy: The core methodological contribution is a three-evaluation residual decomposition applied to JARVIS-Leaderboard electronic-structure test sets. This separates errors into three components:
1. Agent-vs-Database: Fidelity of the agent pipeline (does the agent correctly retrieve and report the tool's output?).
2. Database-vs-Experiment: Inherited bias of the underlying DFT functional.
3. Agent-vs-Experiment: Total error of the tool-augmented system.
Test Sets: Evaluations include standard JARVIS-Leaderboard sets (bandgap, bulk modulus, superconducting $T_c$ , SLME, dielectric constant) and two "memorization-resistant" sets: DEF-57 (defective supercells) and HYP-60 (hypothetical compositions) designed to fall outside the LLM's training distribution.

Key Results

Pipeline Fidelity: For bulk modulus and bandgap, the agent reproduces JARVIS-DFT entries to numerical precision (MAE = 0.0). This confirms that errors in these properties against experimental references stem entirely from the DFT functional bias in the database, not from agent malfunction.
Property-Specific Performance:
- Bulk Modulus & Dielectric Constant: Tool augmentation improved accuracy against experiment by 27% and 46%, respectively, confirming value when the LLM lacks strong parametric priors or the database is accurate.
- Bandgap, $T_c$ , and SLME: Tool-augmented Mean Absolute Error (MAE) was higher than tool-free predictions for these properties.
  - Bandgap: The error was attributed to the specific DFT functional (OptB88vdW/MBJ) underestimating gaps, not the agent. Switching to HSE06 reduced error significantly without changing the agent.
  - Superconducting $T_c$ : The agent exhibited a non-zero agent-vs-database residual, occasionally replacing retrieved DFT values with parametric knowledge that was closer to experiment for well-known materials (e.g., MgB2, Nb3Sn).
  - SLME: Errors were amplified due to the multiplicative propagation of upstream DFT errors in this derived property.
Memorization-Resistant Evaluation: On DEF-57 and HYP-60 (defective and hypothetical structures), tool-free LLMs failed catastrophically (MAE > 1.25 eV), clustering around textbook host values. In contrast, tool-augmented agents achieved near-perfect accuracy (MAE < 0.003 eV), demonstrating that tools are indispensable when parametric knowledge is unavailable.
Autonomous Workflows: The system successfully executed complex, multi-step workflows, such as a 10-operation semiconductor defect analysis pipeline (searching databases, constructing supercells, relaxing structures, simulating XRD, and calculating properties) and autonomous heterostructure interface construction.

Key Contributions

AGAPI Platform: An open-access, reproducible agentic platform integrating 8 open-source LLMs with 28 tools across materials databases, force fields, and simulation codes.
Quantitative Error Decomposition: The first side-by-side quantitative comparison of tool-augmented vs. tool-free predictions that explicitly separates agent pipeline fidelity from inherited database bias.
Empirical Evidence on Tool Utility: The study demonstrates that tool augmentation is not universally superior; its benefit depends on the specific property, the accuracy of the underlying DFT functional, and the extent of the material's documentation in the LLM's training corpus.
Adaptive Tool Selection Proposal: Based on the residual decomposition, the authors propose a mechanism for adaptive tool selection where agents dynamically decide to invoke tools only when the expected database-vs-experiment residual is smaller than the expected parametric knowledge error.

Significance
The paper claims to provide the first rigorous framework for distinguishing between "agentic malfunction" and "database-functional mismatch." It establishes that for well-characterized materials, an LLM's parametric knowledge can sometimes outperform specific database entries, while for novel or defective structures, tools are essential. By isolating these error sources, AGAPI moves the field beyond the assumption that "more tools equals better accuracy," advocating instead for context-aware, adaptive agentic strategies that weigh the reliability of the tool against the capabilities of the model. The platform and its evaluation methodology are made available to accelerate the development of reliable AI-driven materials design.