← Latest papers
🔬 materials science

ORCHID-SAGE: Fully Reproducible CALPHAD Assessment from Evidence to Phase Diagram

The paper introduces ORCHID-SAGE, a standard that encodes expert judgment into formal protocols to enable the fully reproducible, automated, and updatable assessment of CALPHAD thermodynamic databases, as demonstrated by successfully regenerating a Cr-Ni database using AI agents and public data.

Original author: Raymundo Arróyave

Published 2026-11-13
📖 8 min read🧠 Deep dive

Original author: Raymundo Arróyave

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer

To design a new metal alloy that can withstand the heat of a jet engine or the corrosion of a chemical plant, scientists must first know which combinations of elements will remain stable and which will fall apart. This knowledge comes from a method called CALPHAD, which builds a thermodynamic database by fitting mathematical models to experimental data. For decades, creating these databases has been a highly manual process. An expert gathers data from scattered research papers, makes a series of subjective choices about which data to trust, and adjusts the model parameters by hand until the predicted phase diagram matches the observations. The final result is a set of numbers that describes the material, but the path taken to reach those numbers—the specific decisions, the discarded data, and the reasoning behind every adjustment—is rarely recorded. Once the database is published, it is effectively a black box: others can use it, but they cannot rebuild it, update it with new data, or verify exactly how it was made.

This lack of transparency creates a bottleneck for the future of materials science, particularly for self-driving laboratories where artificial intelligence agents are beginning to design and test new materials. If an AI cannot understand how a database was constructed, it cannot reliably update it when new experiments reveal errors or gaps. In a new study, researchers have introduced a system called ORCHID-SAGE to solve this problem. They have created a standard for recording every single step of the assessment process, turning the expert's intuition into a formal, machine-readable protocol. By applying this system to the chromium-nickel alloy, a material critical for stainless steel, the team demonstrated that a complete thermodynamic database can be regenerated from scratch using only its recorded history. The result is a "living" database that can be continuously refined by AI agents as new evidence emerges, while keeping the human expert's judgment visible and accountable on the record.

The core innovation of ORCHID-SAGE is the shift from publishing a static set of results to publishing a dynamic recipe for how those results were achieved. The researchers broke the assessment process down into three distinct layers of documentation. The first layer is a general protocol, a set of rules that applies to any alloy system, defining how data should be weighed and how models should be tested. The second layer is a specific protocol, which applies those rules to a particular material, such as chromium-nickel, and records every decision made along the way, noting whether a human or an algorithm made the choice and why. The third layer consists of run cards, which are detailed logs of every computer execution, capturing the exact software versions, inputs, and outputs used at each step. Together, these documents form a complete record that can be read by both humans and machines.

To test this system, the researchers focused on the chromium-nickel system, a classic case where experimental data often conflicts, making it a difficult test for any assessment method. They began by using AI agents to scan the scientific literature and extract data points, converting them into a standardized format that includes not just the values but also their uncertainties and links to the original sources. They then used a machine-learned potential to generate a rough starting model of the alloy's behavior. This initial model was far from perfect, but it provided a solid foundation. The team then refined this model against the experimental data using a gradient-based optimization method, which adjusts the model parameters to minimize the error between the prediction and the observation. Crucially, every step of this refinement was logged. The researchers did not just record the final numbers; they recorded the sequence of decisions, such as which phases to include, how to handle conflicting data, and how to weight different types of measurements.

The true test of the system came when the researchers attempted to regenerate the database without any help from the original authors. They handed the specific protocol and the recorded data to two different entities: a computer script running on a completely different machine with a different processor, and an AI agent that had no prior knowledge of the assessment. Both were given only the record of the process and the tools to execute it. The script successfully rebuilt the database, producing a phase diagram that matched the original predictions within a tiny margin of error. Even more impressively, the AI agent, working through a server that allowed it to read the protocol and execute the steps, also regenerated the database. It took the agent about two and a half hours and cost less than two dollars in computing resources to complete the task. The final predictions from the regenerated database were identical to the original, confirming that the record contained everything necessary to reproduce the work.

One of the most significant findings was the discovery of a "flat direction" in the mathematical landscape of the model. In the final step of the fitting process, the researchers found that two different sets of parameters produced almost identical results. In a traditional assessment, an expert might pick one set arbitrarily and publish it as the definitive answer. However, because the ORCHID-SAGE record captured the entire process, including the existence of these two equally valid solutions, the system could report that these specific parameters were undetermined by the data. This level of honesty about uncertainty is something that is often lost in standard publications, where the final numbers are presented with a false sense of precision. The record showed that while the parameters varied, the physical predictions—such as the temperature at which the alloy melts or the composition of its phases—remained stable and accurate.

The study also highlighted the physical limitations of the current model. By comparing the regenerated database against the experimental evidence, the researchers identified a specific region in the chromium-rich part of the alloy where the model and the data disagreed. The model predicted that a certain solid phase should be stable at higher chromium contents than what was observed in experiments. Because the entire process was recorded, this discrepancy was not hidden; it was flagged as a specific area where new measurements would be most useful. This is the essence of a living database: it does not just store knowledge, it identifies the boundaries of that knowledge and points to where it needs to be expanded.

The implications of this work extend beyond a single alloy. The researchers demonstrated that the expert knowledge required to build a thermodynamic database can be encoded into a protocol that an AI agent can follow. This does not mean that human experts are no longer needed. The system still relies on humans to make the high-level judgments, such as selecting which phases to include or deciding how to handle conflicting evidence. However, by recording these decisions explicitly, the system ensures that the human contribution is transparent and reproducible. The AI agent acts as a diligent assistant, executing the steps and checking the results, while the human expert remains the guide, making the critical choices that shape the model.

This approach transforms the thermodynamic database from a static artifact into a dynamic tool that evolves with new evidence. In a self-driving laboratory, an AI agent could use this database to predict the best conditions for synthesizing a new material. If the experiment fails, the new data can be fed back into the system, and the protocol can be run again to update the database. Because the entire history is recorded, the new version of the database remains traceable to the original evidence and the decisions that led to it. The researchers showed that this cycle is possible, with the chromium-nickel database being successfully regenerated and refined using only the recorded protocol.

The cost of this reproducibility is the effort required to create the record. Assembling the evidence, documenting every decision, and verifying the hashes of the files takes time and care. However, the researchers found that once the record exists, the cost of using it is minimal. The regeneration of the chromium-nickel database by an AI agent took less than three hours and cost a negligible amount of money. In exchange for this initial investment, the scientific community gains a database that can be verified, corrected, and updated without needing to reconstruct it from scratch. The database is no longer fixed at the moment of publication; it becomes a living resource that grows more accurate as new data arrives.

The study concludes that the reproducibility of a CALPHAD assessment is achievable, provided that the decisions of the assessor are recorded with the same care as the data itself. The ORCHID-SAGE system provides a framework for this, ensuring that the wisdom of the expert is preserved in a format that machines can understand and execute. This opens the door to a future where thermodynamic databases are continuously refined by AI agents, guided by human experts, and always open to inspection. The chromium-nickel assessment serves as a proof of concept, showing that the path from evidence to phase diagram can be mapped, recorded, and retraced, turning a once opaque process into a transparent and evolving scientific tool.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →