Predicting Scale-Up of Metal-Organic Framework Syntheses with Large Language Models

This paper introduces ESU-MOF, a dataset and positive-unlabeled learning framework that fine-tunes large language models to predict the scalability potential of Metal-Organic Framework syntheses with 91.4% accuracy, thereby accelerating industrial deployment by addressing fragmented scale-up knowledge.

The Gist

Imagine you are a master chef who has just invented a delicious new recipe for a cake. You made a perfect, tiny slice in your home kitchen. It tastes amazing! But now, a massive bakery wants to buy your recipe to bake thousands of cakes a day.

Here's the problem: Just because a cake tastes good in a tiny pan doesn't mean it will work in a giant industrial oven. Sometimes, the heat distribution is wrong, the ingredients clump together, or the mixing takes too long. In the world of chemistry, this is the difference between making a few crystals in a lab and making tons of them for a factory.

For years, scientists have discovered thousands of new "chemical cakes" called Metal-Organic Frameworks (MOFs). These are like super-sponges made of metal and organic links, used for things like cleaning water or storing hydrogen fuel. But most of these discoveries stay stuck in the lab because nobody knows if they can be made on a huge scale without breaking the bank or the machine.

The knowledge about how to scale these up is scattered like puzzle pieces across thousands of different research papers. It's a mess.

The Solution: A "Super-Reader" Robot Chef

The authors of this paper built a smart computer system (using a Large Language Model, or LLM) to solve this puzzle. Think of this AI not just as a search engine, but as a super-reading robot chef that has read every single chemistry paper ever written.

Here is how they trained this robot, step-by-step:

1. Gathering the Clues (The Dataset)

The robot needed to learn what a "scalable" recipe looks like.

• The "Yes" Pile: The team found papers that explicitly said, "We made this in kilograms!" or "We scaled this up to a pilot plant!" These are the Strong Positives.
• The "Maybe" Pile: They also found papers that described making a tiny amount of a chemical that later turned out to be scalable. These are the Auxiliary Positives.
• The "Unknown" Pile: The vast majority of papers just say, "We made a tiny bit of this." We don't know if it could be scaled up or if it's impossible. The robot treats these as Unlabeled.

2. The "Positive-Unlabeled" Trick

Usually, to teach a computer to distinguish between "Good" and "Bad," you need to show it examples of both. But here, they didn't have a clear list of "Bad" recipes (because a recipe not reported as scalable might just be unreported, not bad).

So, they used a clever math trick called Positive-Unlabeled (PU) Learning.

• The Analogy: Imagine you are trying to find all the hidden treasure chests in a forest. You have a map with 10 confirmed treasure spots (Positives). You also have a map of the whole forest, but most spots are just blank (Unlabeled).
• The robot learns: "Okay, these 10 spots are definitely treasure. The rest of the forest might have treasure, or it might be empty. I need to learn the pattern of the treasure spots to guess where the others are."
• The robot doesn't assume the blank spots are empty; it assumes they are a mystery to be solved.

3. The "Secret Sauce" of the Robot

The robot learned to look for specific "scalability signals" in the recipes, just like a seasoned chef looks for clues:

• Solvents: Did they use water or cheap, safe chemicals? (Good for scaling). Or did they use toxic, expensive, or weird solvents? (Bad for scaling).
• Temperature & Time: Was it cooked at a mild temperature for a short time? (Good). Or did it require extreme heat for days? (Bad).
• Complexity: Did the recipe need 10 different weird ingredients mixed in a specific order? (Hard to scale). Or was it simple? (Easy to scale).

4. The Calibration (The "Reality Check")

When the robot first guessed, it was a bit too shy. It tended to say, "I'm only 60% sure this is scalable," even when it was actually a "Yes."
The team applied a calibration step. Think of this like adjusting the sensitivity of a metal detector. They told the robot: "You are missing about 16% of the good recipes because they aren't written down yet. So, if you think something is 60% likely, bump it up to 72%." This made the robot much more accurate.

The Results: A Crystal Ball for Chemists

After training, the robot became a Crystal Ball for Industrial Chemistry.

• Accuracy: It can predict whether a new, tiny lab recipe will work on a factory scale with 91.4% accuracy.
• Speed: Instead of a chemist spending weeks reading papers or trying to guess, the robot can scan a new recipe in seconds and say, "This one looks promising for mass production," or "Skip this one, it's probably too hard to scale."

Why This Matters

This is like giving the chemical industry a filter.

• Before: Scientists discover a new material, spend years trying to scale it, and often fail because the recipe was never meant for a factory. It's a waste of time and money.
• Now: They can run the new recipe through the AI first. If the AI says "Green Light," they know it's worth investing in. If it says "Red Light," they can move on to the next idea immediately.

In short, this paper teaches a computer to read the "hidden language" of chemistry papers to predict which new discoveries are ready to leave the lab and change the world, saving us from chasing dead ends and helping us find the real winners faster.

Technical Summary

1. Problem Statement

Metal-Organic Frameworks (MOFs) have seen exponential growth in structural discovery, yet the transition from laboratory-scale crystallization (milligrams) to industrial-scale production (grams to kilograms) remains a critical bottleneck.

• The Challenge: Scalability "know-how" is fragmented across disparate scientific reports. A protocol successful at the milligram scale does not guarantee success at the kilogram scale, and the literature rarely explicitly states which protocols are scalable.
• The Gap: Traditional machine learning approaches struggle because scalability is not an intrinsic property of the crystal structure but a property of the synthesis protocol (solvents, temperature, time, modulators). Furthermore, the data is asymmetric: the literature contains many "positive" examples of scale-up, but the absence of a scale-up report for a specific MOF does not mean it is non-scalable; it often just means the data is missing (Unlabeled).
• Objective: To develop a data-driven method that predicts whether a newly reported small-scale MOF synthesis protocol has credible potential for gram-scale or larger production, enabling rapid industrial triage.

2. Methodology

The authors propose a workflow combining Large Language Models (LLMs) for data mining with Positive-Unlabeled (PU) Learning for classification.

A. Dataset Construction (ESU-MOF)

The team constructed the Experimental Scale-Up (ESU-MOF) dataset by mining literature from Web of Science (1995–2026):

• Positive Pool ($P$): Papers containing keywords like "scale-up," "gram-scale," "kilogram," or "pilot." (117 candidate groups).
• Unlabeled Pool ($U$): General MOF synthesis papers (solvothermal, single metal/linker) without explicit scale-up keywords. (946 candidate groups).
• Labeling Strategy:
  • $P_s$ (Strong Positives): Protocols with explicit scale-up evidence.
  • $P_a$ (Auxiliary Positives): Small-scale protocols from the unlabeled pool that match a $P_s$ MOF by name and composition (representing the "small-scale" version of a known scalable MOF).
  • $U$ (Unlabeled): Protocols with no known scale-up evidence (containing both true negatives and hidden positives).
  • $N$ (Negatives): A small, expert-curated set of protocols deemed non-scalable (e.g., toxic solvents, extreme conditions), held out strictly for evaluation.
• Extraction: An LLM agent extracted structured protocol records (metal, linker, solvent, T, time, yield) from PDFs, achieving 97.6% accuracy on key parameters. The final dataset contains 3,568 protocols (2,684 Unlabeled, 723 Positives, 161 Negatives).

B. Learning Framework: Positive-Unlabeled (PU) Learning

Standard binary classification fails here because the "Unlabeled" set contains hidden positives. The authors employed PU Learning:

• Training: The model is trained on $P_s$ and $P_a$ (mapped to label "P") and $U$ (mapped to label "U"). The negative set $N$ is excluded from training.
• Assumption: The SCAR (Selected Completely At Random) assumption is applied, positing that the probability of a scalable protocol being reported as a "positive" in literature is constant ($c$) and independent of its specific chemical features.
• Correction: The raw probability output by the model ($q(x)$) underestimates true scalability. A correction factor $\hat{c}$ (estimated as 0.837) is applied: $s_{PU}(x) = q(x) / \hat{c}$.
• Calibration: A Platt scaling layer is applied to the corrected scores to produce a final calibrated probability, with a decision threshold optimized for industrial triage.

C. Model Architecture

• Base Model: GPT-4.1.
• Fine-tuning: The model is fine-tuned on JSON representations of the synthesis protocols.
• Output: A single-token classification ("P" for scalable, "U" for unknown/unlikely) based on token-level log-probabilities.

3. Key Contributions

1. ESU-MOF Dataset: The first large-scale, literature-mined dataset specifically designed for MOF scale-up prediction, containing 3,568 structured protocols with PU learning labels.
2. PU Learning for Materials Science: Demonstrated that PU learning is the correct mathematical framework for predicting scalability, addressing the "missing negative" problem inherent in scientific literature.
3. LLM-Driven Workflow: Integrated LLMs for both data extraction (converting unstructured text to structured JSON) and prediction (fine-tuned classification), creating an end-to-end pipeline.
4. Calibration Strategy: Developed a rigorous three-stage scoring method (Raw Score $\to$ PU Correction $\to$ Platt Scaling) to handle label bias and provide reliable probability estimates.

4. Results

The model was evaluated on two distinct benchmarks:

A. Gold Benchmark (Classification Accuracy)

• Task: Distinguish explicit scale-up protocols ($P_s$) from expert-curated negatives ($N$).
• Performance:
  • Balanced Accuracy: 91.4% (vs. 78.5% for the base zero-shot LLM and ~66% for traditional ML).
  • F1 Score: 93.2%.
  • ROC-AUC: 95.8%.
  • Significance: The fine-tuned model significantly outperformed zero-shot LLMs and traditional machine learning baselines (Logistic Regression, Random Forest, BERT), proving that the model learned specific chemical heuristics for scalability.

B. Deployment Benchmark (Ranking Utility)

• Task: Rank auxiliary positives ($P_a$) within the large pool of unlabeled candidates ($U$) to simulate real-world screening.
• Performance:
  • ROC-AUC: 94.5%.
  • Precision@10: 80.0% (8 out of the top 10 ranked protocols were truly scalable).
  • Top-3 Hit Rate: 88.9% across test papers.
• Significance: The model successfully surfaces promising candidates from the "noise" of general literature, a critical capability for industrial screening where researchers cannot test every protocol.

5. Significance and Implications

• Accelerating Industrial Adoption: This work bridges the gap between academic discovery and industrial application by providing a tool to prioritize MOFs that are likely to be manufacturable, saving time and resources.
• Shift in Paradigm: It moves MOF discovery from a purely structure-centric approach to a process-centric approach, recognizing that the synthesis recipe is as important as the crystal structure.
• Generalizability: The PU learning framework and LLM extraction pipeline can be adapted to other areas of materials science where "negative" data is missing or where scalability/feasibility is the primary bottleneck.
• Limitations & Future Work: The current dataset is limited to single-metal/single-linker systems (excluding multivariate MOFs). As more pilot-scale data becomes available, the dataset will expand to cover more complex chemistries.

In conclusion, the paper demonstrates that scalability is a learnable property embedded in synthesis protocols. By leveraging LLMs and PU learning, the authors have created a robust tool that can predict industrial viability early in the discovery cycle, transforming how materials are selected for commercialization.