🤖 machine learning

Bridging Chemists and AI: An Expert-Augmented Framework for Interpretable Route Evaluation

This paper presents an expert-augmented, data-driven framework that integrates machine learning with chemists' domain knowledge to evaluate and interpret multi-step synthetic routes, achieving significantly higher accuracy and interpretability than previous baselines.

Original authors: Yujia Guo, Mikhail Kabeshov, Tat Hong Duong Le, Samuel Genheden, Marco V. Mijangos, Varvara Voinarvoska, Giulia Bergonzini, Ola Engkvist, Samuel Kaski

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yujia Guo, Mikhail Kabeshov, Tat Hong Duong Le, Samuel Genheden, Marco V. Mijangos, Varvara Voinarvoska, Giulia Bergonzini, Ola Engkvist, Samuel Kaski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master chef trying to invent a new, complex dish. You have a computer program that can suggest thousands of different ways to cook it, step-by-step. But here's the problem: the computer is great at listing possible steps, but it's terrible at knowing which steps will actually work in a real kitchen, which ones are too expensive, or which ones are just a waste of time.

This is the exact challenge chemists face when designing how to build new medicines. They need to choose the best "recipe" (synthetic route) from thousands of computer-generated options.

This paper introduces a new system that acts like a super-chef assistant. It combines the raw data-crunching power of Artificial Intelligence with the gut feeling and experience of real human chemists to decide which recipes are actually good.

Here is how the system works, broken down into simple steps:

1. The Problem: Too Many Options, Not Enough Judgment

Previously, computers tried to judge these recipes by comparing them to old recipes found in patents. But patents are like "fast food" menus—they show what someone managed to make in a hurry, not necessarily the best way to make it. Also, there aren't enough examples of "failed" recipes in the data to teach the computer what not to do.

2. The Solution: A Two-Phase Training Camp

The authors built a model that learns in two phases, much like training a new employee:

Phase 1: The Data-Driven Boot Camp (The "Student")
First, the computer model is trained on a massive library of millions of existing chemical routes (mostly from patents). It learns to spot patterns and calculate a "distance score." Think of this as the student learning to measure how similar a new recipe is to a known, working recipe. If the steps look very similar to a proven recipe, the score is high. If they look weird, the score is low.
- The Tech: They used a special type of AI called DeepSets. Imagine a basket of ingredients where the order you drop them in doesn't matter; the AI looks at the whole basket at once to understand the recipe, rather than getting confused by the order of steps.
Phase 2: The Expert Mentorship (The "Fine-Tuning")
This is the secret sauce. The computer is still a bit too robotic. So, the authors brought in real expert chemists. These experts looked at a smaller set of routes and gave them a simple rating: Good, Plausible, or Bad.
- The computer then took its "Student" knowledge and adjusted its brain (using a technique called LoRA, which is like adding a small, specialized note to a textbook) to match the human experts' opinions.
- Now, the computer doesn't just say "this looks 85% similar to a patent." It says, "This is a Good route because the steps are safe and efficient," or "This is Bad because it uses a dangerous chemical."

3. The Result: A Trustworthy Guide

The system produces two things:

A Number: A score showing how close the route is to a perfect reference.
A Label: A clear, human-readable category (Good, Plausible, or Bad).

How well did it work?

Accuracy: When the computer guessed the "Good/Bad" rating, it was right about 60% of the time for the very top choice. This is a huge jump from previous methods, which were only right about 17% of the time.
Agreement: When comparing the computer's scores to the human experts' scores, they agreed very closely (about 78% correlation). This means the computer is learning to "think" like a chemist.

4. Why This Matters

Before this, chemists had to manually read through hundreds of computer-generated recipes to find the few that were actually usable. This was slow, expensive, and hard to scale.

This new system acts as a filter. It automatically sorts the "garbage" recipes from the "gold" ones, saving chemists time and allowing them to focus on the most promising ideas. It bridges the gap between cold, hard data and the warm, intuitive judgment of human experts.

In short: The paper shows that if you teach a computer to look at data and then teach it to listen to human experts, it becomes much better at deciding which chemical recipes will actually work in the real world.

Technical Summary: Bridging Chemists and AI – An Expert-Augmented Framework for Interpretable Route Evaluation

Problem Statement

The selection of efficient multi-step synthetic routes is a critical bottleneck in organic synthesis, particularly within medicinal and process chemistry. While Computer-Aided Synthesis Planning (CASP) tools have advanced in generating diverse route options, the subsequent evaluation of these routes remains a significant challenge. Existing data-driven assessment systems often oversimplify the multi-objective nature of synthesis design (balancing feasibility, cost, safety, sustainability, etc.) and rely heavily on proxy datasets, such as patent-extracted routes, rather than universal, grounded criteria.

Patent data often reflects time-constrained choices rather than optimal chemistry, and existing datasets are sparse because typically only one or a few syntheses are reported per molecule. Consequently, models trained solely on patent data risk overfitting to the "appearance" of a patent route rather than learning the underlying reasoning and evaluation heuristics used by expert chemists. Furthermore, the lack of high-quality negative reaction data makes forward synthesis feasibility prediction difficult. Currently, the field relies heavily on manual evaluation by synthetic chemists, a process that is slow, expensive, and difficult to scale.

Methodology

The authors propose an expert-augmented, data-driven scoring framework (illustrated in Figure 1) that integrates machine learning with chemists' domain knowledge to provide both numerical and explainable route assessments. The methodology proceeds in three phases:

1. Pre-training of a DeepSets-based Scoring Model (Phase 1)

The core of the system is a neural network architecture based on DeepSets, designed to handle sets of reactions with permutation invariance. This allows the model to robustly process routes with varying numbers of reactions and unordered inputs.

Architecture: The model consists of two modules:
- Encoder ( $\phi$ ): Processes individual reaction features (reaction class, feasibility score, target compound representation, and optional route embedding) to produce a reaction encoding.
- Scorer ( $\sigma$ ): Aggregates the reaction encodings and combines them with route-level properties (cost, volume, complexity, and molecule fingerprints) to predict a final score.
Training Objective: The model is initially trained on a large corpus of patent routes (47,303 molecules, ~1.7 million routes) to predict the Tree Edit Distance (TED) between a generated route and a reference route. TED was selected as the metric because it offers a robust measure of structural and strategic similarity, outperforming other candidate metrics in distinguishing high-quality routes from poor ones.
Input Features: The system utilizes reaction embeddings (tested: Structural Difference Fingerprint [SDF], Chemical Reaction Fingerprint [RXNFP], Differential Reaction Fingerprint [DRFP]), reaction class feasibility scores, and route properties like cost and complexity.

2. Expert-Augmented Fine-Tuning (Phase 2)

To bridge the gap between numerical similarity (TED) and human judgment, the framework introduces a qualitative assessment layer.

Heuristic Criteria: Expert chemists established a clear three-tier classification system ("Good," "Plausible," "Bad") based on a 5-point reaction-to-route scoring scale.
- 5 points (Good): Highly likely to succeed, negligible functional group impact.
- 4 points (Plausible): Correct disconnection but requires minor adjustments or includes unnecessary but valid steps.
- 3 points: Retrosynthetically correct but relies on specific protecting group schemes or redox maneuvers.
- 2 points: Major selectivity issues or conceptual alterations required.
- 1 point: Highly unlikely to succeed or unresolved starting material issues.
Fine-Tuning Strategy: The pre-trained DeepSets backbone is fine-tuned using a small-data, expert-supervised adaptation.
- LoRA (Low-Rank Adaptation): The weights of the base encoder and scorer are frozen. Only the LoRA parameters of the encoder's hidden layers and a new classification head are optimized using expert-labeled data (120 routes).
- Output: This allows the model to learn a distinct route feasibility rating while preserving the original distance predictor as a secondary signal.

3. Expert-Augmented Assessment System (Phase 3)

The final system outputs both a quantitative multi-objective score and an interpretable qualitative feasibility rating (Good/Plausible/Bad), aligning with real-world decision-making in retrosynthetic planning.

Key Contributions

Expert-Augmented Framework: A novel pipeline that combines large-scale data-driven modeling (pre-training on patent data) with small-scale, expert-aligned assessment (fine-tuning on chemist evaluations).
Dual-Output System: The model provides both a quantitative score (TED prediction) and an interpretable qualitative rating (5-point scale mapped to 3-tier categories), addressing the need for transparency in AI-driven synthesis planning.
Permutation-Invariant Architecture: The use of DeepSets ensures the model handles routes as unordered sets, focusing on reaction-level properties independent of sequence order, which simplifies the learning task while maintaining robustness.
Efficient Fine-Tuning: The use of LoRA allows the system to adapt to expert judgment using a limited dataset (120 routes) without catastrophic forgetting of the pre-trained knowledge.

Results

The system was evaluated on a test set of 120 expert-labeled routes using 5-fold cross-validation.

Qualitative Prediction (Rating):
- The model achieved a Spearman correlation of 0.78 ± 0.05 and a Pearson correlation of 0.77 ± 0.06 with expert ratings on the 5-point scale.
- For the three-tier classification (Good/Plausible/Bad), the model achieved an overall accuracy of 67 ± 6.4%.
- Performance was particularly strong in distinguishing "Good" (Precision: 0.781, Recall: 0.795) and "Bad" routes, though some uncertainty remained between "Good" and "Plausible" categories.
Quantitative Prediction (Distance & Ranking):
- Distance Prediction: The SDF embedding outperformed other methods (DRFP, RXNFP, and no embeddings), achieving the lowest Mean Squared Error (7.102 ± 0.200) and highest R-squared (0.723 ± 0.003).
- Ranking Accuracy: The model achieved a top-1 ranking accuracy of 60.2% (0.603 ± 0.007) on the test dataset, substantially outperforming the previous baseline (17.5%).
- Top-k Performance: The SDF-based model identified approximately 95% of reference routes within its top-20 predictions.
Embedding Analysis: Interestingly, the study found that the simpler SDF embedding outperformed more sophisticated methods (DRFP, RXNFP), suggesting that for route distance prediction, simpler structural representations may be more effective and less prone to overfitting.

Significance and Claims

The paper claims that this expert-augmented framework successfully bridges the gap between data-driven machine learning and chemical domain knowledge. By moving beyond reliance on patent proxies and incorporating explicit expert heuristics, the system offers a more reliable, transparent, and actionable evaluation of synthetic routes.

The authors emphasize that the system does not merely predict a score but captures the nuance of expert chemical judgment, including how experts resolve multi-objective trade-offs. The dual-output nature (quantitative score + qualitative rating) makes the system suitable for practical integration into synthesis planning workflows. The authors note that while the model shows promising performance, it still faces challenges in distinguishing between adjacent categories (e.g., Good vs. Plausible), suggesting that future work could benefit from more granular human-in-the-loop learning to capture tacit knowledge beyond structural similarity.

The work positions itself as a step toward aligning AI with expert human judgment, a capability the authors suggest could be extended to other domains requiring complex, multi-objective decision-making, such as healthcare.