Molecular traps reveal limitations of synthesis planning algorithms
This paper introduces "molecular traps"—deliberately designed molecules that exploit common algorithmic weaknesses—to systematically expose significant failures in current computer-aided synthesis planning tools regarding chemoselectivity, regioselectivity, and steric hindrance, thereby offering a critical stress-testing framework for developing more practical and reliable synthesis algorithms.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
For the last decade, computers have become remarkably adept at mapping out how to build complex molecules, the tiny structures that make up medicines, plastics, and the materials of modern life. This field, known as computer-aided synthesis planning, asks a machine to look at a finished chemical and work backward to figure out which simpler ingredients and steps could create it. The goal is to find a path that is not just theoretically possible, but practical in a real laboratory, meaning the reaction must happen efficiently and produce the right version of the molecule without creating a messy mix of unwanted byproducts. While computers have gotten faster at proposing these paths and scoring high on standard tests, a lingering question remains: do these algorithms truly understand the subtle rules of chemistry, or are they simply memorizing patterns from a database of past successes?
A team of researchers led by Yang Tian at Tsinghua University and colleagues from InsightChemy Co., Ltd. decided to test the limits of these digital chemists by creating a specific kind of challenge. They designed what they call "trap molecules." These are not real chemicals waiting to be made, but rather artificial structures crafted specifically to confuse the software. The researchers built these traps to exploit three common areas where human chemists must exercise extreme care: choosing which part of a molecule to react when several parts look similar, ensuring the reaction happens at the exact right spot on a ring structure, and avoiding situations where the molecule is too crowded for the reaction to occur. By feeding these tricky designs into three different leading algorithms, the team discovered that the software frequently fails to recognize these basic chemical constraints, often suggesting routes that would produce the wrong product or fail entirely in a real lab.
The researchers constructed their test set by generating one hundred molecules designed to trip up the software on chemical selectivity, another hundred for positional selectivity, and twenty-seven for steric hindrance. In the first category, chemical selectivity, the problem arises when a molecule contains multiple groups that could react under the same conditions. A human chemist knows that if a molecule has two different reactive sites, one might react faster than the other, or a specific condition might be needed to protect one while the other changes. The researchers built trap molecules that included a target reaction site alongside a sensitive group that would likely interfere. When the algorithms were asked to plan the synthesis, they often proposed steps that ignored this sensitivity, suggesting a path that would destroy the sensitive group or react with the wrong part of the molecule. In the pre-set tests, where the researchers looked for specific known errors, the algorithms failed on between fifty-one and fifty-eight of the one hundred molecules. When the researchers added a second layer of review using an AI agent to spot other types of errors, the failure rates climbed, with the algorithms making mistakes on between sixty-six and seventy-seven of the one hundred cases.
The second challenge involved positional selectivity, which deals with molecules that have two or more identical-looking spots where a reaction could happen. In a real lab, if a reaction occurs at both spots, it creates a mixture of isomers—molecules with the same atoms but arranged differently—that are notoriously difficult to separate. The researchers created trap molecules with two nearly identical reaction centers, separated only by tiny differences in their surroundings, such as a methyl group on one side and an ethyl group on the other. They wanted to see if the software would recognize that reacting at both spots would be a dead end. The results were stark. In the initial check for known errors, the algorithms missed the problem in forty-eight to sixty-nine of the one hundred molecules. Once the AI agent reviewed the remaining predictions for subtle issues, the total number of failures rose dramatically, with the algorithms suggesting impractical routes for eighty-five to ninety-three of the one hundred trap molecules.
The final category tested steric hindrance, which refers to the physical crowding around a reaction site. If a molecule is packed with bulky groups, the reagents simply cannot get close enough to react, much like trying to fit a large object into a tiny space. The researchers designed molecules with large, bulky groups blocking the reaction center. They found that the algorithms, which are trained mostly on successful reactions, struggled to recognize when a reaction was physically impossible due to this crowding. In the pre-set tests, the algorithms failed to identify the problem in twenty-one to twenty-six of the twenty-seven trap molecules. Even after the AI agent review, the failure rate remained extremely high, with the algorithms suggesting impossible routes for twenty-five to all twenty-seven of the molecules.
Beyond these specific categories, the study also uncovered a broader issue: the algorithms sometimes generated "hallucinations." These were predictions that made no chemical sense at all, such as suggesting that a complex molecule could be built from a single simple gas, or proposing reactions where atoms appeared to move to impossible locations. The researchers noted that these absurd predictions, which stem from the algorithms not having learned stable rules of chemistry, might be an even more fundamental problem than the selectivity errors. The study concludes that while current software has made impressive strides in speed and volume, it lacks a deep understanding of the boundaries that define a successful chemical reaction. The use of these trap molecules offers a new way to stress-test these systems, revealing that without a better grasp of failure modes and physical constraints, the routes proposed by computers may remain impractical for real-world use.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.