🧬 biology

Physics-Aware Auxiliary Losses Improve Out-of-Distribution Generalization of a GNN Synthesizability Filter

This paper demonstrates that incorporating cheap, closed-form physical priors—specifically topological complexity and strain energy—as auxiliary losses in a Graph Neural Network significantly improves its out-of-distribution generalization for predicting molecular synthesizability, while also highlighting the critical necessity of multi-seed evaluation to avoid misleading conclusions from single-seed experiments.

Original authors: Riya Bisht, Dhruv Agarwal

Published 2026-06-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Riya Bisht, Dhruv Agarwal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are building a robot chef to invent new recipes for a restaurant. This robot is incredibly creative; it can whip up millions of unique dish ideas every second. However, there's a problem: the robot often suggests dishes that are impossible to cook in a real kitchen (like a cake made of glass or a soup that requires a fire that doesn't exist).

To stop the robot from wasting time on impossible ideas, you hire a "Kitchen Inspector" (a computer program) to check the recipes. The problem is, this inspector was trained only on standard, everyday recipes (like pizza and pasta). When the robot suggests a wild, exotic dish from a different culture (one the inspector has never seen), the inspector gets confused and starts rejecting good ideas or accepting bad ones.

This paper is about making that "Kitchen Inspector" smarter so it doesn't panic when it sees something new.

The Problem: The "Bookish" Inspector

The current inspectors (called SAScore, SCScore, etc.) are like students who only memorized a specific textbook. They are great at grading recipes that look exactly like the ones in the book. But if you give them a recipe from a completely different book (a different "distribution" of data), they fail. They rely purely on statistics: "This ingredient combination looks weird, so it must be bad."

The authors wanted to know: Can we teach the inspector a few basic laws of physics so it understands why a recipe might be impossible, even if it's never seen that specific recipe before?

The Solution: Adding "Common Sense" Physics

Instead of just feeding the inspector more recipes, the authors added two simple "physics rules" as extra homework for the AI during its training. Think of these as two new lenses the inspector wears:

The "Tangled String" Rule (Topological Complexity):
Imagine a recipe that requires tying a knot in a piece of string so tight that it breaks the string. The AI learns to recognize when a molecular structure is just too "tangled" or complex to be real, based on a mathematical measure of complexity (the Bertz index). It's like learning that some knots are physically impossible to tie without breaking the rope.
The "Squeezed Spring" Rule (Strain Energy):
Imagine a recipe that asks you to bend a metal spoon until it snaps. In chemistry, this is called "strain." If a molecule is bent or twisted in a way that makes it unstable (like a spring under too much pressure), it's hard to make. The AI learns to look for these "squeezed springs" using a standard physics calculator (MMFF94) and penalizes recipes that look too strained.

The Experiment: The "New Cuisine" Test

The authors trained their AI on a massive dataset of standard "drug-like" molecules (the everyday recipes). Then, they tested it on a completely different set of molecules found in nature (natural products), which are often stranger and more complex.

They ran the test five times with different random starting points to make sure the results weren't just luck.

The Results: Small but Real Improvements

Here is what they found:

Inside the "Textbook": When testing on the same kind of molecules the AI was trained on, the new physics rules didn't change anything. The AI was already perfect at grading the recipes it knew.
Outside the "Textbook" (The Real Test): When the AI had to grade the strange, new "natural product" recipes, the versions with the physics rules did better than the standard version.
- The standard AI got a score of 0.977.
- The AI with the "Tangled String" rule improved to 0.983.
- The AI with the "Squeezed Spring" rule improved to 0.980.
- The AI with both rules improved the most to 0.984.

While these numbers look small (a difference of less than 1%), in the world of AI, this is a statistically significant win. It means the AI is slightly better at guessing what is possible when it encounters something totally new.

A Warning: Don't Trust a Single Run

The authors also shared a cautionary tale. When they first ran the experiment just once (with one random seed), the results looked weird and confusing. It seemed like the physics rules sometimes hurt the AI. But when they ran it five times and looked at the average, the "weird" story disappeared, and the clear, positive trend emerged.

The Lesson: In AI research, don't trust a single experiment. You need to run it multiple times to see the real story, because one lucky (or unlucky) roll of the dice can tell a false tale.

Summary

The paper proves that giving a drug-discovery AI a tiny bit of "physics common sense" (understanding complexity and strain) helps it generalize better when it encounters novel molecules it has never seen before. It doesn't need to be a complex physics engine; just a few simple, cheap rules added as extra training tasks are enough to make the AI more reliable in the real world.

Technical Summary: Physics-Aware Auxiliary Losses Improve Out-of-Distribution Generalization of a GNN Synthesizability Filter

Problem Statement
Generative models in drug discovery (e.g., VAEs, GFlowNets, diffusion models) frequently propose novel molecular structures that lie far outside the distribution of training data used for downstream synthesizability filters. Existing standard filters—SAScore, SCScore, RAscore, and DeepSA—are purely statistical models trained on labeled corpora of "makeable" versus "hard" molecules. Consequently, these filters degrade precisely when they are most needed: when evaluating novel, out-of-distribution (OOD) structures generated by creative models. The authors posit that purely statistical baselines lack the inductive bias required to generalize to novel chemical spaces, particularly when moving from drug-like molecules to complex natural products.

Methodology
The authors propose integrating cheap, closed-form physical priors as auxiliary supervision signals into a standard Graph Neural Network (GNN) backbone, rather than using Physics-Informed Neural Networks (PINNs). They argue that PINNs are unsuitable for synthesizability because the domain is discrete (molecular graphs) rather than continuous, and there is no governing differential equation for synthesizability (which depends on reagents, kinetics, and human judgment).

Backbone: A 4-layer Graph Isomorphism Network with edge features (GINE). The model outputs a classification logit for synthesizability (easy/hard) and a scalar auxiliary value.
Auxiliary Losses: Two specific physics-aware losses are added to the training objective:
1. Topological Complexity: A Huber regression loss supervising the auxiliary head with the Bertz index (a closed-form complexity metric computed by RDKit). This forces the encoder to learn calibrated structural complexity.
2. Strain Energy: A soft penalty loss supervising the model using MMFF94 force-field energy. This penalizes confident "easy" predictions on molecules with high strain energy, acting as an asymmetric Lagrangian-style constraint.
Training Objective: The total loss is a weighted sum of Binary Cross-Entropy (BCE) for the main classification task and the auxiliary losses ( $L = BCE + \lambda_c L_{complexity} + \lambda_s L_{strain}$ ). Crucially, auxiliary heads are used only during training; inference relies solely on the classification head, ensuring no increase in deployment cost.
Experimental Protocol:
- Data: A corpus of 65,177 molecules combining HIV and Tox21 (drug-like) and COCONUT (natural products). Labels are derived from SAScore thresholds (<4 easy, >5 hard).
- OOD Split: A single-source covariate shift protocol: train on HIV+Tox21, test exclusively on COCONUT natural products.
- Evaluation: A 5-seed multi-run experiment with paired bootstrap confidence intervals to assess statistical significance, avoiding the pitfalls of single-seed evaluation.

Key Contributions

Formulation of Physics-Aware Auxiliary Losses: A clean implementation of two computationally cheap, closed-form physical priors (Bertz index and MMFF94 strain) as regularizers on a standard GNN.
Source-Based OOD Evaluation: A rigorous protocol isolating corpus transfer for synthesizability filters, specifically testing the gap between drug-like molecules and natural products.
Empirical Evidence of OOD Improvement: A multi-seed, confidence-interval-based demonstration that physics-aware variants yield statistically significant OOD improvements over a purely statistical baseline.
Methodological Cautionary Tale: The authors document a "single-seed" failure mode where a non-monotone, misleading narrative was generated, which was overturned only by multi-seed evaluation, highlighting the necessity of robust statistical validation in AI-assisted chemistry.

Results

In-Distribution Performance: All variants (baseline, +complexity, +strain, +both) perform nearly identically on in-distribution data (Test AUC $\approx$ 0.996). The authors note this is expected, as the GNN is essentially memorizing a closed-form labeling function (SAScore).
Out-of-Distribution Performance: On the COCONUT test set, all three physics-aware variants significantly outperform the baseline (Mean OOD AUC 0.9774):
- +Complexity: $\Delta = +0.0060$ (95% CI [+0.0023, +0.0102])
- +Strain: $\Delta = +0.0032$ (95% CI [+0.0008, +0.0052])
- +Both: $\Delta = +0.0066$ (95% CI [+0.0038, +0.0093])
- The combination of both losses yields the best performance. Every confidence interval excludes zero, confirming statistical significance.
Comparison: While SAScore itself achieves a perfect AUC (1.000) on the test set, this is circular as the labels are derived from SAScore. Independent learned scorers like SCScore perform poorly (AUC 0.656) on this split, highlighting the lack of shared ground truth and the specific challenge of natural products.

Significance and Claims
The paper explicitly claims modest but statistically significant results. The primary significance is not that the model achieves state-of-the-art absolute accuracy, but that physics-grounded inductive bias acts as an effective regularizer for OOD generalization in a setting where purely statistical models fail.

The authors emphasize that:

The effect is visible only under OOD evaluation; in-distribution metrics are indistinguishable.
The approach keeps inference costs identical to the baseline, making it a "drop-in" improvement.
The "physics as supervision, not PINN residual" paradigm is a viable and generalizable strategy for discrete, combinatorial domains where continuous field assumptions do not hold.
The study serves as a methodological warning against relying on single-seed evaluations in AI-driven scientific pipelines, which can produce false, non-monotone narratives.