🤖 AI

Improving Molecular Property Prediction in Small Language Models Using Graph-based Tools

This paper proposes a modular Context-Augmented Prompting framework that enhances small language models' molecular property prediction by integrating graph-based tools to provide confidence scores and explanatory subgraphs, achieving significant accuracy improvements over SMILES-only baselines while acknowledging a remaining performance gap compared to specialized GNNs.

Original authors: Konstantinos Bougiatiotis, Dimitrios Kelesis, Georgios Paliouras

Published 2026-07-16

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Konstantinos Bougiatiotis, Dimitrios Kelesis, Georgios Paliouras

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but the only clue you have is a long, confusing list of ingredients written in a secret code. You know that if you mix certain ingredients together, you might get a delicious cake, or you might get a toxic explosion. In the world of science, this "secret code" is called a SMILES string, and the "ingredients" are atoms in a molecule. Scientists have been using these strings to predict if a new chemical will be a life-saving medicine or a dangerous poison.

For a long time, the best detectives were specialized computers called Graph Neural Networks (GNNs). These computers are like master chefs who can look at a recipe and instantly see how the ingredients connect in 3D space, understanding that a specific cluster of atoms might be the "danger zone." However, these master chefs are often "black boxes"—they give you the answer but can't explain why in a way humans can understand.

Recently, a new type of detective has emerged: Small Language Models (SLMs). Think of these as very smart, well-read teenagers who have read millions of books and can guess the answer based on the text alone. But here's the catch: they are "structurally blind." If you just give them the list of ingredients (the SMILES string), they miss the crucial connections between them. They might guess the cake is safe because they've read about cake before, but they can't see the toxic knot of ingredients hiding in the middle. This paper asks a simple, playful question: What if we let the "blind" detective borrow the eyes of the "master chef" just for a second? What if we could give the language model a little hint, a map, and a short explanation from the expert, and see if that helps it solve the mystery?

The Paper's Story: Giving the Detective a Flashlight

This paper, titled "Improving Molecular Property Prediction in Small Language Models Using Graph-based Tools," is all about teaming up these two types of detectives. The authors, researchers from Greece, wanted to see if they could make "Small Language Models" (SLMs) much better at predicting molecular properties by letting them use "tools" provided by a Graph Neural Network (GNN).

Imagine the SLM as a student taking a test. Usually, the student only gets the question written on a piece of paper (the SMILES string). Sometimes, the student gets it right, but often they get it wrong because they can't "see" the shape of the molecule. The authors proposed a new way of taking the test called Context-Augmented Prompting. Instead of just handing the student the question, they let the student call a "help desk" (the GNN expert) before answering.

Here is how the "help desk" works:

The Prediction: The GNN looks at the molecule and says, "I think this is toxic, and I'm 90% sure."
The Map: The GNN uses a special tool (GNNExplainer) to highlight the exact tiny part of the molecule that makes it toxic. It cuts out that specific piece and turns it back into a text string the student can read.
The Reasoning: The student is then asked to look at that specific piece and explain why it might be dangerous.

The researchers tested this idea on three different "students" (SLMs): Llama 3.2, Qwen 2.5, and DeepSeek. They gave them a test on two different sets of chemical puzzles: MUTAG (a small set of 188 molecules) and Tox21 (a larger set of 1,000 molecules). They tried five different ways of giving the test:

The Baseline: Just the SMILES string (no help).
The Map: SMILES + the toxic subgraph.
The Hint: SMILES + the expert's prediction and confidence score.
The Reasoning: SMILES + a short explanation generated by the model itself.
The Full Package: All of the above combined.

What They Found

The results were like watching a student go from failing a test to acing it, but with some interesting twists.

First, the "Full Package" approach was a game-changer for the harder test (Tox21). When the models were just given the raw SMILES string, the DeepSeek model only got 38.50% of the answers right. But when they gave it the full package—the expert's hint, the map of the toxic part, and the reasoning—the score jumped to 67.00%. That is a massive 74.03% relative improvement! On the other dataset, Tox21, the Qwen model saw a 44.21% improvement, and Llama 3.2 saw a 30.93% boost.

However, the paper suggests that not all students benefit equally. The DeepSeek and Qwen models seemed to really know how to use the extra tools, integrating the hints and maps perfectly. Llama 3.2, on the other hand, showed "mixed behavior." Sometimes the extra help made it do better, but other times, especially on the smaller MUTAG dataset, the extra information seemed to confuse it, and it didn't improve beyond the baseline. The authors suggest that smaller, general-purpose models might get "distracted" by too much information if they aren't trained to handle it well.

The "Blind Spot" Remains

Here is the most important part of the story: even with all the help, the students didn't beat the master chef. The specialized GNN expert model got 84.21% on the MUTAG test and 71.00% on the Tox21 test. Even the best-performing SLM with all the tools (DeepSeek on Tox21) only reached 67.00%.

This suggests that while giving a language model a map and a hint helps it see better, it still can't fully replace the deep, structural understanding that a dedicated graph model has. The paper argues that text-based reasoning has a limit; it can't fully capture the complex 3D geometry of a molecule just by reading a description of it.

Proving the Map Matters

To make sure the "maps" the GNN was drawing were actually useful and not just random guesses, the researchers did a cool experiment. They took the expert model and started deleting parts of the molecule.

When they deleted the parts the GNN said were important (the "explainer-ranked" edges), the expert's accuracy dropped like a stone.
When they deleted random parts, the accuracy stayed high for much longer.

This proved that the "maps" the GNN was generating were pointing to the actual "critical spots" in the molecule. It was like showing that if you remove the engine from a car, it stops working, but if you remove a random screw, it keeps running. This gave the researchers confidence that the tools were providing genuine, necessary evidence.

The Verdict

The paper concludes that this "agentic" approach—where a small language model acts like a detective that can call an expert for help—is a very promising way to improve chemical predictions without needing to build a massive, expensive new AI from scratch. It suggests that by combining the "reading" power of language models with the "seeing" power of graph tools, we can get much closer to the truth.

However, the authors are careful not to call this a solved problem. They emphasize that while the gains are substantial (sometimes over 25% better), there is still a gap between these text-based helpers and the specialized graph models. The future, they suggest, lies in these "neuro-symbolic" teams, where the AI can use tools to check its own work, creating a system that is both smart and explainable. It's a step toward a future where computers don't just guess, but can actually show their work and explain their reasoning to the humans they are helping.

Technical Summary: Improving Molecular Property Prediction in Small Language Models Using Graph-based Tools

Problem Statement

Molecular Property Prediction (MPP) is a critical component of drug discovery, traditionally dominated by Graph Neural Networks (GNNs) due to their ability to model the topological structure of molecules. However, GNNs often function as "black boxes," lacking human-interpretable rationales. Conversely, Small Language Models (SLMs) and Large Language Models (LLMs) have shown promise in zero-shot reasoning but suffer from "structural blindness" when processing molecules via Simplified Molecular Input Line Entry System (SMILES) strings. SMILES is a sequence-based abstraction that frequently fails to encode complex 2D branching and spatial relationships, leading to inconsistent performance when predictions rely on specific local motifs or global connectivity.

The core challenge addressed is how to bridge the gap between the structural reasoning capabilities of GNNs and the linguistic reasoning of SLMs without retraining the language models or resorting to computationally expensive architectural integrations.

Methodology: Context-Augmented Prompting Framework

The authors propose a modular Context-Augmented Prompting framework that enables agentic tool use at inference time. In this setup, an SLM acts as a reasoning controller that queries pre-trained graph-based tools to generate structured context, which is then fed back into the prompt to enhance zero-shot prediction.

The Agentic Loop

Given an input SMILES string $s$ , the SLM can invoke three primary tools:

ExpertPredict( $s$ ): Converts the SMILES to a molecular graph $G$ and queries a pre-trained GNN (specifically a GIN model) to return a predictive label $\hat{y}$ and confidence score $\hat{p}$ .
ExplainSubgraph( $s$ ): Utilizes GNNExplainer to identify the most influential subgraph (motif) contributing to the prediction. This is extracted as an edge-importance mask, thresholded to retain the top 10% of edges, and converted back into a subgraph SMILES string ( $s_c$ ).
ChemReason( $s, s_c, \hat{y}, \hat{p}$ ): An instruction block where the SLM itself generates a natural-language rationale explaining why the identified motif $s_c$ supports the predicted property (e.g., toxicity). This ensures the reasoning is generated by the SLM based on GNN-derived evidence, avoiding bias from larger teacher models.

Prompt Configurations

The study evaluates five prompting configurations ranging from baseline to full context:

SMILES (Baseline): Input is the raw SMILES string only.
SMILES + SUBGRAPH: Includes the raw SMILES and the extracted subgraph SMILES ( $s_c$ ).
SMILES + HINT: Includes the raw SMILES and the GNN expert's prediction and confidence.
SMILES + REASONING: Includes the raw SMILES and a rationale generated by the SLM regarding the motif's relevance.
ALL CONTEXT: Combines the raw SMILES, expert prediction/confidence, subgraph SMILES, and the SLM-generated rationale.

Experimental Setup

The framework was evaluated on two benchmark datasets:

MUTAG: 188 nitroaromatic compounds labeled for mutagenicity.
Tox21: A subset of 1,000 molecules converted to a binary toxicity classification task.

Three SLMs were tested in a zero-shot manner (no fine-tuning on the datasets):

DeepSeek (Lite-Code): 2.4B active parameters (16B MoE architecture), optimized for code and structured reasoning.
Qwen 2.5 (Code): 3B parameters, tuned for step-wise generation.
Llama 3.2: 3B parameters, a general-purpose conversational baseline.

A GIN model was trained on the training splits (70/30 split) to serve as the "expert" tool, while SLMs were evaluated solely on the test splits.

Key Results

The study demonstrates that enriching prompts with graph-derived context yields substantial accuracy gains, though performance is highly dependent on the specific SLM architecture and the difficulty of the dataset.

Significant Gains on Complex Tasks: On the more challenging Tox21 dataset, the ALL CONTEXT configuration achieved relative improvements of 74.03% for DeepSeek and 44.21% for Qwen 2.5 over the SMILES-only baseline.
Model Dependency: DeepSeek and Qwen 2.5 showed consistent and strong improvements across configurations. In contrast, Llama 3.2 exhibited mixed behavior; while some single-tool variants helped, the ALL CONTEXT configuration did not improve upon the baseline for MUTAG and showed smaller gains on Tox21. The authors suggest smaller general-purpose models may suffer from "information distraction" when integrating heterogeneous evidence.
Robustness of Combined Context: Single-tool signals (e.g., hints or subgraphs alone) were sometimes unstable or even degraded performance (e.g., DeepSeek on MUTAG with hints). However, the ALL CONTEXT configuration was generally more robust, suggesting complementary effects where expert priors, structural evidence, and semantic reasoning reinforce each other.
Persistent Gap to Specialized Models: Despite the improvements, SLMs with ALL CONTEXT did not consistently surpass the specialized GNN expert (GIN). For instance, on MUTAG, the GIN achieved 84.21% accuracy, while the best SLM (DeepSeek) reached 75.44%. This highlights that purely text-conditioned reasoning cannot fully replace the inductive bias of graph representation learning.

Validation of Causal Relevance

A necessity-based edge-drop intervention confirmed that the subgraphs extracted by GNNExplainer are not merely correlational. Removing edges ranked by importance caused a steeper drop in expert accuracy compared to random edge removal, indicating that the extracted motifs contain decision-critical information.

Significance and Claims

The paper claims to offer a simple, modular, and accessible approach to improving molecular property prediction without retraining language models. Its primary contributions are:

Bridging Modalities: It successfully integrates graph-based structural learning with sequence-based SLM reasoning via GNN-derived hints, mitigating the "structural blindness" of SMILES-only inputs.
Quantification of Efficacy: It provides empirical evidence that while graph-derived context substantially enhances predictive accuracy, the gains are dataset- and model-dependent.
Agentic Governance: The framework demonstrates a "two-level governance" pattern where the SLM acts as a semantic controller. It can interpret graph-based explanations, identify inconsistencies (e.g., when the GNN expert is wrong), and decide whether to accept or override the expert output based on domain constraints.
Practical Deployment: Unlike computationally expensive hybrid architectures (e.g., MolCA, MolLM), this approach preserves the frozen weights of SLMs, making it a practical step toward causally grounded neuro-symbolic agents in scientific domains.

The authors conclude that while tool-augmented SLMs offer a lightweight path to combining neural prediction with symbolic graph evidence, a gap remains between text-conditioned reasoning and specialized graph models, underscoring the continued necessity of graph inductive biases in molecular science.