A systematic investigation of molecular encoding methods for drug property predictions across neural network and Transformer encoder-based model

In the race to discover new medicines, scientists face a daunting challenge: the universe of possible drug molecules is vast, but testing them all in a lab is impossible. To navigate this, researchers use computers to predict how a molecule will behave before a single drop is synthesized. This process relies on a crucial first step: translating the complex, three-dimensional shape of a chemical into a language a computer can read. Think of this translation as converting a physical object into a barcode; the way that barcode is designed determines how well the computer can understand the object's properties, such as whether it will dissolve in water, cross into the brain, or cause harm. For years, scientists have debated which "barcode" system works best, often assuming that more complex computer models would automatically solve the problem. However, the effectiveness of these predictions depends heavily on the quality of the initial translation, not just the sophistication of the computer analyzing it.

A team of researchers at National Tsing Hua University in Taiwan set out to settle this debate by systematically testing how different ways of encoding molecular structures affect prediction accuracy. They focused on two distinct types of computer architectures: a standard neural network, which functions like a simple, layered decision-maker, and a more advanced Transformer-based model, a type of artificial intelligence originally designed to understand human language and capable of weighing the importance of different parts of a sentence simultaneously. The researchers fed both models a variety of molecular "barcodes," including traditional topological fingerprints that map atomic connections, substructure-based codes that identify specific chemical groups, and string-based representations that treat molecules as text. They trained these systems on seven well-known datasets covering diverse tasks, from predicting toxicity and mutagenicity to estimating how well a drug can penetrate the blood-brain barrier.

The results offered a clear, practical guide for the field. Contrary to the assumption that the more complex Transformer model would consistently outperform the simpler neural network, the study found that the standard model often achieved equal or slightly better accuracy across most tasks. In fact, the choice of the molecular encoding method proved to be far more critical than the choice of the computer model itself. For instance, on the task of predicting whether a molecule would be toxic, a specific type of fingerprint called MACCS yielded the best results for the simple model, while a different fingerprint called FCFP worked best for the complex model. This suggests that there is no single "perfect" code for all chemical problems; the best approach depends entirely on the specific property being predicted. The researchers also validated their findings on external datasets, confirming that their models remained robust when faced with new, unseen data, with average accuracy scores consistently exceeding 0.9 on several biological classification tasks.

Beyond raw performance, the study tackled a major hurdle in artificial intelligence: interpretability. Often, powerful AI models act as "black boxes," providing an answer without explaining why. The researchers demonstrated that the Transformer model, while not always the most accurate, offered a unique window into its own reasoning. By analyzing the internal "attention" weights of the model—which indicate which parts of the input the computer focused on most heavily—they could identify the specific chemical substructures driving a prediction without needing external explanation tools. In a compelling case study comparing Morphine and Heroin, the model correctly identified that the presence of hydroxyl groups (oxygen-containing structures) in Morphine made it less likely to cross the blood-brain barrier compared to Heroin, which lacks these groups. The model assigned higher attention to these hydroxyl features in Morphine, mirroring established chemical knowledge. This ability to highlight chemically meaningful patterns suggests that these models can not only predict outcomes but also help scientists understand the structural reasons behind them.

Ultimately, this work shifts the focus from simply building larger, more complex models to carefully selecting the right tools for the job. The researchers concluded that while advanced architectures like Transformers offer valuable insights into how molecules interact, they do not automatically guarantee superior predictions. Instead, the most effective strategy involves matching the specific molecular encoding method to the prediction task and the chosen model architecture. By prioritizing interpretability alongside accuracy, this approach provides a more principled path forward for drug discovery, ensuring that the computer models used to design life-saving medicines are not only powerful but also understandable and reliable.