All-Atom Structure Tokenizer: AtomLM-9b for Biomolecular Structure Prediction
The paper introduces an all-atom structure tokenizer and the AtomLM-9b language model, which together achieve biomolecular structure prediction performance comparable to AlphaFold 3 by compressing complex geometries into discrete tokens and reconstructing high-fidelity atomic structures.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
To understand the challenge faced by the researchers, one must first grasp the sheer complexity of the molecules that make up life. Proteins, DNA, and RNA are not static statues but intricate, three-dimensional machines built from chains of smaller chemical units. Their function depends entirely on how these chains fold and twist in space, bringing specific atoms into contact with one another. For decades, scientists have struggled to predict these shapes from the chemical code alone. The problem is that the number of ways a chain can fold is astronomical, and calculating the precise position of every single atom in a large molecule is computationally overwhelming. Traditional methods often simplify the problem by ignoring the fine details of individual atoms, focusing instead on the backbone of the chain, which can miss critical interactions that determine how a drug binds to a target or how an enzyme works.
A team of researchers has now developed a new approach that bridges the gap between high-level chemical descriptions and the precise, atom-by-atom reality of molecular structures. They created a system that translates the continuous, fluid geometry of a molecule into a set of discrete, manageable symbols, much like turning a complex melody into a sequence of musical notes. By compressing the detailed shape of a molecule into a vocabulary of 8,192 unique tokens, they were able to train a powerful artificial intelligence to predict these symbols based solely on the chemical identity of the molecule. Once the AI predicts the sequence of symbols, a second part of their system expands them back into a full, three-dimensional model containing every heavy atom. This method allows the computer to learn the rules of molecular folding without getting bogged down by the sheer volume of raw data, achieving a level of accuracy that rivals the most advanced existing tools.
The core of this achievement is a tool they call an "all-atom structure tokenizer." Imagine trying to describe the shape of a complex sculpture to a friend over the phone. If you tried to list the coordinates of every single point on the surface, the conversation would never end. Instead, you might describe the sculpture in terms of its major components—a head, a torso, arms—and how they connect. This team did something similar for molecules. They broke down proteins, DNA, and small drug-like molecules into standard chemical units. For a protein, a unit is a single amino acid residue; for DNA, it is a nucleotide; for a small drug molecule, it is a single heavy atom. They then built a neural network that looks at the entire assembly of these units, understanding how they relate to one another in space, and converts that spatial arrangement into a short string of numbers.
This conversion process is the "tokenizer." It takes the continuous, floating coordinates of atoms and squeezes them into a fixed set of 8,192 possible codes. The researchers used a technique called finite scalar quantization, which acts like a high-precision ruler that snaps any measurement to the nearest marked line on a grid. This ensures that the complex, infinite possibilities of molecular shapes are reduced to a finite, manageable vocabulary. Crucially, this process preserves the chemical identity of the molecule. The system knows that a specific token represents a particular amino acid with a specific side chain, or a specific atom in a drug molecule, without needing to store the exact coordinates in the token itself. The coordinates are recovered later, after the sequence of tokens has been predicted.
With this tokenizer in place, the researchers trained a large language model, which they named AtomLM-9b, to predict these structural tokens. This model is based on a powerful existing AI architecture but has been adapted to understand chemistry. Instead of reading sentences and predicting the next word, it reads the chemical identity of a molecule—the list of atoms and how they are bonded—and predicts the sequence of structural tokens that describe its 3D shape. The model does not see the final 3D structure during training; it only sees the chemical recipe and learns to output the correct sequence of tokens. Once the model generates this sequence, the tokenizer's partner, the "de-tokenizer," takes over. It reads the sequence of tokens, along with the original chemical information, and reconstructs the full 3D coordinates of every atom in the molecule. This two-step process separates the task of understanding the sequence from the task of drawing the shape, allowing each part to be optimized independently.
The results of this approach are striking. When tested on a standard set of protein structures used to evaluate prediction tools, the system achieved an all-atom accuracy score of 0.832 on one major benchmark and 0.835 on another. These scores indicate that the reconstructed structures are extremely close to the real, experimentally observed shapes. In a direct comparison on a specific set of protein-protein interaction tasks, the new model achieved a success rate of 75.01%, slightly outperforming the published results of AlphaFold 3, which stood at 72.93%. The model also showed strong performance in predicting the shapes of DNA and RNA, and in determining how small drug molecules bind to proteins, areas where other models sometimes struggle.
The researchers were careful to test the system under strict conditions to ensure the results were genuine. They trained the model on a vast collection of known molecular structures but excluded the specific test cases from the training data to prevent the model from simply memorizing answers. They also verified that the model could handle complex mixtures of different molecule types, such as a protein bound to a piece of DNA and a small drug molecule, all at once. The system successfully predicted the shapes of these complex assemblies, demonstrating that the shared vocabulary of tokens works across different types of chemistry. The model does not rely on pre-existing structural templates or multiple sequence alignments, which are often used by other methods to boost accuracy. Instead, it learns the physical rules of folding directly from the data.
One of the most significant aspects of this work is the clarity of the interface between the language model and the physical structure. By using a discrete set of tokens, the researchers created a clean boundary where the AI predicts a sequence, and a separate, deterministic process turns that sequence into a 3D model. This separation allows for a rigorous evaluation of how well the AI understands the relationship between chemistry and shape. The study shows that this discrete approach is not just a theoretical exercise but a practical method that can generate high-fidelity atomic structures. The model's ability to predict the positions of side chains and ligand atoms with high precision suggests that it has learned the subtle geometric constraints that govern molecular interactions.
The team also explored the limits of their system through a series of tests where they removed specific components to see how much each part contributed to the final result. They found that the hierarchical way the model processes information, moving from individual atoms to chemical units and then to the whole assembly, was essential for accuracy. They also discovered that the ability to refine the positions of atoms across different units—allowing a protein residue to adjust its shape based on a nearby drug molecule—was critical for getting the binding poses right. Without these refinements, the model's ability to predict how molecules interact dropped significantly. These findings confirm that the system is not just guessing but is capturing the physical reality of molecular assembly.
While the system is highly accurate, the researchers acknowledge that it is not perfect. The finite vocabulary means that some very rare or unusual chemical configurations might not be represented as precisely as common ones. The model focuses on heavy atoms and does not currently predict the positions of hydrogen atoms or the effects of the surrounding water environment, which are important for some specific biological questions. However, for the vast majority of structural biology tasks, the level of detail provided is sufficient to understand how molecules function. The success of this approach suggests that treating molecular geometry as a language, with a shared vocabulary for different types of chemical units, is a powerful way to solve the problem of structure prediction.
This work represents a shift in how scientists approach the problem of molecular modeling. Instead of trying to calculate the forces between every atom in a single, massive simulation, the researchers have shown that a language model can learn the patterns of molecular shapes and generate them efficiently. The combination of a tokenizer that compresses geometry and a language model that predicts the sequence offers a new path forward. It demonstrates that the complexity of life's building blocks can be captured in a compact, discrete form, and that artificial intelligence can learn to navigate this form with a high degree of precision. The results provide a new tool for drug discovery and biological research, offering a way to predict the shapes of molecules that have never been seen before, based only on their chemical composition.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.