📄 molecular biology

FlexRibbon: Joint Sequence and Structure Pretraining for Protein Modeling

FlexRibbon is a novel protein foundation model that jointly pretrains on amino acid sequences and 3D structures using a combination of masked language modeling and diffusion-based denoising, achieving state-of-the-art performance across diverse tasks—particularly in mutation-rich regimes where traditional MSA-based methods struggle—without relying on multiple sequence alignments.

Original authors: Zhu, J., Shi, Y., Bi, R., Jin, P., Liu, C., Zhang, Z., Huang, H., Guo, Z., Hu, P., Ju, F., Huang, L., Tai, X., Li, C., Gao, K., Wei, X., Xia, H., Zhang, J., Min, Y., Wang, Z., Wang, Y., He, L., Liu, H

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhu, J., Shi, Y., Bi, R., Jin, P., Liu, C., Zhang, Z., Huang, H., Guo, Z., Hu, P., Ju, F., Huang, L., Tai, X., Li, C., Gao, K., Wei, X., Xia, H., Zhang, J., Min, Y., Wang, Z., Wang, Y., He, L., Liu, H., Qin, T.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine the world of proteins as a massive, bustling city. For a long time, scientists trying to understand this city had to choose between two very different maps.

The first map was like a textbook of grammar. It only looked at the sequence of letters (amino acids) that make up a protein, ignoring what the city actually looks like. It was great at understanding the "language" of life but often got lost when trying to figure out the 3D shape of the buildings.

The second map was like a crowdsourced survey. It relied on looking at thousands of similar cities (evolutionary cousins) to guess what a new one would look like. This was incredibly accurate for standard buildings, but if you tried to map a brand-new, weirdly shaped skyscraper or a city that had changed its layout too much, the survey failed because there were no similar examples to compare it to.

Enter FlexRibbon, a new kind of mapmaker that decided to stop choosing between the textbook and the survey. Instead, it learned to read the grammar and look at the blueprints at the same time.

The Secret Sauce: A Two-Way Street

Most previous models were like a one-way street: they took a sequence of letters and tried to guess the shape. FlexRibbon built a two-way street. It learned that the letters determine the shape, but the shape also tells you what the letters should be.

To teach this, the researchers used a clever training game. They took a protein, scrambled its 3D shape with noise (like shaking a snow globe), and asked the model to clean it up. At the same time, they hid some of the letters in the sequence and asked the model to guess them, but with a twist: it had to use the messy 3D shape as a hint to make the right guess. This forced the model to understand that the sequence and the structure are locked together, like a lock and key that can't be separated.

Why This Matters: The "Messy" Parts

The real magic happens in the messy parts of the city. In biology, some proteins have flexible, floppy arms (like antibody loops) or change shape when they grab onto something. The old "survey" models (which rely on finding similar cousins) often get stuck here because these floppy parts change too much to have a clear pattern in the evolutionary data.

FlexRibbon, however, handles these tricky regions differently. Instead of relying on a "crowd of cousins" to find a pattern, it bypasses the need for that survey entirely. By directly modeling the link between the individual sequence and its structure, it can predict these flexible, highly mutated regions with surprising accuracy, even when evolutionary signals are weak or missing.

The Proof: 12 Different Challenges

The team didn't just say it worked; they put FlexRibbon through a gauntlet of 12 different tasks to see if it could handle real-world problems. Here is what they found:

Antibody Design: When asked to design antibodies (the body's defenders) to stick to viruses, FlexRibbon succeeded 61.3% of the time for standard antibodies and 51.1% for nanobodies (tiny antibodies). This was a big jump over the previous best methods, which were stuck around 46.7% and 44.0% respectively.
Peptide Binding: For predicting how small protein chains (peptides) stick to larger proteins, FlexRibbon hit a success rate of 91.4%, beating the next best method by 7.0% to 10.2%.
Drug Discovery: When trying to guess how well a drug molecule would stick to a protein, FlexRibbon achieved a correlation score of 0.848 and an error rate of 1.150, outperforming all other methods tested.
Shape-Shifting: The model could also predict how a protein changes shape when a drug binds to it, achieving a structural similarity score (TM-score) of 0.985 for one test case, which is incredibly close to the real thing.

What It's NOT

It's important to know what FlexRibbon doesn't do. It doesn't rely on the "crowdsourced survey" (Multiple Sequence Alignments) that older models used to make predictions. If you give it a protein with very few evolutionary cousins, the old models might struggle because their survey has no data, but FlexRibbon succeeds by relying on its internal understanding of the direct sequence-structure link, allowing it to navigate regions where alignment signals are weak.

The Bottom Line

The paper suggests that by teaching a model to learn sequences and structures together, rather than one after the other, we get a much more flexible and accurate tool. It's not just a better guesser; it's a better designer. Whether it's creating new antibodies to fight diseases or figuring out how drugs fit into their targets, FlexRibbon shows that when you let the sequence and the structure talk to each other, the whole system works better.

The authors measured these results on specific benchmarks like the CASP15 challenge and the PoseBusters V1 dataset, finding that FlexRibbon consistently hit new high scores, especially in the tricky, mutation-heavy areas where other methods often stumble. It's a step toward a future where we can design proteins from scratch with the same ease as writing a sentence.

Technical Summary: FlexRibbon

Problem Statement

Protein foundation models (PFMs) have advanced rapidly but generally fall into two dominant paradigms with distinct limitations. Sequence-based language models (e.g., ESM-2, ProtT5) capture sequence semantics at scale but lack physical relevance and three-dimensional geometric information, limiting their ability to model the structural basis of protein function. Conversely, MSA-based predictors (e.g., AlphaFold 2/3) achieve high accuracy by exploiting evolutionary couplings but rely heavily on homologous sequences. This dependence makes them less reliable in regimes with sparse alignments, highly mutated regions, or flexible interfaces (such as antibody CDR loops and intrinsically disordered regions). Furthermore, existing joint models often treat structure as an auxiliary signal rather than modeling it bidirectionally, or suffer from memory bottlenecks when attempting full-atom structural learning alongside sequence semantics.

Methodology

FlexRibbon is a 3-billion-parameter pretrained protein model designed to jointly learn from amino acid sequences and three-dimensional structures without requiring Multiple Sequence Alignments (MSAs).

1. Pretraining Strategy

The core innovation is a unified pretraining objective that combines Masked Language Modeling (MLM) with Diffusion-based Denoising. This enables bidirectional sequence-structure learning:

Structure-Informed Masked Language Modeling (SIMLM): The model integrates diffusion noise into structural representations during sequence recovery. It employs three training modes: (1) Sequence-to-Structure (standard diffusion), (2) Coupled Perturbation (masking residues and adding noise to their local structures), and (3) Sequence-Masked Global Perturbation (masking residues while applying noise to all structures). This encourages robust learning of the bidirectional relationship between sequence identity and 3D conformation.
Diffusion Pretraining: The model uses a variance-exploding diffusion process to denoise 3D coordinates of heavy atoms. It enforces rotational invariance via data augmentation (random SO(3) rotations) rather than heavy equivariant architectures, and removes translational freedom by centering structures on their center of mass.

2. Architecture

FlexRibbon utilizes a hierarchical three-stage architecture to balance efficiency and expressivity:

Sequence Module: A Transformer encoder that jointly embeds protein residues and small-molecule atoms. It incorporates 2D covalent bond features via a learned projection, allowing the model to recover bonding patterns from atom identities.
Coarse-Grained Structure Module: A Diffusion Transformer (DiT) that denoises coordinates at the residue level for proteins and atom level for small molecules, conditioned on sequence embeddings.
All-Atom Structure Module: A second DiT that refines the coarse-grained outputs into full-atom resolution. It uses learnable attention biases for covalent bonds to ensure chemical validity, reconciling global residue context with detailed atomic interactions.

3. Training and Loss Functions

The training objective integrates four components:

Diffusion Loss ( $L_{MSE}$ ): Denoising loss for structure reconstruction.
MLM Loss ( $L_{MLM}$ ): Sequence recovery via masked residue prediction.
Distance Loss ( $L_{Dist}$ ): Regularizes predicted inter-residue distances to maintain realistic tertiary geometry.
Smooth-lDDT Loss ( $L_{smooth-lDDT}$ ): Aligns training with local geometric accuracy metrics.
Adaptive Weighting: A confidence-weighted diffusion loss scales contributions based on pLDDT-derived sigmoid weights, reducing noise from low-confidence regions (e.g., predicted structures from AlphaFold 2) while emphasizing reliable signals.

The training proceeds in three progressive stages (A, B, C) increasing sequence length (384 $\to$ 768 $\to$ 1024 residues) and introducing MLM and confidence head training in later stages.

Key Contributions

Novel Pretraining Strategy: FlexRibbon unifies protein structure prediction and design by combining MLM with diffusion-based denoising, learning a bidirectional sequence-structure mapping rather than a one-way sequence-to-structure mapping.
Hierarchical Modeling: It introduces a strategy that balances scalable capacity across sequence and structure representations, overcoming the memory bottlenecks of previous full-atom models and enabling effective large-scale structural learning.
Co-design Capability: The model enables the co-design of protein sequence and structure, particularly excelling in flexible and highly mutated regions (e.g., antibody/nanobody CDR loops) where MSA-based methods struggle.
Broad Generalization: It demonstrates consistent gains across 12 diverse tasks, showing that sequence-structure pretraining transfers broadly beyond simple protein folding.

Experimental Results

FlexRibbon was evaluated on three task families: flexible interface prediction/design, intermolecular interaction prediction, and protein function prediction.

Flexible Interfaces: On antibody and nanobody interface prediction (SAb23H2), FlexRibbon achieved success rates of 61.3% (antigen-antibody) and 51.1% (antigen-nanobody), outperforming IgGM by 14.6% and 7.1% respectively. In protein-peptide interface prediction (FoldBench), it achieved a 91.4% success rate, surpassing AlphaFold 3 and PepGLAD.
Antibody/Nanobody Design: FlexRibbon achieved an amino acid recovery (AAR) of 41.4% for antibodies and 21.8% for nanobodies, with success rates of 46.0% and 43.7%, respectively, outperforming baselines like dyMEAN and IgGM.
Intermolecular Interactions:
- Docking: On the PoseBusters V1 benchmark, FlexRibbon achieved a 71.82% success rate (random-1) and 78.70% (oracle), reaching parity with MSA-based methods like AlphaFold 3 and significantly outperforming single-sequence baselines.
- Conformational Change: In ligand-induced conformational change prediction, FlexRibbon achieved a TM-ens score of 0.889 (zero-shot), improving upon ESMDiff by 0.038.
- Binding Affinity: On CASF-2016, FlexRibbon achieved the best RMSE (1.150) and Pearson correlation (0.848) among compared baselines.
Function Prediction: In Gene Ontology (GO) and Enzyme Commission (EC) tasks, FlexRibbon achieved F1 scores of 0.891 (EC), 0.539 (GO-BP), 0.694 (GO-MF), and 0.560 (GO-CC), outperforming sequence-only and hybrid baselines.
Structure Prediction: On CASP15, FlexRibbon achieved a TM-score of 0.703, surpassing ESMFold and ESM3.
Confidence Estimation: The model's predicted pTM scores showed a strong correlation (R = 0.89) with observed TM-scores on CASP15.

Significance and Claims

The paper claims that FlexRibbon establishes a new state-of-the-art performance across 12 diverse tasks, with particularly strong gains in mutation-rich settings where MSA-based methods often fail. The authors assert that the results highlight the consistent advantages of joint sequence-structure pretraining, demonstrating that such an approach enables the co-design of sequence and structure and captures flexible conformations critical for biological function. By successfully modeling bidirectional dependencies without relying on MSAs, FlexRibbon is presented as a general-purpose foundation model that extends the benefits of protein modeling beyond folding to complex design and interaction prediction tasks.