Deep Learning for Protein Complex Prediction and Design

This thesis leverages deep learning to advance protein complex prediction and design by developing specialized architectures that capture structural hierarchies and creating search algorithms to efficiently navigate sequence spaces for identifying interacting homologs and designing novel protein sequences.

The Gist

Imagine proteins as intricate, 3D puzzle pieces floating inside your body. To understand how life works, scientists need to know exactly how these pieces snap together to form larger machines called protein complexes. Sometimes, two different pieces join (a heterodimer), and sometimes two identical pieces join (a homodimer).

This thesis by Ziwei Xie tackles the problem of predicting how these pieces fit together and, conversely, how to design new pieces that will snap onto a specific target. The author uses three main "tools" (deep learning methods) to solve these puzzles.

Here is a simple breakdown of the three main contributions:

1. GLINTER: The "Handshake Detector"

The Problem: When two proteins meet, they touch at specific spots called "interfaces." Predicting exactly where they touch is like trying to guess which two people in a crowded room are about to shake hands, just by looking at their individual outfits.

The Solution (GLINTER):
Think of GLINTER as a super-smart detective that looks at two clues at once:

1. The Shape: It looks at the 3D shape of the individual proteins (like looking at the cut of their clothes).
2. The History: It looks at the "family history" of these proteins. If two proteins have evolved together over millions of years, they likely have a "handshake" pattern. GLINTER uses a special AI (a transformer) to read this evolutionary history.

The Result: By combining the physical shape with the evolutionary history, GLINTER can predict the handshake spots much better than previous methods. It works well for both identical twins (homodimers) and different partners (heterodimers). This helps scientists figure out how to assemble the puzzle pieces correctly.

2. ESMPair: The "Matchmaker for Lost Relatives"

The Problem: To predict how two different proteins interact, scientists need to find their "relatives" (homologs) from other species to see how they evolved together. However, in complex organisms (like humans/eukaryotes), there are many look-alike relatives (paralogs). It's like trying to match a specific person's twin from a different country, but there are 50 people who look exactly like them in that country. Traditional methods often pick the wrong "twin," leading to a bad prediction.

The Solution (ESMPair):
ESMPair is a new matching algorithm that uses a "Language Model" (an AI trained on millions of protein sentences).

• Instead of just looking at how similar the names (sequences) are, ESMPair looks at how the proteins "pay attention" to each other in their evolutionary history.
• Imagine you are trying to pair up dancers. Instead of just checking if they have the same shoe size, ESMPair listens to the music they both know and sees who naturally moves to the same rhythm.

The Result: ESMPair is much better at finding the correct evolutionary partners, especially for complex organisms where there are many look-alikes. When it feeds these correct pairings into the main prediction engine (AlphaFold-Multimer), the resulting 3D structures are significantly more accurate. It also works great for "cross-kingdom" pairs (like a human protein meeting a bacteria protein), which are usually very hard to predict.

3. RedNet: The "Custom Suit Designer"

The Problem: Sometimes, you don't just want to predict how proteins fit; you want to design a new protein that acts like a key to lock onto a specific target (like a drug). This is called "binder design." The challenge is making a key that fits the lock perfectly but doesn't fit any other similar locks nearby.

The Solution (RedNet):
RedNet is a design tool that works like a master tailor.

• The Skeleton: It starts with a fixed "skeleton" (the backbone) of the protein you want to design.
• The Fabric: It then decides which amino acids (the fabric) to use to cover that skeleton.
• The Contrastive Trick: This is the clever part. RedNet doesn't just try to make the suit fit the target. It uses a "contrastive" method: it asks, "Does this suit fit the target better than it fits a look-alike target?" It learns by comparing the "good fit" against the "bad fit."

The Result: RedNet designs proteins that are not only stable but also highly specific. They stick tightly to the intended target but ignore very similar "impostor" targets. This is crucial for making drugs that cure a disease without causing side effects by hitting the wrong protein.

Summary

In short, this thesis builds a toolkit for the future of biology:

1. GLINTER helps us see where proteins touch.
2. ESMPair helps us find the right evolutionary partners to make those predictions accurate.
3. RedNet helps us design new proteins that act as precise, custom-made keys for specific biological locks.

Together, these tools show that by combining deep learning with the rules of evolution and physics, we can better understand and engineer the molecular machines of life.

Technical Summary

Based on the provided thesis text, here is a detailed technical summary of the work presented in Chapter 3: Improved Protein Heterodimer Structure Prediction with Protein Language Models.

Problem Statement

Accurate prediction of protein complex structures, particularly heterodimers (complexes formed by two different protein chains), remains a significant challenge in computational structural biology. While AlphaFold2 revolutionized monomer structure prediction, its extension to complexes, AlphaFold-Multimer, faces a critical bottleneck: the construction of interolog Multiple Sequence Alignments (MSAs).

To predict a complex structure, AlphaFold-Multimer requires a joint MSA where homologous sequences from the two constituent chains are correctly paired across species. Traditional methods rely on heuristic strategies such as:

1. Phylogeny-based pairing: Grouping sequences by species and ranking them by similarity to the query, then pairing sequences of the same rank.
2. Genome co-localization: Pairing sequences based on their proximity in bacterial operons.

These methods struggle significantly with eukaryotic targets and cases involving paralogs (multiple similar sequences within the same species), often leading to incorrect pairings and poor structure prediction accuracy. The core problem addressed is how to automatically and effectively identify interacting homologs (interologs) to construct high-quality joint MSAs for heterodimer prediction, especially in difficult taxonomic domains.

Methodology: ESMPair

The thesis proposes ESMPair, a novel MSA pairing algorithm that leverages Protein Language Models (PLMs), specifically the pre-trained ESM-MSA-1b model, to identify and pair homologs.

The Pipeline:

1. MSA Generation: For a query heterodimer, individual MSAs are generated for each chain using JackHMMER against the UniProt database.
2. Species Grouping: Homologs in each MSA are grouped by species.
3. Attention-Based Scoring: The ESM-MSA-1b model is used to estimate column-wise attention scores between the query sequence and every homolog in the MSA.
   • The model computes a column attention weight matrix ($A \in \mathbb{R}^{N \times N}$) for each layer, head, and column.
   • These matrices are symmetrized and aggregated (summed) across layers, heads, and columns to produce a pairwise similarity matrix $S$.
   • The first row of $S$ ($S_1$) represents the similarity between the query and all hit sequences.
4. Ranking and Pairing: Within each species group, sequences are ranked based on their similarity score in $S_1$. Sequences from the two chains that share the same species and the same rank are paired and concatenated to form the interolog MSA.
5. Structure Prediction: The resulting paired MSA is fed into AlphaFold-Multimer to predict the complex structure.

Key Technical Distinction: Unlike previous methods that rely on sequence identity or genomic proximity, ESMPair utilizes the co-evolutionary signals captured in the attention maps of a large-scale protein language model to determine the likelihood of interaction between homologs.

Key Contributions

1. First PLM-based MSA Construction: This work represents the first application of pre-trained Protein Language Models to construct joint MSAs for protein complex prediction.
2. ESMPair Algorithm: The development of a simple yet effective pairing strategy that uses column attention scores from ESM-MSA-1b to resolve paralog ambiguity.
3. Ensemble Strategy: The demonstration that combining ESMPair with other pairing methods (e.g., Genome-based, Block Diagonalization) via an ensemble approach yields superior results compared to any single strategy.
4. Factor Analysis: A quantitative analysis linking MSA properties (diversity, depth, species count) and attention scores to prediction accuracy.

Results

The method was evaluated on three test sets: pConf70 (low confidence targets), pConf80, and DockQ49 (targets with low predicted accuracy).

• Performance Improvement: ESMPair consistently outperformed the default AlphaFold-Multimer pairing strategy and other baselines (Genome, Block Diagonalization).
  • On the pConf70 set, ESMPair achieved a Top-5 Best DockQ score of 0.259, compared to 0.234 for AlphaFold-Multimer (a ~10.7% relative improvement).
  • On the DockQ49 set, the Top-5 Best DockQ score improved from 0.247 (AF-Multimer) to 0.265 (ESMPair).
• Taxonomic Robustness:
  • ESMPair showed the most significant gains on Eukaryotic targets, where phylogeny-based methods struggle due to high paralog counts.
  • It demonstrated exceptional performance on cross-kingdom pairs (Eukaryote-Bacteria), outperforming baselines by a large margin (e.g., DockQ 0.394 vs. 0.314 for AF-Multimer).
• Low Confidence Targets: The relative improvement of ESMPair over AlphaFold-Multimer was negatively correlated with the predicted confidence score (pConf). The method provided the largest gains on difficult, low-confidence targets (pConf < 0.7), where improvements in DockQ reached up to 100% in some cases.
• Ensemble Benefits: Combining ESMPair with other strategies (e.g., ESMPair + Genome) further increased the success rate to 44.6% (Top-5 Best DockQ 0.277), and a three-strategy ensemble reached 46.8% success rate.
• Case Studies:
  • ESMPair correctly predicted the binding sites and ligand orientations for targets where AlphaFold-Multimer failed (e.g., 7VSI, 7AQU).
  • In the case of the NFM–INA intermediate filament heterodimer, ESMPair correctly predicted a four-helix bundle structure consistent with experimental evidence, whereas AlphaFold-Multimer predicted separated coils.

Significance and Claims

The paper claims that ESMPair effectively addresses the bottleneck of MSA pairing in heterodimer prediction by leveraging the rich co-evolutionary information encoded in Protein Language Models.

• Generalizability: The method is described as a general and automatic algorithm that does not rely on domain-specific heuristics (like operon location), making it robust across different taxonomic domains, particularly where traditional methods fail (Eukaryotes).
• Complementarity: The work highlights that different pairing strategies capture different signals; therefore, an ensemble approach is the most effective way to identify interologs.
• Impact: By improving the quality of input MSAs, ESMPair significantly enhances the accuracy of downstream structure prediction models like AlphaFold-Multimer, particularly for challenging targets that are currently difficult to model. The authors note that this approach has profound implications for biological applications relying on high-quality complex structure predictions.

The thesis concludes that while ESMPair is a significant step forward, the field should continue to explore ways to leverage PLMs for MSA construction and selection, and that these methods can be applied to improve other MSA-based applications in structural biology.