💻 bioinformatics

pLM-bVAE: protein language model-enhanced binary representation improves quantum annealing-based protein sequence design

The paper introduces pLM-bVAE, a protein language model-enhanced binary variational autoencoder that improves binary encoding and sequence reconstruction accuracy, thereby enabling quantum annealing to generate higher-scoring and more diverse protein sequences compared to standard methods.

Original authors: Ueda, H. H., Tucs, A., Tsuda, K., Umetsu, M., Saito, Y.

Published 2026-11-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ueda, H. H., Tucs, A., Tsuda, K., Umetsu, M., Saito, Y.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Designing new proteins is like trying to find a single, perfect key in a room filled with billions of locks. Proteins are the molecular machines that power life, and their function is determined by the specific order of their building blocks, called amino acids. Because there are so many possible ways to arrange these blocks, searching for a protein with a desired trait—such as binding to a virus or glowing in the dark—is a task of overwhelming complexity. Scientists have turned to computers to help navigate this vast space, using a special type of computing called quantum annealing. This technology is designed to solve complex puzzles by finding the lowest energy state, which in this context corresponds to the best possible protein sequence. However, for a quantum computer to work on this problem, the protein sequences must be translated into a language the machine understands: a string of zeros and ones.

The challenge lies in this translation. If the conversion from a protein sequence to a string of zeros and ones is clumsy, the computer loses the subtle relationships between different proteins, and the final result is often a poor design. For some time, researchers have used a tool called a binary variational autoencoder to handle this translation. Think of this tool as a translator that compresses a long, complex sentence into a short code and then tries to write the sentence back from that code. While this method works, it was originally built for human language, not for the unique grammar of proteins. The researchers behind this study, led by Hiromi H. Ueda and Yutaka Saito, asked whether they could improve this translator by giving it a deeper understanding of proteins before it even began its work.

To answer this, the team developed a new system they call pLM-bVAE. They took the existing translation tool and enhanced it by feeding it information from a "protein language model." This model is a massive computer program that has already read millions of protein sequences and learned the rules of how they fit together, much like a child learning the grammar of a language by listening to thousands of conversations. By letting this pre-trained knowledge guide the translation process, the new system creates a much more accurate code for the quantum computer to work with. The researchers tested this approach on three different types of protein data: short chains that bind to the immune system, parts of antibodies that target cancer cells, and the outer shells of a virus used for gene therapy. In every case, the new system proved superior. It was able to take a protein sequence, turn it into a binary code, and then turn that code back into the original protein sequence with far greater accuracy than the old method.

The improvement was not just in accuracy but in the quality of the search itself. When the team used their new system to guide the quantum computer in designing new proteins, the results were striking. The computer generated a significantly higher number of high-performing protein sequences. More importantly, these new designs were more distinct from the examples the computer had seen during its training. In previous attempts, the computer often got stuck recycling familiar patterns, but the new system encouraged it to explore further, finding novel solutions that were both effective and unique. The researchers also observed that the search process became more focused. Instead of wandering aimlessly through the possibilities, the quantum computer followed a clearer path toward better solutions, a shift that was directly linked to the improved quality of the protein codes provided by the new system.

This work demonstrates that the way we translate biological information into digital code matters deeply. By borrowing the deep knowledge of how proteins naturally evolve and function, the researchers created a bridge that allows quantum computers to work much more effectively in the realm of biology. The study does not claim to have solved the problem of protein design entirely, but it shows a clear and measurable step forward. The new method suggests that combining the pattern-recognition power of modern artificial intelligence with the raw optimization speed of quantum computing can yield results that neither could achieve alone. As the field moves forward, this approach offers a promising path for engineering the next generation of medicines and biological tools, turning the vast, chaotic space of protein possibilities into a navigable landscape.

Technical Summary: pLM-bVAE: Protein Language Model-Enhanced Binary Representation Improves Quantum Annealing-Based Protein Sequence Design

Problem Statement
Quantum annealing (QA) offers a promising approach for navigating the vast combinatorial spaces of protein sequence design. However, applying QA to this domain requires mapping amino acid sequences into binary vectors for optimization and subsequently decoding optimized binary vectors back into amino acid sequences. The efficacy of this pipeline relies heavily on the quality of the binary encoding and the accuracy of sequence reconstruction. Previous studies utilized Binary Variational Autoencoders (bVAE) to bridge the sequence and binary spaces. However, standard bVAE architectures were originally developed for natural language modeling and do not explicitly incorporate protein-specific sequence representations. This limitation raises the possibility that existing methods may fail to optimally preserve relationships among protein sequences or achieve high-fidelity reconstruction, thereby limiting the performance of QA-based optimization.

Methodology
The authors propose pLM-bVAE, a novel architecture that integrates residue-level representations derived from pretrained protein language models (pLMs) into the bVAE framework.

Architecture: The model consists of a frozen pLM encoder and a trainable bVAE component. The pLM encoder (e.g., ESM2, ProtT5, Ankh) embeds full-length protein sequences into evolutionary-informed representation vectors. The bVAE then processes the vectors corresponding to specific variant regions (defined at the residue level for each dataset) to generate binary latent vectors and reconstruct the sequences.
Datasets: The method was evaluated on three public protein variant datasets:
1. MHC-I binding peptides (HLA-A*02:01).
2. Trastuzumab-scFv CDRH3 variants.
3. AAV capsid surface region variants.
Training and Optimization: The authors employed Bayesian optimization (using Optuna) to tune hyperparameters for the pLM-bVAE, including layer counts, learning rates, and dropout rates. They compared various pLM encoders (different sizes and architectures) to identify the optimal configuration.
QA Framework: The trained pLM-bVAE was integrated into a Multi-Objective Quantum Annealing (MOQA) framework. In this workflow:
1. BiLSTM predictors were trained to estimate objective function scores (e.g., binding affinity, viability) from sequences.
2. A Factorization Machine (FM) served as a surrogate model for the objective scores, converting the problem into a Quadratic Unconstrained Binary Optimization (QUBO) formulation.
3. A D-Wave Advantage quantum annealer optimized the QUBO to search for high-scoring binary vectors.
4. The resulting binary vectors were decoded into amino acid sequences using the pLM-bVAE decoder.

Key Contributions

Development of pLM-bVAE: The introduction of a hybrid architecture that leverages the evolutionary knowledge embedded in pLMs to enhance the binary representation capabilities of bVAE for protein sequences.
Systematic Evaluation: A comprehensive assessment of reconstruction accuracy (using Levenshtein distance and BLOSUM62 scores) and binary encoding performance (preservation of sequence-distance relationships via Spearman correlation of Hamming and Levenshtein/BLOSUM distances).
Integration with QA: The demonstration of how improved binary representations translate to tangible gains in QA-based protein sequence design, specifically in generating high-scoring, novel sequences.

Results

Reconstruction and Encoding Accuracy: On all three datasets, pLM-bVAE (specifically using the ESM2_650M encoder) demonstrated superior performance compared to the standard bVAE. It achieved higher sequence reconstruction accuracy (lower Levenshtein distance, higher BLOSUM62 scores) and better preservation of sequence relationships in the binary latent space. Hyperparameter optimization (pLM-bVAE HPO) further improved these metrics, with the peptide dataset showing near-perfect reconstruction (Levenshtein distance of 0 for almost all sequences).
Total Model Performance: A rank-normalized total performance score ( $S_{total}$ ) confirmed that pLM-bVAE (HPO) outperformed standard bVAE and other pLM-bVAE configurations. Notably, medium-sized models (ESM2_650M) outperformed larger variants (ESM2_3B), suggesting that increasing parameter size does not linearly improve performance in this specific context.
QA-Based Design Performance: When integrated into the QA framework, pLM-bVAE (HPO) generated a significantly larger number of high-scoring sequences (those exceeding the 10th highest score in the training/validation data) compared to bVAE. Furthermore, these high-scoring sequences exhibited greater dissimilarity (novelty) from the training data, indicating an improved ability to explore the sequence space effectively.
Optimization Dynamics: Analysis of the optimization process in the binary latent space revealed that QA using pLM-bVAE (HPO) followed a more directed search pattern. This was evidenced by larger Maximum Mean Discrepancy (MMD) values between initial and optimized distributions and a faster increase in the best oracle score over iterations, suggesting higher optimization efficiency.

Significance
The paper claims that pLM-bVAE provides an effective binary representation that addresses the limitations of standard bVAE in protein sequence design. By incorporating protein-specific evolutionary information, the method improves both the fidelity of sequence reconstruction and the preservation of sequence relationships in the binary latent space. Consequently, this leads to more efficient quantum annealing optimization, enabling the discovery of a greater number of high-performing, novel protein sequences. The study establishes that enhancing the encoding mechanism with pretrained language models is a critical step for improving the efficacy of quantum computing applications in protein engineering.

More like this