🧬 biology

Preference-based Antibody Expression Ranking: Scaling with Large-scale Weak Supervision

This paper proposes a unified preference-based learning framework that adapts Direct Preference Optimization to integrate scarce quantitative expression data with large-scale weak immunization supervision, enabling protein language models to effectively rank antibody expressibility in data-constrained settings.

Original authors: Josh Qixuan Sun, Morteza Babaie, Wenyang Hou, Mark Crowley, David Young

Published 2026-07-21

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Josh Qixuan Sun, Morteza Babaie, Wenyang Hou, Mark Crowley, David Young

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are a master chef trying to create the perfect new recipe for a dish that everyone loves. In the world of medicine, this "dish" is an antibody—a tiny, Y-shaped protein that acts like a superhero for our immune system, hunting down viruses and bacteria. Scientists are constantly trying to design new, custom-made antibodies to fight diseases, but there's a huge catch: just because a recipe looks good on paper doesn't mean it will actually cook up in the kitchen. Some designs turn into a delicious, high-yield meal, while others are a total flop, producing nothing at all.

The problem is that figuring out which recipes work is incredibly expensive and slow. It's like having to bake a cake to see if the ingredients mix right, but you can only afford to bake a few cakes a year. Scientists have a mountain of "maybe" recipes (millions of them) but very few "definitely works" recipes (only a handful). This paper tackles that exact bottleneck. It introduces a clever way to teach a computer how to rank these recipes, not by baking every single one, but by learning from a mix of a few known winners and a massive pile of "probably okay" guesses. The goal? To sort the good recipes from the bad ones so scientists can focus only on the most promising ones, saving time and money in the race to cure diseases.

The Recipe for Better Antibodies

Think of designing an antibody like trying to write a sentence in a language you've never fully mastered. You have a dictionary of 20 amino acids (the letters), and you need to string them together to make a sentence (the protein) that actually does something useful. The trouble is, most of the sentences you write might be gibberish or, worse, they might look like a sentence but fail to make any sense when you try to say them out loud. In the lab, this "making sense" part is called expression. If an antibody doesn't express well, it's like a recipe that calls for "a pinch of magic dust"—it just won't work in the real world.

For a long time, scientists have been stuck with a tiny cookbook. They have a few hundred recipes where they know exactly how much "food" (yield) the antibody produced. But they need to test thousands of new ideas. The paper suggests a brilliant workaround: why not use a massive library of "maybe" recipes to help teach the computer?

The Two-Step Magic Trick

The authors, Josh Qixuan Sun and his team, came up with a two-step strategy to teach a computer how to be a better recipe judge.

Step 1: The "Immersion" Phase
First, they took a smart computer model (a "protein language model") that already knows a lot about how proteins generally work. Then, they fed it a massive library of 4.2 million antibody sequences that were found in nature (specifically from camels and llamas). These sequences are like a huge pile of "probably okay" recipes. We don't know exactly how well they cook, but we know they exist in nature, so they are likely safe and stable. The computer reads through all 4.2 million of them, learning the "vibe" of a good antibody without needing to know the exact yield. It's like a student reading a million mystery novels to get a feel for the genre before trying to write their own.

Step 2: The "Taste Test" Phase
Next, the team introduced a tiny, high-quality cookbook called Exp-1K. This book has only 1,254 recipes, but for each one, they know the exact score: how much protein it produced. Some produced a lot (high yield), some produced a little, and about 27% produced nothing at all (zero yield).

Here is where the magic happens. Instead of just asking the computer, "How good is this recipe?" (which is hard to get right with so little data), they asked a different question: "If you had to pick between Recipe A and Recipe B, which one is better?"

They used a technique called Direct Preference Optimization (DPO). Imagine you are training a dog. You don't just say "Good dog" or "Bad dog." You hold up two treats and say, "I like this one more than that one." The computer learns to rank the antibodies.

Strong Supervision: If Recipe A made 100 mg of protein and Recipe B made 10 mg, the computer learns: "A is better than B."
Weak Supervision: If Recipe C is from the big pile of 4.2 million (we don't know its score, but it exists in nature) and Recipe D made zero protein, the computer learns: "C is probably better than D."

By mixing these two types of lessons, the computer learns to rank thousands of new designs, putting the most promising ones at the top of the list.

What They Found

The team tested their method on a diverse set of data. They compared their new "preference" method against older ways of teaching computers, like simple regression (guessing a number) or just looking at the data without any special training.

The results were clear:

The Two-Step Team Wins: The method that used both the massive "maybe" library (Step 1) and the "taste test" ranking (Step 2) consistently outperformed everything else. It was better at predicting which antibodies would actually work and better at ranking them from best to worst.
Zero-Shot is Weak: If they just used the computer without any of this special training (zero-shot), it performed terribly. It couldn't tell the difference between a good recipe and a bad one. This suggests that just knowing the "alphabet" of proteins isn't enough; you need to learn the specific "dialect" of antibody expression.
More Data is Better: They tested what happened when they fed the computer more and more of the 4.2 million "maybe" recipes. The more data they added, the better the computer got. It didn't get confused; it got smarter. This suggests that the "weak" data from the big library acts like a safety net, keeping the computer from making wild guesses based on the tiny amount of perfect data.

What They Ruled Out

The paper also tested some ideas that turned out to be dead ends:

Just reading the big library isn't enough: If they only did Step 1 (reading the 4.2 million sequences) and then tried to use a standard "guess the number" method on the small cookbook, it didn't work well. The computer learned the language but didn't learn how to rank the recipes.
Too much "weak" data can drown out the "strong" data: They found that if they didn't balance the training carefully, the massive pile of "maybe" recipes could overwhelm the tiny pile of "definite" recipes. They had to tune a dial (called $\alpha$ ) to make sure the computer paid enough attention to the high-quality data while still learning from the massive library.

The Bottom Line

This paper suggests that we don't need to wait for millions of perfect lab experiments to design better antibodies. Instead, we can use a clever mix of a few high-quality experiments and a mountain of "probably okay" natural data to teach computers how to rank designs.

The authors show that this approach is a practical and robust way to solve the problem of scarce data. It's not a magic wand that solves everything instantly, but it suggests that by changing how we ask the computer to learn (focusing on preferences rather than just numbers), we can make antibody design faster, cheaper, and more reliable. For a curious teenager, think of it as upgrading from a chef who tastes every single dish to a chef who has a super-intelligent sous-chef that can taste-test thousands of dishes at once and tell you exactly which ones to serve.

Technical Summary: Preference-based Antibody Expression Ranking

Problem Statement
Antibody expression ranking is a critical bottleneck in therapeutic development, yet computational modeling is severely hindered by the scarcity of labeled experimental data. Existing public benchmarks typically contain only hundreds of sequences with limited diversity in length and yield distributions, failing to reflect the complexity of real-world design. Furthermore, standard supervised learning approaches struggle to generalize when labeled data is sparse. Conversely, industrial discovery workflows generate millions of antibody sequences via immunization (e.g., from camelids) that lack quantitative expression measurements but serve as a source of "weak positive supervision," as empirical evidence suggests high expressibility rates for these sequences compared to AI-generated or transgenic mouse-derived candidates. The core challenge is to design a framework that can effectively integrate scarce, high-quality quantitative data with large-scale, unlabeled weak supervision to optimize antibody expressibility.

Methodology
The authors propose a unified preference-based learning framework that adapts Direct Preference Optimization (DPO) to protein language models (pLMs). The approach consists of two primary stages:

Domain-Alignment (Continual Pretraining): The model undergoes continual pretraining using masked language modeling (MLM) on a large-scale dataset of immunization-derived camelid sequences (Camel-4M, 4.2M sequences). This stage, referred to as "evo-tuning," adapts general pLMs to the specific sequence distribution of antibodies, establishing a robust representation space before any expression-specific supervision is applied.
Preference Optimization (DPO): The framework utilizes DPO to jointly learn from two heterogeneous supervision sources:
- Strong Supervision: Quantitative yield data from a proprietary dataset (Exp-1K, 1,254 sequences) is converted into pairwise preferences. A pair $(y_w, y_l)$ is formed if the yield difference exceeds a margin $\delta$ (set to 30 mg/L).
- Weak Supervision: Immunization-derived sequences (treated as expressible) are paired against sequences with zero measured yield (non-expressible) from the strong supervision set.
- Union-Masked Log-Likelihood Approximation: Since pLMs rely on pseudo-log-likelihood (PLL) rather than autoregressive likelihood, computing exact log-probabilities for every position is computationally expensive. The authors introduce a union-masked approximation where differing positions between two sequences are jointly masked and evaluated in a single forward pass. This significantly reduces computational cost during preference optimization.
- IMGT Alignment: To handle variable-length antibody sequences, sequences are aligned using IMGT numbering via ANARCI software, ensuring consistent construction of preference pairs across the sequence space.

The training objective maximizes the likelihood of preferred sequences relative to less preferred ones while regularizing against deviations from the reference model (the domain-aligned pLM) using a KL-divergence penalty.

Key Contributions

Unified Framework: The paper develops a single training objective that integrates scarce quantitative yields and large-scale weak positive supervision, avoiding the need for separate task-specific heads or reward models.
Efficient DPO for pLMs: The authors adapt DPO to masked language models by introducing a union-masked log-likelihood approximation and IMGT-based alignment, enabling efficient training on variable-length sequences.
Empirical Validation: The method is validated on a diverse internal dataset (Exp-1K) and a massive weak supervision corpus (Camel-4M), demonstrating consistent outperformance over standard regression and classification baselines across multiple protein language backbones (AntiBERTa2, IgBERT, ProtBERT).
Scaling Analysis: The study demonstrates that the framework scales effectively with increasing volumes of weak supervision, proving its ability to leverage unlabeled industrial data to mitigate labeled data scarcity.

Results
Evaluated on the Exp-1K test set, the proposed two-stage method (Continual Pretraining + Preference Optimization) consistently outperformed baselines, including zero-shot models, supervised regression, and supervised classification:

Performance Metrics: The method achieved superior results in both binary expressibility prediction (measured by MCC and Recall) and yield ranking (measured by Kendall's $\tau$ and Spearman's Correlation Coefficient). For instance, on the AntiBERTa2 backbone, the full framework achieved an MCC of 43.7 and a Kendall's $\tau$ of 46.3, significantly surpassing supervised baselines (MCC ~29.1, $\tau$ ~37.6).
Scaling Behavior: Performance improved steadily as the size of the weak supervision dataset increased from 1K to 4.2M sequences, provided that Stage I continual pretraining was utilized. Without Stage I, performance gains were less stable.
Generalization and Stability: The full framework exhibited a smaller generalization gap between validation and test sets compared to models trained without weak data or using standard regression. Optimization trajectories were smoother and less prone to oscillation, suggesting that large-scale weak supervision acts as a regularizer, anchoring the model in a biologically viable sequence space.
Ablation Findings: The study found that preference learning alone on small labeled datasets performed slightly worse than direct regression, highlighting the necessity of large-scale weak supervision to provide sufficient gradient signals. Additionally, filtering preference pairs based on sequence similarity reduced performance, indicating that the model benefits from learning across the full distribution of sequence space.

Significance
The paper claims that preference-based optimization offers a practical and robust solution for antibody design in data-constrained scenarios. By effectively learning from weak supervision, the framework addresses the limitations of traditional supervised learning when labeled data is scarce. The results suggest that preference learning acts as a "robust biophysical regularizer," enabling the extraction of useful signals from massive, unlabeled industrial datasets. This approach provides a scalable pathway for fine-grained antibody developability prediction, bridging the gap between evolutionary probability distributions and specific biophysical engineering goals without requiring explicit reward models.