🤖 AI

Natural-Language-Guided Generator-Agnostic Shortlisting for Protein Binder Design

This paper demonstrates that large language models can generate interpretable, multi-metric ranking policies to effectively shortlist top protein binder candidates from large design pools, outperforming single-feature baselines by leveraging precomputed structural and interface quality scores.

Original authors: Gyubok Lee, Kiwoong Yoo, Jimin Seo, Kyunghoon Hur, Edward Choi

Published 2026-08-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gyubok Lee, Kiwoong Yoo, Jimin Seo, Kyunghoon Hur, Edward Choi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern quest to design new medicines, scientists are increasingly turning to computers to invent proteins from scratch. These custom-made molecules, known as binders, are designed to latch onto specific disease-causing targets, much like a key fitting into a lock, to stop a virus or repair a broken cell. The process of creating them has become remarkably fast; powerful computer programs can now generate thousands of potential binder designs in a single run. However, a significant bottleneck remains: while computers can produce these designs in vast numbers, the physical laboratories that test them can only handle a tiny fraction. It is expensive and time-consuming to test every single candidate in a lab, so researchers must choose a small, high-quality group from the thousands generated. The central challenge is no longer just making the designs, but figuring out which few among the thousands are actually worth testing.

This is the problem a team of researchers set out to solve. Instead of trying to build a better machine to create the proteins, they focused on the step that comes after: the selection process. They asked whether a large language model, a type of artificial intelligence known for understanding and generating human language, could act as a smart filter. Their goal was to see if these AI systems could look at a list of pre-calculated scores for each candidate—scores that estimate how stable the protein is and how well it might stick to its target—and then write a custom rule to rank them. Rather than relying on a single, rigid formula that treats every protein target the same, they wanted to see if the AI could learn to weigh different clues differently depending on the specific job at hand.

The researchers gathered data from ten different protein targets, each with a pool of computer-generated binder designs that had already been tested in real labs. For every design, they had a set of seventeen different numbers describing its predicted quality, such as how confident the computer was that the structure would hold together or how well the surfaces of the two molecules seemed to fit. They then asked the AI to look at these numbers and propose a ranking policy. This policy was essentially a set of instructions telling the computer which numbers to pay attention to and how much importance to give each one. The AI could choose to focus on just one score, or it could combine several, assigning different weights to each to create a final score for every candidate.

The results showed that the AI-generated rules were effective. When the researchers used the AI to pick the top ten candidates from each pool, it successfully identified a higher percentage of the actual working binders compared to using the single best traditional score on its own. Specifically, the AI's method found the correct binders about 59 percent of the time when looking at the top ten picks, a modest but meaningful improvement over the best single-score method, which found them about 57 percent of the time. The study suggests that the AI did not just guess; it learned to combine different types of evidence. It consistently relied on a core set of confidence scores from the most reliable prediction tools, but it also added specific adjustments based on the unique characteristics of each protein target. For some targets, it placed extra weight on how well the shapes of the molecules fit together, while for others, it prioritized different stability measures.

Crucially, the study found that the AI's ability to adapt to each specific target was key. When the AI was allowed to look at the specific pool of candidates for a single target and adjust its rules accordingly, it performed even better at ranking the candidates in the correct order, ensuring that the very best designs were at the very top of the list. This approach did not replace the existing tools used to generate the scores; instead, it acted as a flexible layer on top of them. The researchers concluded that using an AI to synthesize these ranking rules offers a practical and understandable way to narrow down massive lists of candidates. By creating a decision layer that can interpret a mix of different signals, scientists can make better choices about which designs to move forward to the lab, potentially speeding up the discovery of new treatments without needing to change the underlying design machinery.

Technical Summary: Natural-Language-Guided Generator-Agnostic Shortlisting for Protein Binder Design

1. Problem Statement

Modern de novo protein binder design workflows (e.g., RFdiffusion, ProteinMPNN) coupled with structure-prediction filters (e.g., AlphaFold2, Boltz-2) can generate large pools of candidate binders. However, wet-lab validation capacity remains a severe bottleneck, necessitating an efficient "shortlisting" phase to select a small top- $K$ subset for experimental testing.

Current practices rely on fixed thresholds, single-score rankings (e.g., ipTM, pDockQ2), or hand-tuned combinations of structural confidence and interface-quality proxies. These workflow-specific filters often fail to fully resolve the final selection problem, particularly when aggregating candidates from heterogeneous sources. The authors posit a gap in the literature: a need for a post-generation, generator-agnostic shortlisting layer that can synthesize ranking policies from precomputed proxy scores without requiring re-generation of candidates or retraining of generative models.

2. Methodology

The authors frame binder shortlisting as a policy synthesis problem. Given a fixed pool of candidate binders for a target protein and a shared panel of precomputed proxy scores, the goal is to learn a ranking policy $\pi$ that selects the top- $K$ candidates to maximize the recall of experimentally verified binders.

Data and Features:

Datasets: The study utilizes a development set (11 targets from BoltzGen) and a held-out evaluation set comprising 10 targets (including EGFR, Nipah, RBX1, TREM2, and pMHC minibinders) and a specific 3-target subset (Nipah, RBX1, TREM2).
Proxy Score Panel: A fixed set of 17 features is computed for all candidates prior to shortlisting. These include:
- Model-native confidence: ipTM, pTM, pLDDT, and interface PAE from AF2-Multimer, Boltz-2, and Protenix/PXDesign.
- Interface descriptors: Rosetta interface $\Delta G$ , buried SASA, and shape complementarity computed on predicted complexes.
- Specialized metrics: Boltz-2 ipSAE and pDockQ2.

Policy Space:
A policy is defined as a triple $(F, w, g)$ , where $F$ is a subset of features, $w$ are integer weights, and $g$ is a scoring function. The executor normalizes selected features within the target pool, applies direction correction (higher-is-better or lower-is-better), computes a weighted sum, and returns the top- $K$ candidates.

Evaluation Strategies:
The paper compares four categories of shortlisting methods:

Fixed Heuristics: Single-feature rankings (e.g., sorting solely by Protenix binder ipTM).
Supervised ML: Logistic Regression (LR) and XGBoost models trained on the development split to predict binder probability.
Global LLM Policies: Large Language Models (LLMs) generate a single ranking rule based on development-set statistics and feature descriptions, applied unchanged to all held-out targets.
Target-Conditioned LLM Policies: LLMs generate a specific ranking rule for each held-out target, utilizing unlabeled score distribution statistics of the current candidate pool in addition to development context.
- Iterative Variants: LLMs receive feedback from prior policy evaluations (accepted/rejected policies and their metrics) to refine the ranking rule.

The study evaluates performance using Recall@10, Hit@10 (Precision@10), and NDCG@10, averaging results over five independently sampled LLM policies to reduce variance.

3. Key Contributions

Formulation of Post-Generation Shortlisting: The authors define a standardized task for selecting final candidates from fixed pools using a common 17-feature proxy panel, decoupling shortlisting from the generative pipeline.
Controlled Comparison: A rigorous benchmark comparing fixed heuristics, supervised ML baselines, and LLM-generated policies (both global and target-conditioned) on identical held-out candidate pools.
LLM as an Interpretable Decision Layer: The demonstration that LLMs can synthesize interpretable, multi-feature ranking policies that combine heterogeneous proxy metrics (AF2, Boltz-2, Rosetta, Protenix) to prioritize binders.

4. Results

10-Target Held-Out Split:

Baseline Performance: The strongest single-feature fixed baseline is Protenix binder ipTM, achieving a Recall@10 of 0.571.
LLM Performance: Averaging five global iterative gpt-4o policies yields the highest LLM performance with a Recall@10 of 0.589, modestly outperforming the strongest single-feature baseline. Target-conditioned iterative gpt-4o achieves the highest NDCG@10 (0.523) among LLMs.
Supervised ML: Logistic Regression trained on the development set (LR-BG) achieves a Recall@10 of 0.537, performing competitively but generally below the iterative LLM approaches.

3-Target Held-Out Subset (Nipah, RBX1, TREM2):

Best Performance: Target-conditioned iterative gpt-5.4 achieves the strongest overall performance with a Recall@10 of 0.519 and NDCG@10 of 0.583.
Comparison: This outperforms the strongest fixed baseline (Protenix binder ipTM at 0.504) and the supervised LR baseline (0.507).
Stability: XGBoost baselines showed instability in this small setting, failing to recover any verified binders for Nipah or RBX1.

Policy Composition Analysis:

Generated policies do not rely on arbitrary rules but concentrate weights on a "backbone" of structure-confidence and interface-quality scores.
Across 50 target-conditioned gpt-4o policies, Boltz-2 ipTM and Protenix pair ipTM were selected in every policy.
Rosetta shape complementarity appeared in 40 policies, and AF2 ipTM in 27.
Ablation studies indicate that removing Protenix features causes the largest performance drop for gpt-4o, while removing Rosetta descriptors is most harmful for gpt-5.4, suggesting model-specific sensitivities to feature families.

5. Significance and Claims

The paper claims that LLM-generated ranking policies can serve as an interpretable post-generation decision layer for prioritizing binders from large, heterogeneous candidate pools.

Modest Improvement: The authors explicitly state that the improvements over strong single-feature baselines (e.g., Protenix binder ipTM) are "modest" (e.g., 0.589 vs. 0.571 Recall@10).
Complementarity: The generated policies do not replace predictor-native metrics. Instead, they combine complementary signals (AF2, Boltz-2, Rosetta, Protenix) into weighted rules.
Interpretability: Unlike "black-box" supervised models, the LLM policies output explicit feature weights and selection logic, allowing researchers to inspect the decision-making process.
Target Conditioning: While target-conditioned policies improve ranking quality (NDCG), they do not uniformly dominate global policies in terms of top- $K$ recovery (Recall), suggesting that label-free adaptation to score distributions is beneficial but not a panacea.

The work concludes that while strong predictor-native single-score rankings remain essential baselines, LLMs offer a viable mechanism for synthesizing multi-metric policies to optimize the limited wet-lab validation budget.