Property-driven Protein Inverse Folding With Multi-Objective Preference Alignment

This paper introduces ProtAlign, a multi-objective preference alignment framework that fine-tunes pretrained inverse folding models to simultaneously optimize diverse developability properties like solubility and thermostability while preserving structural designability, resulting in the enhanced MoMPNN model for practical protein sequence design.

The Gist

Here is an explanation of the paper "Property-Driven Protein Inverse Folding with Multi-Objective Preference Alignment" (ProtAlign), translated into simple language with creative analogies.

The Big Picture: The "Protein Architect" Problem

Imagine you are an architect. You have a specific blueprint for a building (the protein backbone or structure). Your job is to choose the right bricks, wood, and steel (the amino acid sequence) to build it.

In the world of biology, this is called Inverse Folding. Usually, scientists have been very good at one thing: making sure the building stands up and looks exactly like the blueprint. This is called Designability.

But here's the catch: Just because a building stands up doesn't mean it's a good place to live.

• Is it waterproof? (Solubility - will it dissolve in water?)
• Will it survive a heatwave? (Thermostability - will it melt?)
• Is it easy to build? (Expression - can the factory make it?)

For a long time, protein designers had to choose: "Do I want a building that looks perfect, or one that is durable?" They couldn't have both easily.

The Old Ways (The "Clunky" Solutions)

Before this paper, scientists tried to fix this with three messy methods:

1. Post-Hoc Mutation: Build the perfect building, then try to swap out a few bricks to make it waterproof. Problem: It's like trying to fix a leaky roof by throwing random patches on it. You might fix the leak, but you might break the wall.
2. Inference-Time Biasing: Tweak the instructions while building to favor waterproof bricks. Problem: It's like driving with the steering wheel tied to the left. You might get to the destination, but the ride is shaky and requires a very skilled driver (expert tuning).
3. Retraining: Teach the architect a new rule: "Only build waterproof houses." Problem: Now the architect forgets how to build houses that actually stand up. They become too specialized.

The New Solution: ProtAlign (The "Smart Coach")

The authors introduce ProtAlign, a new framework that acts like a Smart Coach for the protein architect.

Instead of forcing the architect to choose between "Standing Up" and "Being Durable," ProtAlign teaches the architect to balance both at the same time. It uses a technique called Preference Alignment.

How It Works (The "Taste Test" Analogy)

Imagine the architect (the AI model) generates 10 different versions of a protein sequence for a single blueprint.

1. The Rollout: The architect creates 10 different designs.
2. The Judges: We use computer programs (predictors) to grade these designs on two things:
   • Grade A: Does it match the blueprint? (Designability)
   • Grade B: Is it soluble and heat-resistant? (Developability)
3. The Pairing: The system looks at the designs and pairs them up.
   • Design X: Great blueprint match, but melts easily.
   • Design Y: Good blueprint match, AND it's heat-resistant.
   • The Decision: The system tells the architect, "I prefer Design Y over Design X."
4. The Learning: The architect learns from these "Win vs. Lose" pairs. It doesn't just memorize the answer; it learns the logic of why Y is better.

The Secret Sauce: The "Flexible Margin"

Here is the tricky part. Sometimes, a design is great at being heat-resistant but slightly worse at matching the blueprint. If the coach is too strict, the architect might stop trying to be heat-resistant because it hurts the blueprint score.

ProtAlign uses a Flexible Preference Margin.

• Analogy: Imagine a parent grading a student. If the student gets an A in Math but a B in Art, the parent says, "Great job on Math, but let's try to improve Art."
• The Flexibility: If the student is really good at Art but only slightly worse at Math, the parent says, "That's a win! We'll accept a tiny drop in Math to get that huge gain in Art."
• In the Paper: This "margin" allows the AI to accept a small trade-off in one area to get a big win in another, preventing the two goals from fighting each other to the death.

The Result: MoMPNN

The authors applied this coach to a famous protein designer called ProteinMPNN. The result is a new model called MoMPNN.

What did they find?

• It didn't forget how to build: MoMPNN still builds proteins that match the blueprints perfectly (Designability is preserved).
• It learned to be durable: The new proteins are much better at surviving heat and dissolving in water (Developability is improved).
• It works everywhere: Whether they were redesigning old proteins, creating brand new ones from scratch, or designing "stickers" (binders) to catch viruses, MoMPNN beat all the previous specialists.

Why This Matters

Think of this as moving from Craftsman to Engineer.

• Before: You had to hire a specialist to make a protein stand up, and another specialist to make it durable, and hope they could work together.
• Now (ProtAlign): You have one AI that understands the whole picture. It designs proteins that are not just theoretically correct, but practically useful for real-world medicine and industry.

In short, ProtAlign is the tool that finally lets scientists design proteins that are both structurally perfect and ready for the real world, without needing to be experts in every single chemical property themselves.

Technical Summary

Here is a detailed technical summary of the paper "Property-Driven Protein Inverse Folding with Multi-Objective Preference Alignment."

1. Problem Statement

Protein inverse folding involves generating amino acid sequences that fold into a specific target backbone structure. While existing models (e.g., ProteinMPNN) excel at designability (recovering sequences compatible with a target structure), real-world applications require proteins to also possess specific developability properties, such as solubility, thermostability, and high expression levels.

Current approaches to incorporate these properties face significant limitations:

• Post-hoc mutation: Generating sequences first and then mutating them is inefficient; beneficial mutations are sparse and hard to identify.
• Inference-time biasing: Adjusting sampling probabilities or using reward signals often introduces instability and requires delicate hyperparameter tuning to balance property optimization against structural fidelity.
• Retraining on subsets: Training models on datasets filtered for specific properties (e.g., only soluble proteins) often leads to a trade-off where improving one property degrades designability or generalizability to other tasks.

The core challenge is to align a pretrained inverse folding model with multiple, often competing objectives (designability vs. various developability metrics) without sacrificing structural fidelity.

2. Methodology: ProtAlign Framework

The authors propose ProtAlign, a multi-objective preference alignment framework that fine-tunes pretrained inverse folding models using a Semi-Online Direct Preference Optimization (DPO) strategy.

Key Components:

1. Semi-Online Training Regime:

   • Instead of pure online reinforcement learning (which is computationally expensive) or pure offline DPO (which relies on static datasets), ProtAlign uses an iterative loop.
   • Rollout: The current policy generates multiple sequence candidates for a given backbone at a high temperature to encourage diversity.
   • Annotation: These sequences are evaluated by in silico property predictors (e.g., Protein-Sol for solubility, TemBERTure for thermostability, ESM for evolutionary plausibility).
   • Preference Construction: Pairwise preference datasets are constructed for each property by comparing high-scoring sequences ($y_w$) against lower-scoring ones ($y_l$), filtering out ambiguous pairs based on a score threshold.
   • Training: The model is updated offline on these generated preference pairs before the next rollout.

2. Multi-Objective DPO with Flexible Preference Margin:

   • Standard DPO optimizes for a single preference. ProtAlign extends this to $K$ objectives.
   • The loss function incorporates a flexible preference margin ($m_k$) to handle conflicts between objectives.
   • Mechanism: If a "winning" sequence ($y_w$) for property $k$ performs poorly on an auxiliary property $k'$, the margin required to prefer $y_w$ is reduced. This prevents the optimization from overemphasizing one property at the expense of others.
   • Mathematical Formulation: The loss $L_{MO}$ integrates weighted rewards and the adaptive margin derived from other property scores, allowing the model to find a Pareto-optimal balance.

3. Order-Agnostic Estimation:

   • Since ProteinMPNN is an order-agnostic autoregressive model (unlike standard LLMs), calculating exact log-ratios for DPO is difficult. ProtAlign estimates probabilities by sampling multiple random residue decoding orders and averaging the results, significantly reducing variance.

3. Key Contributions

• ProtAlign Framework: A novel multi-objective alignment framework that fine-tunes inverse folding models to satisfy diverse developability objectives while preserving designability.
• MoMPNN Model: The instantiation of ProtAlign on ProteinMPNN, resulting in a model that outperforms existing baselines across crystal structures, de novo backbones, and binder design scenarios.
• Semi-Online Efficiency: A training strategy that decouples rollout and training, reducing computational costs compared to full online RL while avoiding the distributional shift issues of purely offline methods.
• Comprehensive Benchmarking: The introduction of de novo and binder design benchmarks that evaluate models beyond simple sequence recovery, incorporating developability metrics into the standard evaluation pipeline.

4. Experimental Results

The authors evaluated MoMPNN against state-of-the-art baselines (ProteinMPNN, ESM-IF, InstructPLM, SolubleMPNN, HyperMPNN) across three tasks:

• CATH 4.3 Crystal Structures (Redesign):

  • MoMPNN maintained high designability (TM-score, RMSD) comparable to ProteinMPNN.
  • It significantly improved solubility and thermostability compared to ProteinMPNN and other baselines.
  • It outperformed specialized models like SolubleMPNN and HyperMPNN, which often suffered from degraded designability when optimized for a single property.

• De Novo Generated Backbones (RFDiffusion):

  • In this more challenging setting, MoMPNN demonstrated superior structural consistency (higher TM-scores, lower RMSD) compared to ESM-IF and InstructPLM, which showed performance drops.
  • MoMPNN consistently outperformed subset-trained baselines in both structural fidelity and developability metrics.

• De Novo Binder Design:

  • MoMPNN was tested on designing binders for challenging targets (e.g., PD-1, PDL1).
  • It achieved higher sequence success rates (meeting criteria for pLDDT, inter-chain PAE, and RMSD) and backbone success rates than ProteinMPNN.
  • Crucially, it improved developability (solubility and evolutionary plausibility) without sacrificing the ability to bind, demonstrating that the multi-objective alignment generalizes to complex functional tasks.

5. Significance and Impact

• Bridging the Gap: ProtAlign successfully bridges the gap between theoretical sequence recovery and practical protein engineering requirements. It proves that designability and developability are not mutually exclusive but can be jointly optimized.
• Scalability: The semi-online approach makes it feasible to align large protein models with multiple complex properties without the prohibitive cost of full online reinforcement learning.
• Generalizability: The framework is model-agnostic (demonstrated on ProteinMPNN) and can be applied to any inverse folding model to incorporate arbitrary in silico or wet-lab validated properties.
• Future Direction: By establishing a systematic framework for multi-objective protein design, this work opens new avenues for designing proteins that are not only structurally sound but also viable for industrial and therapeutic applications (e.g., high-yield expression, stability in storage).

In conclusion, ProtAlign represents a significant step forward in computational protein design, moving from single-objective optimization to a robust, multi-objective alignment paradigm that produces proteins ready for real-world deployment.