Conditionally Site-Independent Neural Evolution of Antibody Sequences

This paper introduces CoSiNE, a deep neural network-parameterized continuous-time Markov chain that bridges the gap between expressive deep learning and classical phylogenetic models to capture epistatic interactions in antibody evolution, thereby outperforming state-of-the-art language models in variant effect prediction and enabling efficient affinity optimization through a novel Guided Gillespie sampling scheme.

The Gist

Here is an explanation of the paper "Conditionally Site-Independent Neural Evolution of Antibody Sequences" (COSINE), broken down into simple concepts with creative analogies.

The Big Picture: Teaching AI to Play "Evolution"

Imagine your body is a fortress under attack by viruses. To defend itself, it has an elite special forces unit: Antibodies. These are Y-shaped proteins that hunt down specific invaders.

When a new virus shows up, your body doesn't just make one antibody; it runs a rapid-fire evolutionary boot camp. It takes a basic antibody, makes thousands of tiny random tweaks (mutations), and keeps only the ones that stick to the virus best. This process is called Affinity Maturation.

The problem? Scientists have two ways to study this, and both are flawed:

1. The "Photo Album" Approach (Current AI): Most modern AI models look at a pile of finished antibodies and try to guess what they look like. It's like looking at a photo album of a family and trying to guess how the kids grew up without knowing the parents or the timeline. It misses the story of how they changed.
2. The "Math Textbook" Approach (Classical Biology): Old-school models try to simulate the evolution step-by-step using strict math rules. But they are too simple. They assume every part of the antibody changes independently, like a car where the tires, engine, and steering wheel are all upgraded separately. In reality, changing the engine might break the steering wheel. These models miss the complex teamwork between parts.

COSINE is the new hero that combines the best of both worlds. It's an AI that doesn't just look at the photos; it simulates the movie of evolution, understanding how changing one part of the antibody affects the whole team.

---

The Core Idea: The "Team Captain" Analogy

To understand how COSINE works, imagine an antibody is a soccer team with 10 players (amino acids).

• The Old Way (Independent Sites): A coach tells each player, "You can change your jersey color however you want, as long as you stay on the field." The coach doesn't care if the goalie changes color to match the striker. This leads to chaotic, unrealistic teams.
• The COSINE Way (Conditionally Site-Independent): COSINE acts like a smart team captain. When a player (a specific spot on the antibody) wants to change, the captain looks at the entire team's current formation before deciding if that change is good.
  • If the striker changes position, the captain knows the defender needs to shift too.
  • The AI learns that "If Player A is wearing red, Player B must wear blue to keep the team balanced."

This allows COSINE to capture Epistasis: the fancy word for "how parts of a system depend on each other."

---

How It Works: The "Gillespie" Simulator

The paper introduces a method called Gillespie Sampling. Think of this as a time-lapse camera for evolution.

1. The Setup: You start with a "naive" antibody (the rookie).
2. The Simulation: Instead of jumping straight to the final result, COSINE simulates the evolution one tiny mutation at a time, like a video game character taking one step at a time.
3. The Magic: Because it simulates the process step-by-step, it can handle the complex "team dependencies" (epistasis) that older models miss. It proves mathematically that if you take small enough steps, this simulation is almost perfectly accurate.

The "Guided" Feature: Steering the Evolution

The coolest part of the paper is Guided Gillespie.

Imagine you are playing a video game where you want to evolve a monster to be the strongest possible fighter against a specific boss (a virus).

• Normal Evolution: You let the monster evolve randomly. It might get strong, or it might get weird and weak.
• Guided Evolution: You give the AI a "compass." You say, "I want this monster to be strong against this specific boss."

COSINE uses a classifier (a separate AI that predicts how well an antibody binds to a virus) to act as a GPS. As the antibody evolves step-by-step, the GPS nudges it toward the path that leads to the best binding.

• If a mutation makes the antibody stick better to the virus, the GPS says, "Go that way!"
• If a mutation makes it worse, the GPS says, "Turn back!"

This allows scientists to design brand new antibodies from scratch that are guaranteed to be good at fighting a specific disease, without needing to test millions of physical samples in a lab.

Why Does This Matter?

1. Better Medicine: We can design antibodies for new viruses (like future pandemics) much faster.
2. Understanding Biology: It helps us understand the "rules of the game" that nature uses to evolve immune systems.
3. Efficiency: It saves time and money. Instead of growing bacteria and testing them in a lab for months, we can simulate the best candidates on a computer first.

Summary in One Sentence

COSINE is a new AI that simulates the evolutionary "boot camp" of antibodies step-by-step, understanding how every part of the protein works together, and uses a digital "GPS" to steer the evolution toward creating super-antibodies that can fight specific diseases.

Technical Summary

Here is a detailed technical summary of the paper "Conditionally Site-Independent Neural Evolution of Antibody Sequences" (COSINE).

1. Problem Statement

Antibody engineering relies on understanding affinity maturation, the evolutionary process where B cells undergo somatic hypermutation (SHM) and selection to improve antigen binding. Current deep learning approaches for antibody design face a fundamental trade-off:

• Antibody Language Models (e.g., ESM, AbLang): These model the marginal distribution of sequences ($p(x)$) assuming sequences are independent and identically distributed (i.i.d.). While they capture complex epistatic interactions (dependencies between amino acid sites), they fail to model the time-dependent evolutionary dynamics and the specific process of mutation and selection. They often rely on memorizing conserved germline residues rather than understanding the evolutionary trajectory.
• Classical Phylogenetic Models: These use Continuous-Time Markov Chains (CTMCs) to explicitly model evolutionary dynamics. However, they typically assume site-independence (sites evolve independently), ignoring epistasis. This leads to unrealistic evolutionary trajectories and poor performance in protein design, especially over long branch lengths.

The core challenge is to develop a model that captures both the temporal dynamics of evolution (mutation over time) and the complex epistatic interactions between sites, while remaining computationally tractable for protein sequences.

2. Methodology: The COSINE Model

The authors propose COSINE (Conditionally Site-Independent Neural Evolution), a framework that bridges deep learning and phylogenetics.

A. Core Architecture

COSINE is a Continuous-Time Markov Chain (CTMC) where the rate matrix is parameterized by a deep neural network.

• Conditional Site Independence: Instead of a single global rate matrix or a fully coupled state-space matrix (which is intractable for sequences of length $L$), COSINE learns site-specific rate matrices $Q_\ell$ conditioned on the full sequence context $x$.
• Transition Probability: The probability of transitioning from sequence $x$ to $y$ over time $t$ is factorized as:
  $$p_\theta(y | x, t) = \prod_{\ell=1}^L \exp(t Q_\theta(x)_\ell)_{x_\ell, y_\ell}$$
  Here, $Q_\theta(x)_\ell$ is a $20 \times 20$rate matrix for site$\ell$, generated by a neural network taking the entire sequence$x$ as input. This allows the model to capture epistasis (context-dependence) while maintaining the computational efficiency of factorized likelihoods.

B. Theoretical Foundation

The authors prove that COSINE provides a first-order approximation to the true sequential point mutation process (where mutations occur one site at a time).

• Error Bound: The error between the COSINE transition probability and the true sequential process is bounded by $O(t^2)$ (quadratic in branch length).
• Relevance: Since affinity maturation typically involves short branch lengths (rapid selection), the first-order signal dominates the quadratic error, making the approximation highly accurate for this biological regime.

C. Sampling: Gillespie Algorithm

To avoid the error introduced by the factorized approximation (Equation 2) during generation, the authors introduce a Gillespie sampling algorithm adapted for COSINE.

• Instead of sampling sites independently, the algorithm simulates the continuous-time process by sampling the time to the next mutation and the specific site/amino acid based on the instantaneous rates provided by the neural network.
• Lemma 4.2 proves that under the condition that the neural network's rates match the true instantaneous rates for single mutations, this procedure samples exactly from the true sequential point mutation process.

D. Disentangling Selection from Mutation

To use COSINE for Variant Effect Prediction (VEP) and fitness inference, the model must separate the mutational bias (SHM) from functional selection.

• Framework: Following Halpern & Bruno (1998), the observed transition rate $Q_{xy}$ is decomposed into the SHM rate $\mu_{xy}$ and the fixation probability $P_{fix}$.
• Selection Score: The authors define a selection score as the log-likelihood ratio between the COSINE model (which learns the full evolutionary process) and a pre-trained SHM-only model (Thrifty):
  $$\text{Score}(x \to y) = \log p_\theta(y | x, t) - \log q(y | x, t)$$
  This score isolates the fitness contribution, enabling zero-shot prediction of variant effects.

E. Guided Generation (Guided Gillespie)

To optimize antibodies for specific antigens, the authors introduce Guided Gillespie Sampling.

• They adapt classifier guidance (common in diffusion models) to the CTMC framework.
• A predictor network estimates the binding affinity $r$ to a target antigen $z$.
• The sampling process is steered by modifying the rate matrix $Q$ using a guidance strength $\gamma$:
  $$(Q_z^{(\gamma)})_{x,y} = \left[ \frac{p(z|y)}{p(z|x)} \right]^\gamma Q_{x,y}$$
• To make this computationally feasible, they use a Taylor-series approximated guidance (TAG), linearizing the affinity predictor around the current sequence to compute gradients efficiently.

3. Key Contributions

1. COSINE Framework: A novel neural CTMC that learns site-specific rates conditioned on the full sequence, successfully capturing epistatic effects while modeling evolutionary time.
2. Theoretical Guarantees: Mathematical proof that COSINE is a first-order approximation of sequential point mutations with a quadratic error bound, justifying its use for short evolutionary timescales like affinity maturation.
3. Disentanglement Strategy: A principled method to separate SHM biases from selection signals using log-likelihood ratios, enabling accurate fitness landscape inference without manual clamping of scores.
4. Guided Sampling: The first application of classifier guidance to discrete sequence evolution models (CTMCs), allowing for the directed optimization of antibody binding affinity toward specific antigens.
5. State-of-the-Art Performance: Demonstrated superior performance in zero-shot variant effect prediction and in silico antibody optimization compared to existing language models and phylogenetic baselines.

4. Experimental Results

• Model Fit: COSINE was trained on ~2 million evolutionary transitions from ~120,000 clonal trees. It achieved a test perplexity of 1.264 and outperformed the state-of-the-art DASM+Thrifty model in 62.3% of cases, particularly for longer branch lengths.
• Epistasis Capture: Analysis of the categorical Jacobian showed that COSINE successfully learns long-range dependencies, including interactions between heavy and light chains and within Complementarity Determining Regions (CDRs).
• Zero-Shot Variant Effect Prediction (VEP): On the FLAb2 benchmark (four DMS assays), COSINE achieved the highest Spearman correlation with experimental fitness values across most datasets, outperforming ESM-2, ProGen2, and DASM.
• Guided Optimization:
  • Affinity Maturation: Starting from naive antibodies, Guided Gillespie sampling successfully generated sequences with significantly higher predicted binding affinity to SARS-CoV-1 and SARS-CoV-2 compared to unguided sampling.
  • Quality Control: The generated sequences maintained high structural quality (pLDDT) and "humanness" (OASis score), comparable to natural binders.
  • Local Optimization: In a constrained CDR optimization task (max 5 mutations), COSINE achieved higher binding affinity gains than Genetic Algorithms and Product-of-Experts samplers while respecting the mutation budget.

5. Significance

This work represents a paradigm shift in protein sequence design by grounding deep learning in molecular evolution.

• Bridging Paradigms: It reconciles the expressivity of modern language models with the temporal rigor of phylogenetic models, addressing the "i.i.d. assumption" flaw in current antibody engineering tools.
• Practical Utility: The ability to simulate evolutionary trajectories and steer them toward specific functional goals (like binding a new antigen) without requiring extensive experimental fine-tuning offers a powerful tool for therapeutic antibody discovery.
• Theoretical Insight: The derivation of a first-order approximation for sequential mutations provides a new theoretical lens for understanding how neural networks can approximate complex stochastic processes in biology.

In summary, COSINE provides a mathematically grounded, expressive, and efficient framework for modeling antibody evolution, enabling both accurate fitness prediction and the directed design of high-affinity therapeutic antibodies.