Fine-tuning protein language models on human spatial constraint improves variant effect prediction by reducing wild-type sequence bias

This paper introduces Human Spatial Constraint (HuSC), a framework that integrates human population genetic variation with 3D protein structures to fine-tune protein language models, thereby reducing wild-type sequence bias and significantly improving the prediction of missense variant effects.

The Gist

The Big Picture: Why Do Some Mutations Matter?

Imagine your DNA is the instruction manual for building a human. Sometimes, a typo happens in the manual (a genetic mutation). Most of the time, these typos don't matter; the book still works fine. But sometimes, a typo breaks a crucial sentence, causing a disease.

Scientists have been trying to build a "spellchecker" to predict which typos are harmless and which are dangerous. For a long time, the best spellcheckers worked by comparing human instructions to instructions from other animals (like mice, chimps, and fish). The logic was: "If this part of the manual has stayed exactly the same for millions of years across all these animals, it must be super important. If we change it, it will break."

This works well, but it has a blind spot. It misses changes that are important only to humans right now. It's like checking a car manual against a horse's anatomy; you might miss a specific quirk of the car's engine that only matters for modern driving.

The New Solution: "Human Spatial Constraint" (HuSC)

The authors of this paper introduced a new tool called HuSC (Human Spatial Constraint). Think of HuSC as a 3D crowd-sourced map of human genetic variation.

Instead of just looking at the text of the manual, HuSC looks at:

1. The 3D Shape: Proteins are like folded origami. Some parts are on the outside (easy to touch), and some are buried deep inside (hard to reach).
2. The Human Crowd: They looked at genetic data from over 140,000 real people.

The Analogy:
Imagine a crowded dance floor (the protein).

• Old Method (Comparing Species): Looks at the dance floor from a bird's eye view, comparing it to dance floors in other cities (other species) to see which moves are "classic."
• HuSC Method: Looks at the dance floor right now. It counts how many people are dancing in a specific circle.
  • If a specific spot on the floor is empty (no one is dancing there), it means that spot is dangerous or critical. If you step there, you get kicked out (natural selection removes the mutation).
  • If a spot is packed with dancers, it means that spot is flexible and can handle a lot of movement.

HuSC combines the 3D shape of the protein with this "dance floor" data to see where humans are actually allowed to make changes and where they aren't.

The Breakthrough: Fixing the "Spellchecker"

The researchers took a powerful AI model (called a Protein Language Model, or PLM) that was already good at predicting mutations but was "blind" to recent human history. They fine-tuned it using their new HuSC map.

The Result:
The AI got much smarter. It didn't just learn to spot the "classic" mistakes; it learned to spot the human-specific mistakes.

• Before: The AI was like a librarian who only knows books from the 1800s.
• After: The AI is now a librarian who knows the 1800s books and the latest bestsellers from 2024.

What Did They Discover?

1. It's Better at Predicting Disease: HuSC is better at telling if a mutation will cause disease than any previous method, even those that compare humans to chimps.
2. Human-Specific Secrets: They found parts of our proteins that are highly constrained (strict) in humans but look "relaxed" in other animals.
   • Example 1 (Immune System): They found strict rules in genes related to our immune system (like the "SLAMF6" protein). It seems humans have evolved very specific rules for how our immune cells talk to each other that other animals don't have.
   • Example 2 (Gene Regulators): They found strict rules in genes that control how we read DNA (Zinc Finger proteins). These are like the "editors" of our genetic code, and they are evolving rapidly just for us.

The "Aha!" Moment: Why Did the AI Get Better?

The researchers asked: "Why did adding this human data make the AI smarter?"

They found that the original AI was too confident about the "standard" version of proteins. It thought, "This is the wild-type (standard) amino acid, so it must be perfect, and any change is bad."

The Correction:
The HuSC training taught the AI to calm down.

• In areas where humans naturally have a lot of variation (the "packed dance floor"), the AI learned: "Hey, this spot is flexible. Changing the amino acid here isn't necessarily a disaster."
• In areas where humans have no variation (the "empty dangerous spot"), the AI learned: "Okay, this spot is critical. Don't touch it."

By reducing its overconfidence in flexible areas, the AI became much better at ranking which mutations are truly dangerous and which are harmless.

Summary

This paper is like upgrading a GPS.

• Old GPS: Only knew the main highways built 100 years ago (evolutionary conservation across species).
• New GPS (HuSC): Knows the main highways plus the current traffic jams, construction zones, and new shortcuts specific to your city (human population variation).

By combining the "long history" of evolution with the "current traffic" of human genetics, the researchers built a much more accurate tool for predicting how genetic changes affect our health.

Technical Summary

1. Problem Statement

Protein Language Models (PLMs), such as ESM2, have achieved state-of-the-art performance in predicting the effects of missense variants by learning evolutionary constraints from diverse protein sequences across the tree of life. However, these models have two critical limitations:

1. Lack of Intraspecies Context: PLMs are trained on interspecies conservation (millions of years of evolution) and do not explicitly model recent selective pressures or variation patterns within the human population (thousands of years).
2. Wild-Type Bias: PLMs often exhibit an overconfidence in wild-type (reference) sequences, particularly in regions that are actually mutationally tolerant. This "black-box" nature limits their interpretability and accuracy in clinical settings where distinguishing pathogenic from benign variants is crucial.

Existing methods for quantifying human-specific constraint often rely on multiple sequence alignments (MSAs) or simple gene-level metrics, failing to integrate the 3D structural context of proteins with population-scale allele frequencies.

2. Methodology

The authors introduce a two-pronged approach: the development of a new constraint metric (HuSC) and a strategy to integrate this metric into PLMs via fine-tuning.

A. The Human Spatial Constraint (HuSC) Framework

HuSC quantifies evolutionary constraint on 3D spatial regions of proteins within the human population.

• Data Integration: The framework maps Single Nucleotide Variants (SNVs) and their Minor Allele Frequencies (MAFs) from gnomAD v2.1.1 (141,456 individuals) onto AlphaFold Database structures (16,259 human proteins).
• Spatial Aggregation: For each residue, the framework aggregates variant frequencies within a 3D spatial neighborhood (e.g., an 8 Å sphere).
• Null Model Construction: To determine expected variation under neutral evolution, the authors built a permutation-based model that accounts for:
  • Local nucleotide mutability (trinucleotide context).
  • Global variation in mutation rates between proteins.
  • The observed allele frequency spectrum.
• Score Calculation: The HuSC score is a signed log-transformed Z-score comparing the observed frequency of missense variants in a 3D region against the expected frequency from the null model.
  • Negative HuSC: Indicates fewer variants than expected (strong constraint).
  • Positive HuSC: Indicates more variants than expected (tolerance).

B. Fine-Tuning PLMs with HuSC

To leverage HuSC for variant effect prediction, the authors fine-tuned the ESM2 family of PLMs (8M to 650M parameters).

• Strategy: They used Low-Rank Adaptation (LoRA) to inject trainable matrices into the self-attention modules while freezing the pre-trained weights. This prevents "catastrophic forgetting" of interspecies evolutionary knowledge.
• Training Signal: The model was trained to minimize a listwise ranking loss between the model's predicted sequence entropy (derived from log-likelihood ratios) and the observed HuSC scores.
• Data Filtering: Training was restricted to proteins and sites with strong constraint signals (HuSC < 0) to maximize the learning of human-specific selection pressures.

3. Key Contributions

1. HuSC Metric: A novel, structure-aware metric that quantifies intraspecies constraint by integrating population genetics with 3D protein structures. It outperforms traditional conservation metrics in predicting pathogenicity.
2. Identification of Human-Specific Constraints: The study identifies protein sites constrained in humans but not conserved across species, revealing lineage-specific evolutionary pressures (e.g., in immune and transcriptional regulation).
3. PLM Fine-Tuning Framework: A method to integrate intraspecies constraint signals into generalist PLMs, demonstrating that human-specific data improves general variant effect prediction across diverse taxa.
4. Mechanistic Insight: The authors elucidate why fine-tuning works: it primarily recalibrates model confidence, reducing the overestimation of wild-type fitness in mutationally tolerant regions.

4. Results

A. Performance of HuSC in Pathogenicity Prediction

• HuSC was evaluated against ClinVar pathogenic/benign variants.
• Metrics: HuSC achieved an ROC AUC of 0.91 and PR AUC of 0.90, significantly outperforming:
  • Interspecies metrics (ConSurf, PhyloP, GERP, PhastCons).
  • Other intraspecies metrics (COSMIS, MTR3D, MTR, pLI, RVIS).
• Biological Validation: HuSC scores correctly identified lower constraint in olfactory receptors and higher constraint in essential, haploinsufficient, and dominant disease genes.

B. Discovery of Human-Specific Constrained Genes

By comparing HuSC (intraspecies) with ConSurf (interspecies), the authors identified residues with low HuSC (constrained in humans) but high ConSurf (not conserved across species).

• Enrichment: These residues were highly enriched in immune-related processes (e.g., T-cell activation, NK cell regulation) and transcriptional regulation (KRAB-Zinc Finger proteins).
• Case Studies:
  • SLAMF6: Human-specific constrained residues localized to the dimerization interface, suggesting lineage-specific selection on receptor-receptor interactions.
  • ZNF460: Constrained residues distributed across zinc finger motifs, indicating ongoing selection on DNA-binding interfaces.

C. Improved Variant Effect Prediction via Fine-Tuning

Fine-tuning ESM2 with HuSC scores improved performance on the ProteinGym benchmark (Deep Mutational Scanning data):

• General Improvement: Spearman correlation with experimental fitness increased across all model sizes (e.g., 650M model improved from 0.45 to 0.48).
• Generalization: Improvements were observed not just in human proteins, but also in prokaryotic and eukaryotic proteins, suggesting HuSC captures fundamental principles of mutational tolerance.
• Assay Specificity: Gains were most significant for stability, enzymatic activity, and organismal fitness assays.

D. Mechanism of Improvement: Reducing Wild-Type Bias

Analysis of the fine-tuned models revealed:

• Reduced Confidence in Wild-Type: The Negative Log-Likelihood (NLL) of wild-type sequences increased significantly (median from 0.24 to 0.58), indicating the model became less overconfident in the reference sequence.
• Targeted Recalibration: This reduction in confidence was most pronounced in variation-tolerant regions (top 10% of variants by fitness).
• Conclusion: The performance gain stems from correcting the model's prior bias toward wild-type sequences in permissive regions, allowing for better ranking of both highly fit and highly deleterious variants.

5. Significance

This work demonstrates that intraspecies population data provides a complementary signal to interspecies evolutionary data. By integrating human-specific spatial constraints into PLMs:

1. Clinical Relevance: It offers a more accurate tool for interpreting the pathogenicity of missense variants, particularly for diseases driven by recent human-specific selection.
2. Model Interpretability: It provides a mechanism to "debias" PLMs, moving them away from an inherent preference for wild-type sequences toward a more realistic assessment of mutational tolerance.
3. Evolutionary Insight: It reveals that functional constraints in humans are not merely a subset of ancient constraints; specific immune and regulatory systems are undergoing rapid, lineage-specific evolution that traditional conservation metrics miss.
4. Scalability: The LoRA-based fine-tuning approach offers a computationally efficient pathway to integrate population-scale data into increasingly large and powerful protein foundation models.