💻 bioinformatics

Homology-Based Variant-Effect Predictors Break Down on Cytochrome P450 Pharmacogenes

This study demonstrates that homology-based variant-effect predictors systematically fail for Cytochrome P450 pharmacogenes because they primarily track protein abundance rather than catalytic activity, necessitating a new modeling approach that incorporates substrate-specific context to accurately predict functional consequences.

Original authors: Xu, H., Samori, I., Nayar, G., Altman, R. B.

Published 2026-10-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xu, H., Samori, I., Nayar, G., Altman, R. B.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine a world where the same dose of medicine saves one person but harms another. This is not a mystery of bad luck, but of biology. Inside our bodies, a family of enzymes called cytochrome P450 acts as a chemical processing plant, breaking down roughly three-quarters of the drugs we take. Because these enzymes vary from person to person, they process medications at different speeds. For some, a standard dose is too weak to work; for others, it builds up to dangerous levels. Scientists have long hoped to predict how a person's unique genetic code would change these enzymes, allowing doctors to prescribe the perfect dose for every individual. To do this, they have relied on computer programs that guess the effect of a genetic change by looking at how similar that change is to variations found in other species over millions of years. The logic was simple: if nature has kept a part of the enzyme the same for eons, changing it must be bad.

However, a new study reveals that this long-held logic breaks down when applied to these drug-processing enzymes. Researchers at Stanford University, led by Helen Xu and Russ Altman, tested the most advanced computer models available against real-world data from the enzyme CYP2C9. They found that the standard tools, which work well for predicting disease-causing mutations, fail to understand how these specific enzymes work. The models are essentially blind to the most important part of the story: the specific drug the enzyme is trying to break down. The study suggests that to truly predict how a person will react to medication, we cannot just look at the enzyme's shape or its history; we must also look at the specific chemical it is meeting.

The researchers began by testing a powerful new tool called AlphaMissense, which uses artificial intelligence to predict whether a genetic change is harmful. They fed it data from six important drug-metabolizing enzymes. The tool struggled significantly, labeling nearly twice as many changes as "uncertain" compared to its performance on the rest of the human body. When the team dug deeper into the data for CYP2C9, they discovered why. The computer model was not actually measuring how well the enzyme worked; it was mostly measuring how much of the enzyme was present in the cell. If a genetic change made the enzyme unstable and caused it to disappear, the model flagged it as a problem. But if the enzyme stayed stable and present, the model assumed it was working fine, even when it was completely unable to process drugs.

To understand this failure, the team looked at the physical structure of the enzyme and the chemistry of the changes. They wondered if the model was missing something obvious, like the location of the change or the type of chemical swap involved. They found that these factors explained almost nothing. The model's errors were not due to a lack of structural data or a misunderstanding of basic chemistry. The problem was deeper. The model was trained on the idea that evolution keeps enzymes the same because they are vital for survival. But these drug-processing enzymes are different. They evolved to handle a wide variety of chemicals from our food and environment, not just a single, specific task. Because they are designed to be flexible, nature does not punish them for changing. A genetic variation that ruins the enzyme's ability to process a specific drug might leave the enzyme perfectly stable and abundant, fooling the computer into thinking everything is normal.

The researchers then tried to build a better model. They combined the standard tools with a new approach that looked at the specific patterns of the enzyme's genetic code, rather than just its evolutionary history. This new, custom-built model was much better at predicting the actual chemical activity of the enzyme. It could tell the difference between a stable enzyme that works and a stable enzyme that is broken. However, even this improved model hit a wall. When the researchers compared their predictions to real-world medical records and clinical labels, the agreement was poor. The model could predict the enzyme's activity in a test tube, but it could not predict how that activity would translate to a patient taking a specific drug.

The reason for this disconnect lies in the nature of the enzymes themselves. These proteins are not single-purpose machines; they are multi-taskers that interact with many different drugs. The study found that the same genetic change can have completely different effects depending on which drug is present. A mutation might make the enzyme work poorly on one medication but have no effect on another. The current computer models treat the enzyme as if it has a single, universal function, ignoring the fact that its job changes with every drug it meets. The researchers found that in the medical databases they examined, more than seventy percent of the genetic variations that had been tested against multiple drugs showed different results for each one.

This finding suggests that the entire approach to predicting drug reactions needs to change. For decades, scientists have tried to assign a single score to a genetic variant, asking if it is "good" or "bad." This study argues that for drug-metabolizing enzymes, that question is the wrong one. A variant is not inherently good or bad; it is only good or bad in the context of a specific drug. The researchers propose that future tools must be built to ask a different question: how does this specific genetic change affect the processing of this specific drug? Until computer models can account for the specific chemical partner the enzyme is working with, they will remain unable to fully predict the complex interactions that determine whether a medication heals or harms. The path forward is not just better data or smarter algorithms, but a redefinition of what we mean by the function of these vital biological machines.

Technical Summary: Homology-Based Variant-Effect Predictors Break Down on Cytochrome P450 Pharmacogenes

Problem Statement
Cytochrome P450 (CYP) enzymes metabolize approximately 75% of clinically used drugs, and genetic variation in these enzymes is a primary source of interindividual differences in drug response. While computational variant-effect prediction (VEP) is essential for scaling pharmacogenomics, most state-of-the-art predictors rely on a homology-based paradigm. These models score variants based on evolutionary conservation, operating under the assumption that residues conserved across evolution are functionally critical. However, the authors argue that this assumption is violated by pharmacogenes like CYPs. Unlike disease genes under strong purifying selection, CYPs evolved to metabolize diverse environmental compounds, meaning reduced activity is often benign unless a specific drug is encountered. Consequently, homology-based models may fail to predict the functional consequences of CYP variants, particularly those that are not disease-causing but alter drug metabolism.

Methodology
The study utilized human CYP pharmacogenes, with a primary focus on CYP2C9 due to the availability of comprehensive Deep Mutational Scanning (DMS) data.

Datasets: The authors integrated four ground-truth resources:
1. DMS Data: CYP2C9 missense variants with paired enzymatic activity (click-seq) and cellular abundance (vamp-seq) scores.
2. Clinical Labels: ClinVar disease significance classifications.
3. Pharmacogenetic Labels: PharmVar star allele functional classes.
4. Substrate-Resolved Data: ClinPGx (PharmGKB) annotations linking variants to specific drug substrates.
Predictors Evaluated: The study assessed several models, including:
- AlphaMissense (AM): A homology-based pathogenicity predictor.
- ESM-2: A protein language model used for zero-shot variant-effect prediction (via masked marginal probabilities, MMP) and as embeddings for a custom model.
- External Baselines: SIFT, REVEL, DeMaSk (trained on DMS), and PharmGScore (pharmacogene-tuned).
Novel Modeling: The authors constructed a k-nearest-neighbors (KNN) regressor using ESM-2 embeddings, trained specifically on CYP2C9 activity. This was ensembled with calibrated AM scores and ESM-2 MMP scores using ridge regression.
Analysis Strategy:
- Calibration: Predictors were calibrated to the DMS activity scale using monotonic regression.
- Stratification: Variants were stratified by cellular abundance (destabilizing/low-abundance vs. abundance-independent) to disentangle protein stability effects from catalytic activity.
- Error Analysis: The study tested whether structural and biochemical priors (sequence position, substitution chemistry, distance to substrate-recognition sites, and secondary structure) could explain prediction errors.
- Clinical Alignment: Predicted activities were correlated against ClinVar and PharmVar labels to assess clinical utility.

Key Results

Systematic Failure of Homology-Based Models: AlphaMissense (AM) assigns CYP missense variants to its "ambiguous" class at nearly twice the proteome-wide rate (18.16% vs. 10.28%). Within this ambiguous class, AM's scores are essentially uninformative regarding measured catalytic activity ( $\rho = -0.130$ ).
Abundance Bias: AM's signal is largely driven by protein abundance rather than catalytic activity. It correlates strongly with DMS activity for destabilizing (low-abundance) variants ( $\rho = -0.568$ ) but performs significantly worse for abundance-independent variants ( $\rho = -0.357$ ). This discrepancy is highly significant ( $p < 10^{-18}$ ).
Limited Explanatory Power of Structural Priors: Structural features (position, chemistry, substrate-recognition site proximity, secondary structure) account for less than 1% of the variance in AM's prediction error, indicating that the failure is not easily explainable by simple structural heuristics.
Performance of the CYP-Specific Ensemble: A custom ensemble model (KNN on ESM-2 embeddings + AM + ESM-2 MMP) achieved the highest correlation with measured CYP2C9 activity ( $\rho = 0.721$ ), outperforming all external baselines (ranging from $\rho = 0.523$ to $0.686$). The most significant improvement was observed within the "ambiguous" class, where the ensemble raised the correlation from $\rho = 0.080$ (AM alone) to $\rho = 0.399$ .
Disconnect from Clinical Labels: Despite improved accuracy against biochemical assays, no model (including the custom ensemble) showed significant agreement with ClinVar or PharmVar clinical annotations. Even the measured DMS activity itself failed to align significantly with clinical labels. Specific variants (e.g., CYP2C9*3) were predicted to retain high activity by all models but were labeled "No Function" in PharmVar.
Substrate Dependence: Analysis of ClinPGx data revealed that 71.9% of variants with multiple drug annotations exhibited discordant functional labels depending on the substrate. This indicates that a single, substrate-agnostic activity score cannot capture the true functional effect of a CYP variant.

Significance and Claims
The paper argues that the failure of variant-effect predictors on CYPs is not merely a matter of poor homology detection or insufficient calibration, but a fundamental definitional mismatch.

The "Ceiling" of Homology: The best possible use of homology-derived information reaches a ceiling that falls short of clinical requirements. The models predict a single, substrate-agnostic effect, whereas CYP function is inherently multi-dimensional and substrate-dependent.
Redefining Function: The authors propose that predicting function for multi-substrate enzymes requires redefining "function" not as a scalar value, but as a quantity conditioned on the substrate.
Missing Variable: The primary missing feature in current models is substrate identity. The authors hypothesize that a variant's effect is a function of the specific drug it encounters, a relationship that current homology-based or even function-trained models cannot capture without explicit substrate conditioning.
Modest Scope: The authors acknowledge that their improved ensemble is protein-specific (CYP2C9) and does not generalize beyond that gene. They also note that the lack of agreement with clinical labels may be partly due to noise in the annotations themselves, though substrate dependence remains the most biologically plausible explanation for the observed discrepancies.

In conclusion, the study demonstrates that while protein-specific models can improve biochemical activity prediction, the current paradigm of predicting a single variant effect is insufficient for pharmacogenes. Moving forward requires shifting from homology-based similarity searches to substrate-conditioned prediction models.

Technical Summary: Homology-Based Variant-Effect Predictors Break Down on Cytochrome P450 Pharmacogenes

More like this