Dissecting the Predictive Accuracy of Polygenic Indexes for Behavioral Phenotypes Across Genetic Ancestries

This study systematically analyzes the reduced predictive accuracy of polygenic indexes for behavioral and social traits across non-European ancestries using UK Biobank and Health and Retirement Study data, revealing that while portability loss is most severe in African populations due primarily to genetic architecture differences, family-based GWAS approaches can modestly mitigate these disparities.

The Gist

Imagine you have a very sophisticated weather forecast model built using decades of data from people living in London. This model is incredibly good at predicting rain, wind, and sunshine for Londoners.

Now, imagine you take that exact same London model and try to use it to predict the weather in Nairobi, Tokyo, and Mumbai.

You might expect it to work okay, but it doesn't. It works best in Tokyo (which is somewhat similar to London in some ways), okay in Mumbai, but it fails miserably in Nairobi. Why? Because the "weather patterns" (the genetic rules) in Nairobi are different. The clouds form differently, the wind blows from different directions, and the seasons don't align.

This paper is essentially a deep dive into exactly that problem, but instead of weather, it's about genetic predictions for human traits (like height, intelligence, or risk of disease).

Here is the breakdown of what the researchers found, using simple analogies:

1. The "London Model" Problem (The Portability Issue)

Most genetic studies (called GWAS) have been done almost entirely on people of European ancestry. Scientists have built "Polygenic Indexes" (PGIs)—which are like genetic scorecards—that predict things like how tall you are or your risk of heart disease based on these European data sets.

The researchers asked: If we take these European scorecards and apply them to people of African, East Asian, or South Asian ancestry, how well do they work?

The Answer: They work, but they lose a lot of power.

• African Ancestry: The model retains only about 24% of its original accuracy. It's like trying to navigate a city with a map of a different city; you're mostly guessing.
• East Asian Ancestry: It retains about 37%.
• South Asian Ancestry: It retains about 51%.

2. Why does the map fail? (The Three Culprits)

The researchers investigated why the map fails. They found three main reasons, which they tested like detectives:

• Culprit #1: The "Road Signs" are different (LD and MAF).
  Imagine your genetic code is a book of instructions. In the European version of the book, the instructions are written in a specific order with specific punctuation (Linkage Disequilibrium). In the African version, the sentences are rearranged, and the punctuation is different.
  The European "decoder" (the PGI) looks for a specific punctuation mark to find the instruction. In the African book, that mark is in a different place or missing entirely. The researchers found that for African ancestry, 82% of the failure is simply because the "road signs" (genetic markers) are in different places or have different frequencies.

• Culprit #2: The "Terrain" is different (Heritability).
  Sometimes, the trait itself is just harder to predict because genetics play a smaller role in that specific population. For example, if height is mostly determined by genetics in Europe but heavily influenced by nutrition in another group, the genetic map will look less accurate there.

• Culprit #3: The "Social Context" (Confounds).
  This is the most interesting part. Standard genetic studies sometimes accidentally pick up on social factors. For example, if a certain genetic marker is common in a family where everyone goes to college, the study might think the gene causes education. But really, it's just that the family had money and culture that supported education.
  When you move to a different country with a different culture, that "social link" breaks. The researchers found that for some traits (like BMI in African ancestry), using a "family-based" map (which ignores these social links) actually made the prediction better. It's like realizing the London weather model was accidentally predicting "umbrella sales" instead of "rain," and fixing it to just predict rain.

3. The "Biological vs. Behavioral" Twist

The researchers also noticed a pattern based on what they were predicting:

• Biological Traits (The "Hard" Stuff): Things like blood pressure, cholesterol, or height are like the engine of a car. They are mostly mechanical. The genetic map works reasonably well across different populations for these.
• Behavioral Traits (The "Soft" Stuff): Things like education, personality, or smoking habits are like the driver's behavior. These are heavily influenced by culture, environment, and society. The genetic map fails much more dramatically here because the "driver" in London behaves very differently from the "driver" in Nairobi, even if their cars (genes) are similar.

4. The Big Takeaway

The paper concludes that we cannot just copy-paste genetic predictions from European populations to the rest of the world.

• For African populations: The main issue is that the genetic "road signs" are different. We need better maps of African genetics to fix this.
• For Behavioral traits: The issue is even deeper. Because these traits are so tied to culture and environment, a genetic map built in one society is often useless in another.
• The Hope: By using "family-based" studies (looking at siblings to rule out social biases), we can clean up the maps a little bit, making them slightly more fair and accurate for everyone.

In short: We are currently trying to use a GPS built for London to drive in Nairobi. It's not just that the roads are different; sometimes the destination itself means something different in a different culture. To fix this, we need to build new GPS systems specifically for every part of the world, not just rely on the one we built for London.

Technical Summary

1. Problem Statement

Polygenic Indexes (PGIs), also known as polygenic scores, are powerful tools for predicting complex traits. However, they are predominantly trained on Genome-Wide Association Studies (GWAS) conducted on individuals of European (EUR) genetic ancestry. When applied to non-European populations, PGIs suffer from a significant "portability gap," where predictive accuracy drops substantially.

While this disparity is well-documented for biologically proximal traits (e.g., height, cholesterol), it remains understudied for behavioral and social phenotypes (e.g., education, personality, substance use), which are hypothesized to be more sensitive to gene-environment interactions and population-specific confounds. Furthermore, the specific mechanisms driving this accuracy loss—whether due to genetic architecture differences (Linkage Disequilibrium and Minor Allele Frequency) or methodological biases (population stratification, assortative mating)—require a more granular analysis across diverse trait categories.

2. Methodology

The authors conducted a systematic analysis using data from two major cohorts: the UK Biobank (UKB) and the Health and Retirement Study (HRS).

• Data & Samples:

  • Phenotypes: 54 health-related, behavioral, social, and cognitive traits were analyzed.
  • Ancestries: European (EUR), South Asian (SAS), East Asian (EAS), and African (AFR). (Note: SAS and EAS were excluded from HRS due to small sample sizes).
  • PGI Construction:
    • Standard PGIs: Derived from between-family GWAS weights (approx. 2.9 million SNPs) using the SBayesR method, sourced from the Social Science Genetic Association Consortium (SSGAC) PGI Repository.
    • Family-based PGIs (fGWAS): Derived from family-based GWAS weights (approx. 1.2 million SNPs) using PRS-CS, sourced from Tan et al. These weights mitigate biases from passive gene-environment correlations, population stratification, and assortative mating.

• Metrics:

  • Predictive Accuracy: Measured as incremental $R^2$ (variance explained by PGI after controlling for principal components and covariates).
  • Relative Accuracy ($RA_{Obs}$): The ratio of incremental $R^2$ in a non-EUR ancestry to that in EUR ancestry.
  • Decomposition of Loss: The authors adapted a theoretical framework (Wang et al.) to quantify the Loss of Accuracy (LoA) attributable to:
    1. LD and MAF differences: Differences in Linkage Disequilibrium patterns and Minor Allele Frequencies between populations.
    2. Heritability ($h^2$) differences: Cross-ancestry variations in SNP-based heritability.
    3. Confounding: Assessed by comparing Standard vs. fGWAS PGIs.

• Statistical Approach:

  • Used bootstrapping (1,000 replications) for confidence intervals.
  • Employed a "candidate causal SNP" approach (top 10,000 independent SNPs + LD neighbors) to model expected accuracy ($RA_{Expected}$) based on LD/MAF.
  • Calculated $LoA(LD+MAF)$ as the ratio of expected loss to observed loss. Values >100% suggest other factors are inflating accuracy or the model underestimates similarity.

3. Key Contributions

1. Expansion to Behavioral Traits: Unlike previous studies focusing on biologically proximal traits, this study systematically evaluates 54 traits, including complex behavioral and social phenotypes, revealing that portability issues are even more pronounced for these traits.
2. Methodological Refinement: The study utilizes genome-wide PGIs (SBayesR) rather than just genome-wide significant SNPs, providing a more realistic benchmark for current PGI applications.
3. Disentangling Mechanisms: It moves beyond simple observation to decompose the accuracy loss into specific genetic factors (LD/MAF vs. Heritability) and methodological artifacts (confounding).
4. Family-Based GWAS Comparison: By introducing fGWAS PGIs, the study isolates the impact of population-specific confounds (e.g., assortative mating) on portability, a novel approach in this context.

4. Key Results

A. Predictive Accuracy and Portability

• Systematic Decline: PGIs trained on EUR data show reduced predictive power in non-EUR groups. The relative accuracy ($RA_{Obs}$) follows a gradient based on genetic distance:
  • African (AFR): Lowest accuracy (~24–28%).
  • East Asian (EAS): Intermediate (~31–37%).
  • South Asian (SAS): Highest among non-EUR (~51–54%).
• Trait Category Differences:
  • Biologically Proximal Traits: Exhibit higher portability (e.g., blood biomarkers).
  • Behavioral/Social Traits: Exhibit significantly lower portability (e.g., cognition, education, personality). This suggests these traits are more influenced by environment-specific factors or gene-environment interactions that differ across populations.
  • Psychiatric Traits: Showed variable results; schizophrenia PGI was surprisingly more predictive in SAS and AFR than EUR, likely driven by higher case prevalence in those groups within the sample.

B. Drivers of Accuracy Loss

• LD and MAF Differences:
  • In AFR, LD and MAF differences explain the vast majority of accuracy loss (~82%).
  • In EAS and SAS, LD/MAF explains a much smaller portion (~34% and ~25%, respectively). This implies that for Asian ancestries, other factors (e.g., imperfect genetic correlations, heritability differences) are the primary drivers of portability loss.
• Heritability ($h^2$):
  • Accounting for cross-ancestry differences in SNP-based heritability helped explain the remaining unexplained loss for some traits (e.g., height in AFR), bringing the total explained loss close to 100%.
  • However, for traits like HDL/LDL cholesterol in AFR, the loss remained unexplained even after accounting for heritability, suggesting other factors are at play.

C. Impact of Confounding (Standard vs. fGWAS)

• fGWAS Performance: Generally, fGWAS PGIs had lower absolute predictive power due to smaller effective sample sizes.
• Portability Improvement: For specific traits, fGWAS PGIs showed modest but significant improvements in portability compared to standard PGIs.
  • Notable Example: For BMI in AFR, the relative accuracy jumped from 0.05 (Standard) to 0.34 (fGWAS).
  • Implication: This suggests that a portion of the portability gap in standard PGIs is driven by population-specific confounds (e.g., assortative mating or indirect genetic effects) present in the discovery GWAS that do not generalize to other ancestries.

5. Significance and Implications

• Equity in Genomic Medicine: The findings highlight that the "one-size-fits-all" application of EUR-trained PGIs is particularly detrimental to African populations and for behavioral traits, potentially exacerbating health disparities.
• Refining Prediction Models: The study demonstrates that simply increasing sample size is insufficient; the type of GWAS (family-based vs. standard) and the trait category (biological vs. behavioral) critically determine portability.
• Future Directions:
  • Diverse GWAS: There is an urgent need for larger, diverse GWAS datasets to capture population-specific causal variants and LD structures.
  • Family-Based Designs: Utilizing family-based GWAS designs could mitigate confounding biases that currently limit the portability of standard PGIs.
  • Contextual Awareness: Researchers must account for the fact that behavioral traits are less portable, likely due to complex gene-environment interplay that varies by culture and environment, not just genetics.

In conclusion, this paper provides a comprehensive dissection of why PGIs fail to generalize, identifying that while genetic architecture (LD/MAF) is the primary culprit for African populations, behavioral traits suffer from additional layers of complexity related to environmental context and methodological biases in standard GWAS.