Summary statistics and approximate bayesian computation are comparable to convolutional neural networks for inferring times to fixation

This study demonstrates that machine learning models trained on raw genotype data do not outperform traditional summary statistics in inferring the time to fixation of hard selective sweeps, suggesting that few undiscovered signals remain in single-population genomic data for distinguishing sweep timing from sweep age.

The Gist

The Big Question: Can AI Find Clues We Missed?

Imagine you are a detective trying to solve a crime. You have a crime scene (a specific spot in a genome) where a "beneficial mutation" (a helpful genetic change) recently took over the population. This is called a selective sweep.

You have two main pieces of information to figure out:

1. How long did the takeover take? (Did it happen in a flash, or was it a slow, grinding struggle?)
2. How long ago did the takeover finish? (Did it happen yesterday, or 1,000 years ago?)

The problem is that these two things look almost identical to the naked eye. A fast takeover that happened a long time ago leaves the same messy fingerprints as a slow takeover that happened recently. This is the "non-identifiability" problem mentioned in the paper.

The Old Way: The Summary Statistic Detective

For decades, population geneticists have used a set of standard tools to solve this. Think of these as standardized checklists (called summary statistics).

• They measure things like "How much genetic diversity is left?" or "How similar are the neighbors?"
• It's like a detective measuring the length of a footprint, the depth of a shoe print, and the mud type.
• These methods work well, but they rely on the detective knowing exactly what to measure beforehand. If there's a clue the detective didn't think to look for (like a specific type of tire track), they miss it.

The New Way: The AI Detective (Neural Networks)

Enter Machine Learning (ML), specifically Convolutional Neural Networks (CNNs).

• Instead of giving the AI a checklist, you hand it the raw crime scene photos (the raw genetic data).
• The AI is like a super-powered detective that looks at the entire picture at once. It doesn't need to be told what to look for; it learns to spot patterns on its own.
• The Hope: The researchers hoped the AI would find "hidden clues" in the raw data that the old checklists missed, allowing it to perfectly distinguish between a "fast/old" event and a "slow/young" one.

The Experiment: The Simulation Lab

To test this, the researchers built a massive virtual laboratory.

• They used a computer program to simulate 200,000 different evolutionary stories.
• They created 5 different "worlds" (demographic scenarios): some where the population size stayed constant, some where it grew, some where it shrank, and some where it chaotically bounced up and down.
• In every simulation, they knew the true answer: exactly how long the takeover took and exactly how long ago it finished.

They then trained three types of detectives on this data:

1. The Old School: Using only the standard checklists (Summary Statistics).
2. The Hybrid: A neural network that looked at the checklists (DNN).
3. The Raw Data Pro: A neural network that looked at the raw images of the genetic data (CNN).

The Results: The AI Didn't Win

The researchers expected the AI (CNN) to crush the competition. They thought, "Surely, looking at the raw data will reveal secrets the checklists can't see!"

But the results were surprising:

• The AI and the Old School were tied. The neural networks trained on raw data performed no better than the methods using the standard checklists.
• In fact, in one chaotic scenario, the AI actually did worse than the checklist method.

The Takeaway: The Clues Are Already Known

What does this mean for the real world?

1. No Hidden Treasures: It suggests that for a single snapshot of a population's DNA, there are likely no secret, undiscovered clues left in the data that can help us separate "how long it took" from "how long ago it happened."
2. The Checklists are Enough: The standard "checklist" methods (Summary Statistics) are already capturing almost all the useful information available in that specific type of data.
3. The Limit is the Data, Not the Tool: The reason we can't perfectly tell the difference between a fast/old sweep and a slow/young one isn't because we lack a better AI. It's because the genetic data itself simply doesn't contain enough information to tell them apart once time has passed.

The Analogy Summary

Imagine trying to guess how long it took to bake a cake and how long ago it came out of the oven, just by looking at a photo of the cake.

• The Old Method: You measure the cake's height and color.
• The AI Method: You feed the photo to a super-computer that analyzes every pixel.

The study found that even the super-computer couldn't guess better than the simple measurements. Why? Because a cake that was baked slowly and cooled for a long time looks exactly like a cake baked quickly and cooled for a short time. The "clue" isn't missing; the clue just doesn't exist in the photo.

Conclusion: While AI is powerful, it can't magic up information that isn't there. For this specific genetic puzzle, the old, trusted methods are just as good as the newest, flashiest technology.

Technical Summary

1. Problem Statement

The central problem addressed is the difficulty of inferring the time to fixation ($t_f$) of a beneficial allele (a hard selective sweep) from genomic data, specifically distinguishing it from the sweep age ($t_a$), which is the time elapsed since fixation until sampling.

• The Challenge: There is a statistical non-identifiability issue where different combinations of $t_f$ and $t_a$ produce identical genetic signatures. For example, a "slow" sweep that fixed recently ($high\ t_f, low\ t_a$) leaves similar patterns of diversity and linkage disequilibrium (LD) as a "fast" sweep that fixed long ago ($low\ t_f, high\ t_a$).
• The Hypothesis: While traditional methods rely on pre-defined summary statistics (e.g., Tajima's D, $\pi$, haplotype frequencies), Machine Learning (ML) models, particularly Convolutional Neural Networks (CNNs), can learn directly from raw genotype data. The authors hypothesized that CNNs might uncover "undiscovered" signals in the raw data that allow for better disentanglement of $t_f$ and $t_a$ than summary statistics alone.

2. Methodology

Simulation Framework

• Tool: The authors used SLiM (v4.0.1) for forward-time evolutionary simulations.
• Scale: Approximately 250,000 simulations were generated across 5 distinct demographic scenarios:
  1. Constant population size.
  2. Population growth.
  3. Population decay.
  4. Cyclic population size changes.
  5. Chaotic population size changes.
• Parameters: Simulations varied in effective population size ($N_A$), selection coefficient ($s$), dominance ($h$), mutation rate ($\mu$), recombination rate ($R$), and sweep age ($t_a$).
• Data Generation: For each simulation, a beneficial mutation was introduced. The time to fixation ($t_f$) was recorded. After fixation, the population evolved for $t_a$ generations before a sample of $n=128$ individuals was taken.
• Data Balancing: Simulations were downsampled to ensure a uniform distribution of $\log_{10}(t_f)$ across the range of ~50 to 20,000 generations.

Model Architectures

The study compared three distinct inference frameworks trained on the simulated data to predict $t_f$:

1. Approximate Bayesian Computation (ABC):

   • Input: A vector of 17 pre-defined summary statistics calculated from the raw data (e.g., nucleotide diversity $\pi$, Tajima's D, Watterson's $\theta_W$, haplotype statistics $h_1, h_2, h_{12}$, etc.).
   • Method: Used the rabc package in R, testing 63 configurations (varying regression methods, tolerance levels, and posterior point estimates).

2. Dense Neural Networks (DNN):

   • Input: The same 17 summary statistics as the ABC model.
   • Architecture: A standard feed-forward neural network with an input layer of 17 neurons, followed by three dense layers with dropout.
   • Purpose: To serve as a control to see if the neural network architecture itself (vs. the regression method in ABC) provided an advantage when using summary statistics.

3. Convolutional Neural Networks (CNN):

   • Input: Raw genotype matrices converted into grayscale images.
     • Rows: 128 sampled individuals.
     • Columns: Up to 128 SNPs closest to the sweep site.
     • Preprocessing: Rows were clustered by Manhattan distance to emphasize haplotype structure; unphased genotypes were encoded (0/0=black, 0/1=grey, 1/1=white).
   • Architecture: A dual-branch network.
     • Branch 1 (Image): Three convolutional layers (kernel sizes 7 and 3) with pooling and dropout.
     • Branch 2 (Position): Processes the normalized SNP positions.
     • Output: Concatenated branches fed into dense layers to predict $\log_{10}(t_f)$.
   • Optimization: Used Bayesian hyperparameter tuning (60 iterations) and Monte-Carlo sampling (100 iterations) to estimate prediction uncertainty.

Evaluation

Models were trained on 80% of the data, validated on 10%, and tested on the remaining 10%. Performance was measured using the Pearson correlation coefficient ($r$) between predicted and true $t_f$ values.

3. Key Results

• Performance Parity: Across all five demographic scenarios, CNNs performed comparably to ABC and DNNs.
  • In the constant population scenario, all three models achieved Pearson correlations $r > 0.7$.
  • The 95% confidence intervals for performance overlapped significantly (e.g., Constant: CNN $r \in [0.705, 0.750]$, ABC $r \in [0.731, 0.773]$).
• Failure to Discover New Signals: The CNNs, despite having access to raw data, did not outperform the methods relying on summary statistics. This suggests that the 17 summary statistics used capture nearly all the information available in single-timepoint, single-population genotype data regarding $t_f$ and $t_a$.
• Demographic Sensitivity:
  • In the cycling demographic scenario, CNNs performed significantly worse ($r=0.656$) than DNNs ($r=0.728$) and ABC ($r=0.691$). This implies that for complex demographies, summary statistics may provide robust features that raw-data CNNs struggle to learn without massive data augmentation.
• Bias in Prediction: All models exhibited a bias where they overestimated $t_f$ for sweeps with short fixation times but long ages ($t_a > 1000$), and underestimated $t_f$ for very fast sweeps. This confirms the inherent non-identifiability of the problem.
• Partial $R^2$ Analysis: Individual summary statistics explained very little unique variation in $t_f + t_a$ (most partial $R^2 < 0.07$), indicating high redundancy among statistics, yet the combination of them was sufficient to match the CNNs.

4. Key Contributions

1. Empirical Benchmarking: Provides a rigorous, large-scale comparison (200k+ simulations) between raw-data deep learning (CNNs) and traditional summary-statistic-based inference (ABC/DNN) for a specific, difficult population genetics task.
2. Negative Result with High Value: The finding that CNNs do not improve upon summary statistics is significant. It suggests that for inferring $t_f$ in single-population, single-timepoint data, the "black box" nature of CNNs does not yield new biological insights beyond what is already captured by established statistics like haplotype homozygosity and site frequency spectrum metrics.
3. Open Science Workflow: The authors released a complete Snakemake workflow and Docker container containing all simulation, training, and analysis code, ensuring full reproducibility of the 250,000 simulations and model training.

5. Significance and Implications

• Limitations of "End-to-End" Learning: The study challenges the assumption that deep learning on raw genomic data will automatically uncover novel patterns missed by human-designed statistics. In the context of hard sweeps, the signal for $t_f$ appears to be fully encapsulated by existing summary statistics.
• Practical Guidance for Researchers: For researchers aiming to infer sweep timing in non-model organisms (where phased data or time-series data is unavailable), using summary statistics with ABC or DNNs is likely sufficient and potentially more robust (as seen in the cycling demographic) than training complex CNNs on raw genotype images.
• Future Directions: The authors suggest that to find "undiscovered" signals, future ML approaches might need:
  • Access to additional data types (e.g., spatial distribution of genotypes, time-series data).
  • Training on more complex or diverse demographic histories.
  • Architectural constraints that penalize the model for reproducing known summary statistics, forcing it to seek genuinely novel features.

In conclusion, the paper demonstrates that while ML is powerful, for the specific task of disentangling fixation time from sweep age in standard genomic datasets, traditional summary statistics remain a highly effective and computationally efficient baseline that current CNN architectures cannot easily surpass.