Substitution rate variation, not hidden paralogy, drives false hybridization signal in phylogenetic network inference

This simulation study demonstrates that substitution rate variation, rather than hidden paralogy, is the primary driver of false hybridization signals in phylogenetic network inference, particularly biasing the find_graphs method and necessitating empirical calibration of statistical thresholds.

The Gist

Imagine you are trying to draw a family tree for a group of reptiles. You want to know if any of them "mixed families" (hybridized) in the past, or if they just branched off cleanly like a standard tree. Scientists use special computer programs to look at DNA and make this guess. But sometimes, these programs get confused and draw a messy web instead of a clean tree, even when no mixing actually happened.

This paper is like a detective story where the researchers set up a series of "fake" DNA scenarios to see what tricks the computer programs fall for. They wanted to find out: Is the computer getting confused because it's looking at the wrong copies of genes (hidden paralogy), or because some genes are just evolving at different speeds (substitution rate variation)?

Here is what they found, using some everyday analogies:

The Two Suspects

1. Hidden Paralogy (The "Wrong Photo Album"): Imagine you are trying to identify a person, but you accidentally grab a photo of their twin instead. In genetics, this is when scientists accidentally compare two different copies of a gene that look similar but aren't the direct parent-child pair they think they are.
2. Rate Variation (The "Speeding Cars"): Imagine a race where some cars drive at a steady 60 mph, while others speed up to 120 mph or slow down to 20 mph depending on the road they are on. In genetics, this means some DNA changes very fast in certain lineages, while others change slowly.

The Experiment
The researchers built a computer simulation based on a real reptile family tree. They created fake DNA data with different levels of "wrong photos" and different levels of "speeding cars." Then, they ran two popular computer programs (let's call them Program A and Program B) to see if they could correctly identify that the family was actually a clean tree, not a messy web.

The Results

• The "Wrong Photo Album" wasn't the problem: Even when the researchers messed up the data with lots of hidden paralogy (the wrong photos), the computer programs were surprisingly smart. They correctly ignored the noise and said, "No, this is just a normal tree; there is no hybridization." Another tool they used (ASTRAL) got it right every single time. So, accidentally picking the wrong gene copy isn't what's causing false alarms about hybridization.

• The "Speeding Cars" caused the chaos: This is where things went wrong. When the researchers introduced "lineage-specific rates" (some DNA lines speeding up or slowing down), Program A got very confused. It started seeing patterns that looked like hybridization, even though none existed. It was like a detective seeing a shadow and thinking it's a ghost, just because the lighting was weird. The program's error scores went way past the "safe zone" limit.

• Program B was more careful: The second program (SNaQ) was much better at ignoring the speed changes. It almost always correctly said, "This is just a tree." However, when it did try to draw a hybrid web, it was less sure of the exact shape of the tree when the speeds were varying.

The Big Takeaway
The paper concludes that the main reason scientists might falsely claim a species hybridized is not because they picked the wrong gene copies, but because different parts of the DNA evolved at different speeds.

Furthermore, the researchers found that the standard "rule of thumb" used to decide if a result is a real hybrid (a specific error score of 3) is actually too strict. Even without any speed variations, this rule often makes the program cry "Wolf!" when there is no wolf. They suggest that instead of using a one-size-fits-all rule, scientists should calibrate their own "safe zones" for every specific group of animals they study.

In short: Don't blame the wrong gene copies for fake hybridization signals; blame the fact that some DNA evolves faster than others. And if your computer program says you found a hybrid, double-check your rules before celebrating.

Technical Summary

Technical Summary

Problem Statement
Phylogenetic network inference methods are increasingly utilized to detect hybridization and gene flow from genomic data. However, the robustness of these methods to common sources of model violation remains poorly characterized. Specifically, there is a need to distinguish whether signals of reticulation (hybridization) inferred by these methods are genuine biological events or artifacts caused by confounding factors such as hidden paralogy (undetected gene duplication and loss) and substitution rate variation across lineages or genes.

Methodology
The authors conducted a simulation study to evaluate the performance of two widely used network inference methods: find_graphs from ADMIXTOOLS 2 and SNaQ. The study utilized an eight-taxon species tree calibrated from an empirical reptile phylogeny as the ground truth. Data were simulated under a factorial design combining:

• Hidden Paralogy: Ranging from none to strong levels.
• Substitution Rate Variation: Three levels including no variation, gene-specific variation, and lineage-specific variation.

The study assessed the ability of the network methods to correctly favor a tree model (no reticulation) when no hybridization was present, and compared these results against the performance of ASTRAL in recovering the correct species tree.

Key Results

• Impact of Hidden Paralogy: Under the conditions examined, hidden paralogy had a limited impact on network inference. Both find_graphs and SNaQ correctly favored a tree model without reticulation, and ASTRAL consistently recovered the correct species tree.
• Impact of Substitution Rate Variation: Lineage-specific rates severely biased the find_graphs method, inflating the worst f-statistic residuals well beyond standard acceptance thresholds. In contrast, SNaQ correctly selected a tree model in almost all conditions; however, under lineage-specific rates, the probability of the network with $h=1$ reticulation displaying the true species tree decreased.
• Threshold Issues: The study demonstrated that the standard "worst residuals" threshold of 3 for find_graphs produces inflated type I error rates even in the absence of rate variation.

Key Contributions and Claims
The primary contribution of this work is the identification of substitution rate variation, rather than hidden paralogy, as the primary driver of false hybridization signals in these specific network inference contexts. The authors claim that the standard statistical threshold of 3 for find_graphs is insufficient for controlling type I error and recommend that researchers empirically calibrate this threshold within each specific study system.

Significance
The paper highlights that model violations, particularly lineage-specific rate heterogeneity, can lead to erroneous inferences of hybridization when using find_graphs. By demonstrating that hidden paralogy is less detrimental than rate variation under these specific simulation conditions, the study refines the understanding of limitations in current phylogenetic network tools. The recommendation for empirical calibration of statistical thresholds aims to improve the reliability of hybridization detection in future genomic studies.