Deep coalescent history of the hominin lineage

By leveraging high-quality telomere-to-telomere genome assemblies, this study extends coalescent-based inference to deep time scales, revealing contemporaneous ancestral population size peaks in humans, chimpanzees, and bonobos that suggest a complex speciation process rather than a clean divergence from a panmictic common ancestor.

The Gist

Imagine human history not as a straight line, but as a tangled ball of yarn. For a long time, scientists have been trying to unravel this yarn to figure out how big our family was at different points in the past. They use a tool called PSMC (Pairwise Sequentially Markovian Coalescent), which is like a time machine that looks at the DNA of a single person and tries to guess how many ancestors they had thousands or millions of years ago.

However, this time machine has a blurry lens. It works great for the last 1 or 2 million years, but when you try to look further back, the picture gets fuzzy. It's like trying to read a book where the ink has faded; you can see the recent chapters, but the ancient ones are hard to decipher.

The Big Discovery: A "Ghost" Peak in the Past

In this new study, the researchers used a brand-new, ultra-high-definition version of the human genome (called a T2T assembly, which means they finally filled in all the missing gaps in the genetic map). With this crystal-clear lens, they looked way back in time—between 3 and 6 million years ago.

What they found was a surprise: a massive "peak" in the human population size. It looks like our ancestors suddenly swelled to a huge number (around 32,500 breeding individuals) and then shrank again.

But here is the twist: This wasn't just a human thing.

When they looked at the DNA of chimpanzees and bonobos, they saw the exact same peak at the exact same time. However, when they looked at gorillas and orangutans, that peak was completely missing.

The Detective Work: Ruling Out False Alarms

Before shouting "Eureka!", the scientists had to make sure this wasn't a glitch. They asked: Could this be a mistake caused by our tools?

They tested for all the usual suspects that mess up DNA analysis:

• Repeats: Like finding the same sentence repeated over and over in a book, which confuses the reader. (They checked and said: "Nope, not the cause.")
• Mutation Speed: Did mutations happen faster in some places? (They checked and said: "Nope.")
• Natural Selection: Did survival of the fittest distort the numbers? (They checked and said: "Nope.")

After ruling out all the technical errors, they concluded: This peak is real.

The Big Question: Why did Humans, Chimps, and Bonobos all swell at the same time?

If humans, chimps, and bonobos all had this population boom at the same time, but gorillas and orangutans didn't, it suggests they were all part of the same "family drama" during that specific era.

The authors propose a theory called "Complex Speciation."

The "Divorce Party" Analogy

Imagine a large family (the common ancestor of humans, chimps, and bonobos) living in a big house.

• The Old Theory (The Clean Split): Imagine the family decides to split. The parents say, "Okay, you two kids (Human and Chimp) are moving out today. Goodbye, no more contact." They move into separate houses immediately.
• The New Theory (The Complex Split): Imagine the parents say, "You two are moving out, but you're going to live in neighborhoods that are still connected by a busy street. For a few million years, people are constantly walking back and forth between the two houses, visiting, trading, and having kids with each other. It's a messy, overlapping transition."

The "Peak" in the population size is the genetic signature of that busy street. Because the two groups were still mixing genes (even though they were starting to become different species), the genetic diversity looked huge. It's like if you took a census of a neighborhood where two distinct communities were still heavily intermingling; the population would look massive and interconnected.

Why does this matter?

1. It pushes back the clock: We can now confidently use DNA to study events that happened 6 million years ago, a time when the very first humans (or human-like creatures) were just starting to appear.
2. It changes the story of our origins: It suggests that humans didn't just "pop" out of existence as a separate species from a clean split with chimps. Instead, our evolution was a slow, messy process of separation where we and our closest cousins were still exchanging genetic material for a long time.
3. It's a shared history: The fact that chimps and bonobos have the same signal means they were all part of this same ancient, structured population before they finally went their separate ways.

In a Nutshell

Think of this paper as upgrading from a grainy black-and-white photo to a 4K video of our ancient past. The video reveals that for millions of years, the ancestors of humans, chimps, and bonobos weren't just two separate groups drifting apart. They were a complex, interconnected web of populations that slowly untangled, leaving a giant "fingerprint" in our DNA that we can finally see clearly.

Technical Summary

1. Problem Statement

Coalescent-based methods (e.g., PSMC, MSMC) are standard tools for inferring historical effective population sizes ($N_e$) from genomic data. However, these methods have historically been limited to inferring demographic history within the last 1–2 million years (Mya).

• The Gap: There is a lack of reliable resolution for events deeper than 2 Mya, such as the emergence of the genus Homo (3 Mya), Australopithecus (4.5 Mya), and the divergence of the human-chimpanzee-bonobo lineage (~6 Mya).
• Limitations of Previous Studies:
  • Data Quality: Previous analyses relied on short-read data aligned to incomplete reference genomes, which often collapsed or omitted difficult regions (repeats, segmental duplications), introducing reference bias and errors.
  • Model Violations: It was unclear whether signals observed at deep time scales were genuine demographic events or artifacts of model violations (e.g., variable mutation rates, recombination hotspots, background selection, or repeats).
  • Conflicting Signals: An ancient peak in effective population size around 3–6 Mya had been observed in previous PSMC analyses of humans, chimpanzees, and bonobos but was often dismissed or ignored due to concerns about its reliability.

2. Methodology

The authors employed a combination of high-quality genomic data, refined computational pipelines, and extensive simulation studies to re-evaluate deep-time demographic inference.

• Data Sources:
  • Utilized Telomere-to-Telomere (T2T) genome assemblies, which provide complete, haplotype-resolved, and highly accurate reference genomes.
  • Species Analyzed: Humans (HG002 and 26 individuals from the 1000 Genomes Project), Chimpanzee (mPanTro3), Bonobo (mPanPan1), Gorilla (mGorGor1), and Orangutans (Bornean and Sumatran).
• Computational Pipeline:
  • Variant Calling: Used dipcall to align alternate assemblies to primary assemblies, masking low-quality regions, indels, repeats, and segmental duplications.
  • Inference Engine: Applied MSMC2 (Multiple Sequentially Markovian Coalescent) to infer $N(t)$ curves.
  • Parameter Optimization: Used 64 time bins with fewer constraints and more iterations than standard implementations to increase resolution at deep time scales.
  • Validation:
    • Non-parametric Bootstrapping: Performed 20 iterations of block bootstrapping to assess confidence intervals.
    • Independent Method: Used Relate (which infers Ancestral Recombination Graphs without SMC assumptions) to cross-verify the existence of ancient coalescent material.
• Robustness Testing (Model Violations):
  • Conducted extensive simulations to test if the observed ancient peaks were artifacts of:
    • Repeat regions and segmental duplications.
    • Spatially variable mutation rates.
    • Temporal changes in mutation rates (rate slowdown).
    • Variable recombination rates and hotspots.
    • Recurrent mutations.
    • Background selection (BGS) and balancing selection.

3. Key Results

A. Discovery of a Consistent Ancient Peak

• Humans, Chimpanzees, and Bonobos: All three species exhibit a distinct, robust peak in inferred effective population size between 3 and 6 Mya.
  • In humans, the peak starts around 3 Mya, reaches a maximum of ~32,500 at ~4.5 Mya, and declines toward 6 Mya.
  • Chimpanzees and bonobos show a similar temporal pattern, though with higher peak magnitudes.
• Absence in Other Apes: No such peak is observed in Gorillas or Orangutans. This is consistent with their earlier divergence from the human-chimpanzee-bonobo clade (~10 Mya for gorillas, ~19 Mya for orangutans), suggesting the signal is specific to the shared ancestry of the Homo-Pan lineage.

B. Data Sufficiency for Deep Inference

• The authors demonstrated that sufficient genetic material exists to infer events at this depth.
  • At 5 Mya, approximately 0.5% to 1% of the genome remains uncoalesced in humans, chimps, and bonobos.
  • Even at 10 Mya, ~0.06% of the human genome remains uncoalesced.
  • African populations harbor thousands of non-contiguous segments with coalescence times older than 4 Mya, providing the statistical power necessary for robust inference.

C. Ruling Out Artifacts

Simulation studies and correlation analyses confirmed that the observed peaks are not artifacts of common model violations:

• Repeats: Masking repeats and segmental duplications did not eliminate the peak.
• Mutation Rate Variation: While variable mutation rates can cause bias, the degree of variation required to generate a false peak of this magnitude exceeds observed biological variation. Furthermore, the peak is consistent across populations, whereas mutation rate artifacts would likely vary stochastically.
• Selection: Background selection and balancing selection were analyzed; while they affect local diversity, they do not generate the specific, widespread ancient peak observed across the genome.
• Conclusion: The peak is a genuine signal of deep evolutionary history, not a methodological artifact.

4. Interpretation and Discussion

The authors propose that the contemporaneous peaks in humans, chimpanzees, and bonobos reflect complex speciation rather than a clean, instantaneous split from a panmictic (randomly mating) common ancestor.

• Complex Speciation Hypothesis: The peak likely represents a period of ancestral genetic structure where the Homo and Pan lineages were not fully separated. Instead, they may have existed as distinct subpopulations within a structured ancestral population that continued to exchange genes (gene flow) for a prolonged period before final reproductive isolation.
• Contrast with Previous Models:
  • Models assuming a clean split (e.g., standard CoalHMM) often estimate ancestral population sizes that are vastly inflated (>10x) compared to long-term estimates, likely because they misinterpret structure as a massive population size increase.
  • The PSMC peak aligns with the estimated divergence time of Homo and Pan (~6 Mya), supporting a scenario where gene flow persisted during the speciation process.
• Why no peak in other apes? Since gorillas and orangutans diverged earlier, they did not share this specific period of structured ancestry with the Homo-Pan clade.

5. Significance

• Extending the Time Horizon: This study proves that coalescent-based inference can reliably reach back 6–10 Mya when using high-quality T2T genomes, opening a new window into deep hominin evolution.
• Resolving Speciation Debates: The findings provide strong genomic evidence supporting the "complex speciation" model for humans and chimpanzees, suggesting that the separation of these lineages was a gradual process involving gene flow rather than a clean split.
• Methodological Validation: By rigorously testing and ruling out model violations, the paper establishes a framework for trusting deep-time demographic inferences, which were previously considered unreliable.
• Implications for Evolutionary History: The results suggest that the emergence of the genus Homo and the divergence of great apes were influenced by periods of population structure and gene flow, challenging the view of speciation as a simple bifurcation event.

In summary, the paper leverages T2T assemblies to reveal a shared, ancient demographic signal in humans, chimps, and bonobos, interpreting it as evidence of a complex, structured ancestral population during the critical period of hominin speciation.