Occam's bias undermines inferences from phylogenetic linear models

This paper argues that traditional phylogenetic linear models suffer from "Occam's bias," a statistical distortion caused by neglecting relationships among predictors, and demonstrates through an analysis of vertebrate extinction risk that adopting multi-response models is essential for accurate ecological and evolutionary inference.

The Gist

The Big Idea: The "Too Simple" Trap

Imagine you are a detective trying to solve a mystery: Why are some animals more likely to go extinct than others?

For decades, scientists have used a standard tool called a "Single-Response Model" (SR). Think of this tool like a one-way street. You point the camera at the animal's extinction risk (the outcome) and ask, "Does big body size cause this?" or "Does having many babies cause this?"

The problem, according to this new paper, is that nature isn't a one-way street. It's a busy, tangled web of intersections. Animals don't just have body size; they have body size, and how many babies they have, and how big their territory is. All of these things influence each other.

The authors call the mistake of ignoring these connections "Occam's Bias."

The Analogy:
Imagine you are trying to figure out why a car is moving fast.

• The Old Way (SR Model): You look at the gas pedal and say, "The gas pedal makes the car go fast!" You ignore the engine, the transmission, and the driver.
• The Problem: The gas pedal is connected to the engine. If you don't account for the engine, you might think the gas pedal is doing all the work, or you might get the direction of the force wrong.
• The New Way (MR Model): You look at the whole car. You see how the gas pedal, the engine, and the driver all talk to each other to make the car move.

What is "Occam's Bias"?

The name comes from "Occam's Razor," a famous rule that says the simplest explanation is usually the best. But the authors argue that in biology, trying to be too simple actually creates a lie.

When scientists use the "one-way street" model, they accidentally create ghost relationships.

• False Positives: They might conclude that "Big body size causes extinction" when, in reality, it's just that big animals also tend to have fewer babies, and fewer babies is what actually causes the risk. The model gets confused and blames the wrong thing.
• False Negatives: They might miss a real connection because the model is too busy looking at the wrong things.
• Direction Swaps: Sometimes, the model gets the answer completely backward. It might say "Big animals are safer" when they are actually in more danger, simply because it didn't account for how body size interacts with other traits.

The "Tangled Web" Experiment

To prove this, the authors looked at 13,949 species of land animals (frogs, lizards, mammals, and birds). They asked: Do life traits like body size, number of offspring, and territory size affect extinction risk?

They ran the test two ways:

1. The Old Way (SR): They asked, "Does body size affect extinction?" while just listing other traits as "background noise."
2. The New Way (MR): They used a "Multi-Response Model" that treated body size, offspring, and territory as a team of friends who all influence each other and the extinction risk simultaneously.

The Results:

• The Old Way gave messy, contradictory, and often wrong answers. For example, it couldn't agree on whether being big was good or bad for survival.
• The New Way cleared up the confusion. It showed that:
  • Small animals (like frogs) are in trouble if they are too big.
  • Large animals (like mammals) are in trouble if they are too big.
  • Having fewer babies is a major risk for everyone.
  • Having a small territory is a major risk for everyone.

The "New Way" matched what biologists actually expect to see in nature. The "Old Way" was just creating statistical ghosts.

Why Does This Happen? (The "Adding More Variables" Trap)

You might think, "If I add more variables to my model, I'm being more careful, right?"
The authors say: No, not if you don't map the connections.

They explain that for every new variable you add, the number of hidden, unmeasured connections between them grows exponentially (like a snowball rolling down a hill).

• If you add 3 variables, you might miss 3 connections.
• If you add 10 variables, you might miss 45 connections!

When you ignore these hidden connections, the math gets distorted. The more data you have (large sample sizes), the more confident the computer becomes in its wrong answer. It's like a GPS that is very confident it's taking you to the wrong house because it didn't account for the traffic patterns between the streets.

The Takeaway for Everyone

This paper is a wake-up call for scientists (and anyone who reads scientific studies).

1. Simplicity isn't always smart. In complex systems like evolution, ignoring how things connect to each other leads to bad conclusions.
2. Don't just "control" for variables. You can't just throw a variable into a model and say "I controlled for it." You have to model how it interacts with everything else.
3. We need better maps. We need to stop using "one-way street" models and start using "tangled web" models (Multi-Response models) to understand how life actually works.

In short: If you want to understand why animals are disappearing, you can't just look at one trait in isolation. You have to look at the whole messy, beautiful, interconnected web of life. If you don't, you might end up blaming the wrong culprit.

Technical Summary

1. The Problem: Occam's Bias in Phylogenetic Modeling

The authors identify a critical statistical flaw in the standard practice of using Single-Response (SR) phylogenetic linear models (e.g., PGLS, phylogenetic GLMMs) to test ecological and evolutionary hypotheses.

• The Core Issue: SR models regress a single response variable against multiple predictors but fail to explicitly model the causal relationships among the predictors themselves.
• Definition of Occam's Bias: The authors define "Occam's bias" as a statistical distortion where coefficients are biased because the model contains fewer cause-effect connections than predicted by biological theory. By omitting the relationships between predictors (e.g., genetic correlations, pleiotropy, or shared environmental drivers), the model implicitly "controls" for these relationships via partial correlation mechanisms, leading to spurious significance and incorrect coefficient directionality.
• The Misconception: The paper challenges the prevailing belief that adding more covariates (control variables) to SR models always improves inference. The authors argue that adding predictors increases the number of unaccounted indirect paths quadratically ($y = (x^2-x)/2$), thereby increasing Occam's bias and distorting effect sizes.
• Consequence: This bias can lead to Type I errors (false positives) and Type II errors (false negatives), as well as reversals in the sign (directionality) of coefficients, potentially invalidating a wide range of existing literature on biodiversity patterns.

2. Methodology

The study employs a dual approach: theoretical simulation and empirical re-analysis of a large-scale biological dataset.

A. Simulation Study

• Setup: The authors simulated a three-trait system (Trait 1, 2, and 3) with known phylogenetic correlations.
  • Trait 1 and Trait 2 were set to have zero correlation ($r_{12} \approx 0$).
  • Correlations between Trait 1-Trait 3 ($r_{13}$) and Trait 2-Trait 3 ($r_{23}$) were systematically varied from -0.7 to 0.7.
• Models Compared:
  • SR Model: Regressed Trait 1 on Trait 2 and Trait 3.
  • MR Model: A Multi-Response (Bayesian) model treating all three traits as response variables with an unstructured variance-covariance matrix, explicitly modeling all pairwise relationships.
• Derivation of the "Occam's Bias Inequality": The authors derived a mathematical inequality (Eq. 1) based on partial correlation formulas to predict when spurious significance occurs:
  $$|r_{13} \cdot r_{23}| > t\text{-critical} \cdot \frac{1}{\sqrt{n - k - 1}} \cdot \sqrt{(1 - r_{13}^2)(1 - r_{23}^2)}$$
  This inequality demonstrates that spurious significance arises when the product of the omitted correlations exceeds a threshold determined by sample size ($n$) and the significance level.

B. Empirical Case Study

• Dataset: A global database of 13,949 tetrapod species (amphibians, lizards, mammals, birds).
• Variables:
  • Response: Extinction risk (binary: Threatened vs. Non-threatened based on IUCN Red List).
  • Predictors: Life-history traits: Body mass, Fecundity, and Geographic range size.
• Analysis:
  • SR Approach: Ran four models regressing extinction risk on various combinations of the three predictors (Body mass only; Body mass + Fecundity; Body mass + Range; All three).
  • MR Approach: A single Bayesian phylogenetic generalized linear mixed model (using MCMCglmm) treating all four variables (including extinction risk) as a multivariate response. This allowed for the estimation of phylogenetic and residual covariances between all pairs of variables.
• Phylogenies: Used consensus trees from established databases (e.g., Jetz & Pyron, Phylacine, Birdtree).

3. Key Contributions

1. Formalization of Occam's Bias: The paper provides a theoretical framework and a mathematical inequality to quantify how omitting predictor-predictor relationships in SR models leads to statistical distortion.
2. Demonstration of Sample Size Paradox: The study reveals that larger sample sizes exacerbate Occam's bias. As $n$ increases, the significance threshold lowers, making it easier for the bias (driven by even weak correlations among predictors) to generate false positives.
3. Critique of Model Selection Metrics: The authors show that standard model selection criteria (like DIC) often favor models with more predictors (which have better predictive power but worse causal inference), misleading researchers into choosing models with higher bias.
4. Advocacy for Multi-Response (MR) Models: The paper argues that MR models, which explicitly estimate the covariance structure among all variables, are necessary to disentangle direct effects from indirect pathways in phylogenetic comparative methods.

4. Key Results

From Simulations

• False Significance: SR models frequently produced statistically significant relationships between Trait 1 and Trait 2 even when the true correlation was zero, provided that both were correlated with a third variable (Trait 3).
• Directionality Reversal: The sign of the coefficient in SR models could flip (positive to negative) depending on the signs of the correlations between the predictors and the omitted variable.
• Sample Size Effect: The threshold for inducing spurious results decreased as sample size increased, meaning large-scale studies are more susceptible to this bias than small ones.

From Empirical Data (Tetrapod Extinction Risk)

• MR Model Findings (The "Truth"):
  • Significant negative relationship between Fecundity and Extinction Risk (lower fecundity = higher risk).
  • Significant negative relationship between Geographic Range and Extinction Risk (smaller range = higher risk).
  • Body Mass: Showed a negative relationship with extinction risk in ectotherms (amphibians/lizards) but a positive relationship in endotherms (mammals/birds), aligning with ecological theory.
• SR Model Findings (The "Bias"):
  • Fecundity: In SR models, fecundity was consistently found to be non-significant, contradicting the MR results and life-history theory.
  • Body Mass: The direction and significance of body mass effects were distorted. For example, in amphibians, adding fecundity as a covariate shifted the body mass coefficient from negative (significant) to non-significant or even positive.
  • Mechanism of Bias: The distortion was mathematically explained by the partial correlation numerator ($r_{EB} - r_{EF} \cdot r_{BF}$). In ectotherms, the negative correlation between extinction and fecundity, combined with the positive correlation between body mass and fecundity, artificially inflated the body mass coefficient toward zero or positive values in SR models.

5. Significance and Implications

• Re-evaluation of Literature: The authors argue that a vast portion of phylogenetic comparative studies relying on SR models with multiple predictors may have produced unreliable inferences regarding the drivers of evolution and extinction.
• Paradigm Shift: The study calls for a shift from "controlling for" variables via SR models to "explicitly modeling" all causal pathways via MR models.
• Biological Interpretation: MR models allow researchers to distinguish between phylogenetic (between-clade) and residual (within-clade) covariation, offering a clearer biological picture than the weighted averages produced by SR models.
• Future Directions: The paper advocates for the continued development and adoption of Bayesian multi-response phylogenetic models to ensure that statistical inferences align with complex biological realities, particularly in the era of large-scale biodiversity datasets.

In conclusion, the paper posits that Occam's bias is a fundamental flaw in traditional phylogenetic linear modeling that cannot be solved by simply adding more covariates. Instead, it requires a structural change in modeling approaches to explicitly account for the network of relationships among all variables.