Taxonomic profilers and their influence on metagenomic diversity analyses

Imagine you are a detective trying to solve a mystery: "Who lives in this tiny, invisible city?" In the world of microbiome research, that city is a sample of dirt, saliva, or poop, and the residents are billions of tiny microbes. To find out who lives there, scientists use special software tools called taxonomic profilers. These tools act like super-fast librarians, scanning millions of DNA "pages" and matching them against a giant reference library of known microbes to create a guest list.

But here's the twist: just like different librarians might use different catalogs or have different rules for what counts as a "guest," these software tools don't always agree on the guest list. This paper, led by Jonathan Rondeau-Leclaire and friends, decided to put four of the most popular "librarians" (mOTUs, MetaPhlAn, Kraken2, and Sourmash) to the test. They didn't just look at fake, made-up data; they dug into 1,211 real metagenomes from eight different datasets, ranging from human guts and skin to moss and bee guts.

The Great Guest List Discrepancy

The team found that the choice of librarian changes the story significantly. If you ask one tool to count the guests, it might say, "Wow, there are 500 different species!" Ask a different tool, and it might say, "Actually, I only see 200."

The paper discovered that alpha diversity (a fancy way of saying "how many different types of guests are in the room and how evenly they are spread out") is extremely sensitive to which tool you pick.

• The "DNA-to-Markers" Librarians: Tools like MetaPhlAn and mOTUs look for specific "name tags" (marker genes) on the microbes. Even when using the same type of library, these two tools gave different diversity counts. For example, mOTUs4 found higher diversity than MetaPhlAn4 in 95.5% of samples when looking at Shannon diversity.
• The "DNA-to-DNA" Librarians: Tools like Kraken2 and Sourmash compare the whole DNA sequence, like matching entire paragraphs. These showed even wilder swings. When using the same database, Kraken2 often reported way more species than Sourmash. In fact, with a specific setting, Kraken2 found an average of 1,167 ± 625.7 species, while Sourmash found only 192 ± 48.69.

The authors suggest that these differences aren't just random noise; they are driven by the tools' internal logic. For instance, Kraken2 is like a librarian who says, "If I see even one tiny clue, I'll list that guest," while Sourmash is more strict, saying, "I need a whole pile of evidence before I add a name to the list."

The Library Matters Too

It's not just the librarian; it's also the library they are using. The paper tested two massive reference databases: RefSeq (based on NCBI taxonomy) and GTDB (based on phylogenomic relationships).

• The GTDB library was much bigger, containing over 113,104 species, compared to RefSeq's 27,285.
• Unsurprisingly, using the bigger library (GTDB) resulted in finding more diversity. The paper notes that if you use GTDB, the tool might group Shigella bacteria under E. coli, changing the entire guest list structure.

The "So What?" Moment: Do the Conclusions Change?

Here is the most critical part of the story. Does it matter if the guest lists are different? Yes, sometimes.

When the researchers asked, "Is the diversity different between sick people and healthy people?" the answer depended entirely on which tool they used.

• In the Gut PD (Parkinson's disease) dataset with 719 samples, the difference between groups was statistically significant for 9 out of 13 methodologies. But the "strength" of that difference (the effect size) varied wildly, ranging from 0.04 to 0.13.
• In the Gut NAFLD dataset, only one specific setup (Kraken with RefSeq at a high confidence level) found a significant difference, while the others said, "Nope, nothing here."
• The paper explicitly argues against the idea that all these tools are interchangeable. They show that alpha diversity analysis can be driven by methodological choices, meaning a researcher could accidentally (or intentionally) pick a tool that makes their hypothesis look true or false.

The Good News: Beta Diversity is Steadier

While the "count" of guests (alpha diversity) was shaky, the "arrangement" of the guests (beta diversity) was more stable. Beta diversity asks, "Do the guests in Group A look different from the guests in Group B?"

• The authors found that even though the specific numbers changed, the statistical conclusions about whether groups were different usually stayed the same.
• For example, in the Gut RA (Rheumatoid Arthritis) dataset, most tools agreed that the groups were distinct, even if they disagreed on exactly how many species were present.
• However, the paper notes that using the RefSeq database with Kraken sometimes led to different results than using GTDB, suggesting that a bigger, more complete library might help tools agree better on the big picture.

The Takeaway

The authors conclude that there is no single "best" tool that works for everything. The "right" choice depends on your specific question, your sample type, and the library you have access to.

They warn researchers against "cherry-picking" a tool just because it gives the result they want. Instead, they suggest:

1. Be transparent: Clearly state which tool, database, and settings you used.
2. Check your work: Run a "sensitivity analysis" to see if your main conclusion holds up if you swap the tool or the database.
3. Don't compare apples to oranges: You can't directly compare a diversity number from a study using Tool A with a study using Tool B, because the numbers themselves are influenced by the software.

In short, the paper suggests that while these tools are powerful, they are not magic wands. They are lenses, and different lenses can show you different versions of the same microscopic world. To get the truest picture, scientists need to know exactly which lens they are looking through.