EMITS: expectation-maximization abundance estimation for fungal ITS communities from long-read sequencing

EMITS is a high-performance Rust-based tool that utilizes an expectation-maximization algorithm to resolve ambiguous read mappings and improve species-level abundance estimation in fungal ITS communities from long-read sequencing, significantly outperforming naive counting methods in accuracy and error reduction.

The Gist

Imagine you are a detective trying to figure out who was in a crowded room based on a series of blurry, overlapping security camera footprints.

In the world of fungal biology, scientists use a specific "barcode" in DNA called ITS to identify different species of mushrooms and molds. Recently, scientists started using "long-read" sequencers (like Oxford Nanopore or PacBio) which are great at reading these barcodes in one go. However, there's a problem: many fungi are like cousins who look almost identical. Their DNA barcodes are so similar that the computer gets confused.

The Old Way: The "Best Guess" Mistake

Traditionally, when a computer sees a blurry footprint, it just picks the single best match it can find and says, "This is definitely Species A."

This is like a detective looking at a blurry photo and saying, "That looks like John, so it must be John," even though John's twin brother, Mike, looks 99% identical.

• The Problem: If the computer is wrong, it miscounts the crowd. It might think there are 100 Johns when there are actually 50 Johns and 50 Mikes.
• The Database Mess: Furthermore, the reference library (UNITE) has multiple entries for the same species (like having 20 different ID cards for John). The old method splits the count of "Johns" across all 20 cards, making it look like there are many different people when it's actually just one.

The New Solution: EMITS (The "Group Think" Detective)

The authors of this paper built a new tool called EMITS. Instead of making a snap judgment, EMITS uses a smart, iterative process called Expectation-Maximization (EM).

Think of EMITS as a detective who doesn't just pick one suspect. Instead, they:

1. Make a guess: "Okay, let's assume 50% of these footprints belong to John and 50% to Mike."
2. Check the evidence: They look at the footprints again. "If it's 50/50, does this blurry print look more like John or Mike?"
3. Adjust the guess: "Actually, this print looks slightly more like John, so let's shift the odds to 60% John."
4. Repeat: They do this over and over again, refining the numbers until the math settles on the most likely truth.

This allows the tool to say, "We can't be 100% sure on this specific footprint, but statistically, it's more likely to be John than Mike," and it distributes the "credit" fairly between the two.

How They Tested It

The team tested EMITS in three ways:

1. The Simulation: They created a fake digital crowd where they knew exactly who was there. They added "noise" (blur) to the data. The old method got confused and made huge mistakes, while EMITS stayed calm and accurate, reducing errors by up to 92%.
2. The "Mock" Community: They took a real jar of 10 known fungi, sequenced it, and let the tools guess. EMITS correctly identified the specific species in tricky groups (like Trichophyton and Penicillium), whereas the old method mixed them up.
3. The Synthetic Crowd: They created a complex mix of 21 species. EMITS not only found the right species but also stopped the computer from inventing "ghost" species that weren't there (reducing false alarms by 54%).

Why This Matters

• It's Fast: The tool is written in Rust, a programming language known for being incredibly fast and efficient.
• It's Smart: It knows that different cameras (sequencers) have different types of blur, so it has "presets" to adjust its detective work accordingly.
• It Cleans Up the Mess: It automatically combines the multiple ID cards for the same species so you get a clear count of the actual species, not just the database entries.

The Bottom Line

EMITS is a new, super-smart calculator for fungal DNA. It stops us from misidentifying look-alike fungi and stops us from double-counting the same species. By using a "group think" approach instead of a "best guess" approach, it gives scientists a much clearer, more accurate picture of the fungal world, which is crucial for medicine, agriculture, and ecology.

It works best when paired with another tool called ITSxRust, creating a complete, high-speed pipeline for analyzing fungal communities from long-read sequencing.

Technical Summary

1. Problem Statement

The paper addresses critical limitations in current fungal metabarcoding workflows using long-read sequencing (Oxford Nanopore Technologies and PacBio) targeting the Internal Transcribed Spacer (ITS) region.

• Misattribution of Abundance: Standard "naive best-hit" classification assigns reads entirely to the reference sequence with the highest alignment score. This fails when closely related species share highly similar ITS sequences (e.g., within Aspergillus, Fusarium, Penicillium, and Trichophyton), leading to reads being incorrectly attributed to the wrong species.
• Database Redundancy: Reference databases like UNITE contain multiple accessions (entries) for the same species. Naive counting fragments the abundance of a single species across these redundant entries, obscuring true species-level patterns.
• Lack of Probabilistic Frameworks: While Expectation-Maximization (EM) algorithms have been successfully applied to 16S rRNA gene analysis (e.g., EMU) to resolve these ambiguities, no equivalent tool existed for fungal ITS communities.

2. Methodology

The authors developed EMITS, a high-performance, Rust-based command-line tool that implements an Expectation-Maximization (EM) algorithm to estimate species-level abundances probabilistically.

• Input & Alignment: EMITS accepts PAF (Pairwise Alignment Format) output from minimap2. Crucially, it requires alignments to retain secondary hits (--secondary=yes), allowing the algorithm to consider reads that map to multiple potential species.
• The EM Algorithm:
  1. Likelihood Calculation: Alignment scores are normalized by query length and converted to likelihoods using a temperature-scaled exponential function: $L(i, t_k) = \exp(\frac{s_{ik}}{q_i \tau})$. The temperature parameter ($\tau$) controls sensitivity to score differences.
  2. E-Step (Expectation): Computes the posterior probability of a read belonging to a specific taxon, weighted by the current abundance estimates of all taxa.
  3. M-Step (Maximization): Updates abundance estimates by summing the fractional assignments of all reads to each taxon.
  4. Iteration: Steps 2 and 3 repeat until convergence (change in abundance < $10^{-6}$) or a maximum iteration limit is reached.
• Taxonomic Aggregation: Post-convergence, EMITS parses UNITE headers to collapse abundances from multiple accessions belonging to the same species, solving the database redundancy issue without requiring a pre-processed database.
• Platform Presets: To accommodate different error profiles of sequencing chemistries, EMITS includes pre-tuned parameters (temperature $\tau$ and minimum identity thresholds) for:
  • ONT R10.4.1 (Default)
  • ONT R9.4.1 (Relaxed)
  • PacBio HiFi (Strict)
  • ONT Duplex (Intermediate)

3. Key Contributions

• First EM Tool for Fungal ITS: EMITS is the first tool to adapt the EM framework specifically for fungal ITS amplicon data from long-read platforms.
• Probabilistic Resolution: It moves beyond deterministic "best-hit" assignment, treating taxonomic assignment as a latent variable to resolve ambiguous reads among congeneric species.
• Integrated Pipeline: It is designed to work seamlessly with ITSxRust (for upstream ITS extraction) and minimap2, forming a complete, high-performance pipeline for long-read fungal profiling.
• Open Source & Reproducible: The tool is open-source (Rust), available via Bioconda and Docker, and includes built-in simulation capabilities for validation.

4. Results

The authors validated EMITS using three complementary approaches:

A. Controlled Simulations

• Noise Robustness: In simulations with tunable alignment score noise (mimicking ONT errors), EMITS reduced L1 error by 80–92% compared to naive counting as noise increased.
• Stability: While naive counting error spiked with noise, EMITS maintained a stable, low error rate (~0.014), demonstrating robustness to the ambiguity inherent in long-read sequencing of related fungi.

B. ONT Mock Community (10 Species)

• Within-Genus Resolution: EMITS correctly resolved species in challenging genera where naive counting failed.
  • Trichophyton: Correctly assigned reads to T. mentagrophytes (2.2%) rather than misattributing them to T. simii (3.1% in naive).
  • Penicillium: Concentrated abundance on the correct species (P. flavigenum) instead of scattering it across P. paneum and P. roqueforti.
• Redundancy Handling: Successfully consolidated reads for Nakaseomyces glabratus (which had 13 UNITE accessions) into a single species estimate, whereas naive counting fragmented the abundance.

C. Synthetic Community (21 Species)

• Accuracy: EMITS detected all 21 expected species (100% sensitivity) and reduced overall L1 error by 13.4% compared to naive counting.
• False Positive Suppression: Reduced false positive abundance (reads assigned to species not present in the ground truth) by 54%. This was particularly effective in the Penicillium genus, reducing spurious assignments by 34%.

5. Significance

• Improved Taxonomic Resolution: EMITS significantly enhances the ability to distinguish between closely related fungal species, which is critical for clinical diagnostics (e.g., distinguishing pathogenic Aspergillus or Candida species) and ecological studies.
• Handling Database Complexity: By naturally aggregating redundant database entries, it provides cleaner, more accurate species-level abundance data without manual curation.
• Practical Adoption: The inclusion of platform-specific presets lowers the barrier to entry for users, ensuring the tool is optimized for the specific error profiles of ONT and PacBio chemistries.
• Complementary to Existing Tools: While amplification bias (PCR efficiency) remains a limitation for all classification methods, EMITS specifically addresses the classification and abundance estimation bottlenecks, offering a substantial improvement over current best-hit methods for taxonomically complex fungal communities.

In summary, EMITS represents a major methodological advancement for fungal metabarcoding, transforming long-read ITS data from a source of ambiguous, fragmented counts into a reliable, probabilistic estimate of species-level community composition.