Variable performances of commercial eDNA inventories challenge their use for surveying stream fish communities

This study demonstrates that commercial eDNA metabarcoding services exhibit highly variable performance and significant inaccuracies in detecting stream fish communities, challenging their current reliability for large-scale ecological assessments without standardized protocols and comprehensive reference databases.

The Gist

Imagine you are trying to take a perfect inventory of every single type of fish living in a small, tropical stream. You want to know exactly who is there, how many of them there are, and what the community looks like.

For a long time, scientists have been excited about a new high-tech tool called eDNA (environmental DNA). Think of eDNA like finding footprints in the mud. Instead of catching the fish to see what they are, scientists just scoop up a bucket of water. Since fish leave tiny bits of their genetic "skin cells" and waste in the water, they can analyze the water to see who has been swimming there recently. It's supposed to be easier, cheaper, and less harmful to the fish than catching them.

Because this sounds so great, many private companies have popped up offering to do this analysis for governments and researchers. They say, "Send us the water, and we'll tell you exactly what fish are in your stream."

The Big Experiment
The authors of this paper decided to put these companies to the test. They went to a small stream in French Guiana (a place with a huge variety of fish) and did two things:

1. They collected water samples and sent them to four different commercial companies, asking each to list the fish they found.
2. Immediately after, they used a traditional method (electrofishing, which gently stuns fish so they can be caught, counted, and identified) to get a "gold standard" list of exactly what was actually in that stretch of water.

The Shocking Results
The results were like asking four different people to describe a painting they saw for five seconds, and getting four completely different descriptions.

• The "Missed" Fish (False Negatives): The companies were terrible at finding the fish that were actually there. Depending on which company you used, they missed anywhere from 32% to 92% of the fish species that were actually swimming in the stream.
  • Analogy: Imagine a classroom with 50 students. If you asked a company to list the students, one company might say there are only 4 students, while another says there are 34. They missed almost everyone!
• The "Ghost" Fish (False Positives): The companies also claimed to find fish that weren't there at all. Some companies listed species that live in other countries or other rivers, never to have been seen in this specific stream.
  • Analogy: It's like looking at a photo of your living room and the company telling you, "We see a polar bear and a kangaroo in there," even though you only have a cat.
• The "Mystery" Differences: The four companies gave wildly different lists. One company found 48 species, while another found only 5. They didn't even agree on the fish they did find.

Why Did This Happen?
The researchers dug deeper and found the problem wasn't just the water sampling; it was the recipe each company used to cook the data.

• The Database Problem: To identify a fish from a DNA snippet, you need a massive library (database) of known fish DNA to compare it against. Some companies used small, incomplete libraries, while others used better ones. When the researchers took the raw data from the companies and re-analyzed it all using the same perfect library, the results got much better and more consistent.
• The "Small Fish" Problem: The companies were great at finding big, heavy fish (like a large shark or catfish) but terrible at finding small, rare fish.
  • Analogy: It's like trying to hear a conversation in a noisy room. You can easily hear the loud, shouting person (the big fish), but you completely miss the person whispering in the corner (the small, rare fish).

What Does This Mean for the Future?
The paper concludes that while eDNA is a powerful tool, we cannot trust commercial companies to give us a perfect list of fish right now.

If a government agency uses these reports to make decisions about protecting a river, they might think a river is empty of rare fish (because the company missed them) or that it has dangerous invasive species (because the company hallucinated them).

The Takeaway:
Think of eDNA as a rough sketch rather than a photograph. It can give you a general idea of who is in the room, but if you need an exact headcount for a legal decision, you still need to do the hard work of checking the room yourself.

The authors are calling for:

1. Standard Rules: All companies need to use the same "recipe" so their results are comparable.
2. Better Libraries: We need a complete, perfect library of fish DNA for every region.
3. Expert Review: You can't just take a company's list at face value; you need a fish expert to look at the results and say, "Wait, this fish doesn't live here," or "You missed that one."

Until these things happen, using eDNA alone to manage our rivers is like trying to navigate a stormy ocean with a map that has half the islands missing and a few dragons drawn in the wrong places.

Technical Summary

1. Problem Statement

Environmental DNA (eDNA) metabarcoding is increasingly promoted as a superior, non-invasive tool for biodiversity monitoring, often recommended to replace or supplement traditional capture-based methods (e.g., electrofishing). However, significant gaps exist in the validation of commercial eDNA services:

• Lack of Standardization: There is no universal protocol for sampling, DNA extraction, or bioinformatic analysis among commercial providers.
• Unknown Accuracy: While many studies compare eDNA to traditional methods, few compare eDNA results against an exhaustive, ground-truth inventory of a local community to quantify false negatives (undetected species) and false positives (erroneous detections).
• Commercial Opacity: Private companies often provide "turnkey" solutions with limited transparency regarding their bioinformatic pipelines, reference databases, and detection thresholds, making it difficult for regulators and scientists to assess data reliability.

The study aims to evaluate the performance of four commercial eDNA companies in a species-rich tropical stream by comparing their species lists against an exhaustive electrofishing inventory and current distribution data.

2. Methodology

The study was conducted in the Bastien stream, a tributary of the Maroni River in French Guiana, a region known for high fish diversity.

• Study Design:

  • eDNA Sampling: Water was sampled on March 18, 2024, following the specific guidelines of four different commercial companies (anonymized as A, B, C, D). This included using company-provided kits, filters, and preservation methods.
  • Blind Testing: Companies were asked only to inventory fish species; the study's objective and exact location were concealed to prevent bias.
  • Ground-Truth Inventory: Immediately after water sampling, a 100-meter section of the stream was enclosed with barrier nets. An exhaustive electrofishing survey was conducted using four passes with a high-voltage (1000V DC) device to capture all fish present. Captured fish were identified to species, measured, and weighed to calculate biomass.
  • Data Reanalysis: Three companies provided raw sequencing data (Illumina FastQ files). The authors re-analyzed this data using a standardized bioinformatic pipeline (OBITools, BLASTn) and a recently published, curated reference database of French Guianan fish (Brosse et al., 2026).

• Statistical Analysis:

  • Detection Rates: False negative rates (species present in the stream but missed by eDNA) and false positive rates (species detected by eDNA but absent in the stream) were calculated for each company.
  • Community Composition: Dissimilarity indices (Jaccard, Raup-Crick for presence/absence; Bray-Curtis, Horn-Morisita for abundance) were used to compare eDNA-derived communities against the electrofishing community.
  • Correlation: Relationships between species abundance/biomass and eDNA read counts were tested.

3. Key Contributions

• First Commercial Comparison: This is one of the few studies to directly compare multiple commercial eDNA providers against a rigorous, exhaustive field inventory in a species-rich tropical environment.
• Quantification of Error: The study provides concrete metrics on the high variability of false negative (32–92%) and false positive (20–59%) rates among different commercial providers.
• Database Impact: By re-analyzing raw data with a unified, high-quality reference database, the authors demonstrated that bioinformatic processing and database completeness are primary drivers of inter-company variability, often more so than field sampling differences.
• Biomass Correlation: The study confirms that eDNA detection in streams is strongly correlated with species biomass but weakly correlated with abundance, suggesting a bias against small, low-biomass species.

4. Key Results

• High Variability in Detection:
  • The number of species detected by the four companies ranged drastically from 5 to 48.
  • The exhaustive electrofishing inventory identified 50 species.
  • False Negatives: The rate of undetected species varied from 32% to 92% depending on the company. Even after re-analysis with a standard pipeline, false negatives remained between 30% and 48%.
  • False Positives: Companies reported 10 species never before recorded in the Maroni River catchment. False positive rates ranged from 20% to 59%.
• Taxonomic Resolution Issues:
  • Only Company A provided species-level identification for nearly all taxa (94%).
  • Companies B, C, and D failed to identify a significant portion of taxa to the species level (identifying only 24–61% to species), with many assignments stopping at genus, family, or even class.
• Impact of Re-analysis:
  • Re-processing raw data with a curated database significantly reduced discrepancies. The number of species detected by all four companies increased from 4 to 34.
  • The number of "impossible" detections (species not found in the region) dropped from 10 to 2.
• Biomass vs. Abundance:
  • Species detected by eDNA had significantly higher biomass than those undetected ($p < 0.005$).
  • There was no significant correlation between eDNA read counts and species abundance (number of individuals), but a strong positive correlation with biomass ($R^2$ 0.31–0.59).
• Community Dissimilarity:
  • Principal Coordinate Analysis (PCoA) showed strong dissimilarity between the eDNA-derived communities and the actual local fish community, regardless of the company or the index used (presence/absence or abundance).

5. Significance and Conclusions

• Challenge to Regulatory Use: The high rates of false negatives and positives, combined with the lack of standardization, challenge the immediate use of commercial eDNA metabarcoding for regulatory fish-based ecological assessments (e.g., Water Framework Directive) in species-rich tropical streams.
• Bias Against Rare Species: The method appears ineffective at detecting rare species or those with low biomass, which are often critical for conservation and biodiversity indices.
• Need for Transparency and Standards: The authors argue that the variability in results is largely driven by bioinformatic choices and reference database quality rather than field sampling. They recommend:
  • Mandatory transparency from companies regarding protocols, databases, and raw data access.
  • The development of standardized protocols including biological and technical replicates.
  • The use of curated, taxonomically verified reference databases.
• Expert Oversight: Commercial eDNA inventories should not be used in isolation. They require supervision by taxonomic experts to validate results, especially when detecting species outside known distribution ranges.

Final Verdict: While eDNA metabarcoding holds potential, the current "turnkey" commercial offerings are too variable and error-prone to replace traditional methods for precise local biodiversity assessment in complex, species-rich ecosystems without significant methodological harmonization and expert validation.