PathogenSurveillance: an automated pipeline for population genomic analyses and pathogen identification

The paper introduces PathogenSurveillance, an open-source, automated Nextflow pipeline designed to streamline whole-genome sequencing-based population genomic analyses and pathogen identification for real-time biosurveillance across diverse organisms and sequencing technologies.

The Gist

Imagine you are a detective trying to solve a mystery in a chaotic crime scene. The "criminals" are invisible pathogens (like bacteria or fungi) and pests that are attacking crops, animals, or people. Usually, to catch them, you need to know exactly what you are looking for before you start, like having a specific "Wanted" poster. But what if the criminal is a brand-new shape-shifter that no one has ever seen before? Traditional methods often fail because they are too rigid.

Enter PathogenSurveillance, a new digital tool created by a team of scientists. Think of it as an automated, super-smart detective robot that can walk into that chaotic crime scene, figure out who the bad guys are, and tell you exactly how they are related to other known criminals—all without needing a human expert to hold its hand.

Here is how it works, broken down into simple concepts:

1. The "Organism-Agnostic" Detective

Most diagnostic tools are like a key that only fits one specific lock. If the lock changes shape, the tool fails. PathogenSurveillance is different. It is organism-agnostic, meaning it doesn't care if the intruder is a bacteria, a fungus, or something else entirely. It looks at the "DNA fingerprint" (the whole genome) of whatever it finds and says, "Ah, I see what you are," even if it's a completely new species.

2. The Automated Library Search

When the robot finds a sample, it doesn't just guess. It immediately runs to the world's biggest digital library (the NCBI database) to find matching "Wanted Posters" (reference genomes).

• The Magic Trick: It uses a clever system to pick the best posters to compare against. It's like a librarian who doesn't just grab the first book they see, but carefully selects the top 10 most relevant books to help you solve the case, saving you time and confusion.

3. The "Mix-and-Match" Capability

Sometimes, a sample is a messy smoothie containing bits of many different organisms (a mix of bacteria and fungi). PathogenSurveillance is like a master chef who can separate that smoothie back into its individual fruits. It can handle mixed samples, separating the prokaryotes (simple cells like bacteria) from the eukaryotes (complex cells like fungi or plants) and analyzing them separately at the same time.

4. The Family Tree Generator

Once the robot identifies the culprit, it wants to know: "Is this a lone wolf, or part of a gang?"

• It builds a family tree (phylogenetic tree) to show how closely related the new sample is to known strains.
• It draws a map of connections (minimum spanning network) to see if these bugs are spreading from one farm to another or jumping from animals to humans.
• This helps scientists answer: "Is this a new, dangerous mutant, or just a common bug we've seen a thousand times?"

5. The "One-Click" Report

The best part? You don't need to be a computer wizard to use it.

• The Old Way: You had to install 20 different programs, write complex code, and hope you didn't make a typo.
• The PathogenSurveillance Way: You give it a simple list of files, type one command, and it does everything. It assembles the puzzle, checks the quality of the pieces, and prints a beautiful, interactive report with charts and graphs that anyone can understand.

Why Does This Matter?

Imagine a new disease starts spreading in a wheat field. In the past, it might take weeks to figure out what it is, by which time the disease has already destroyed the harvest. With PathogenSurveillance, we can identify the threat in hours.

It acts as an early warning system for our food supply, our animals, and our health. By automating the hard work, it allows scientists and even field workers to react quickly to new threats, stopping epidemics before they become pandemics.

In short: PathogenSurveillance is the Swiss Army Knife of disease detection. It takes the complex, scary world of genetic code and turns it into a clear, actionable story, helping us stay one step ahead of nature's invisible invaders.

Technical Summary

1. Problem Statement

Emerging invasive pathogens and pests pose a global threat, often outpacing existing biosurveillance systems. While Whole Genome Sequencing (WGS) offers a comprehensive, organism-agnostic method for detecting these threats, its adoption is hindered by significant barriers:

• Computational Complexity: Traditional WGS analysis requires advanced expertise in bioinformatics, molecular evolution, and specific tool selection.
• Reference Selection: Accurate pathogen identification relies heavily on selecting appropriate reference genomes, a task that is difficult when the taxonomic identity of a sample is unknown (common with emerging threats).
• Workflow Fragmentation: Existing pipelines often lack flexibility (e.g., handling mixed prokaryotic/eukaryotic samples), are limited to specific taxonomic groups, or require manual intervention for steps like phylogeny construction and variant calling.
• Accessibility: Many tools are web-based (limiting privacy and local control) or require complex local installation and dependency management.

2. Methodology

The authors developed PathogenSurveillance, an open-source, automated Nextflow pipeline designed for population genomic analysis and pathogen identification.

• Architecture & Portability:

  • Built on Nextflow and integrated into the nf-core framework, ensuring reproducibility, scalability, and portability across any Linux-based system (local desktops, HPC clusters, clouds).
  • Uses containerization (Docker, Apptainer, Conda) to manage dependencies automatically.
  • Supports both short-read (Illumina) and long-read (PacBio, Oxford Nanopore) data, though it does not support hybrid assembly of the same sample from multiple platforms.

• Automated Workflow Steps:

  1. Input: Accepts a simple TSV file containing sample paths, URLs, or NCBI accession numbers.
  2. Initial Taxonomic Profiling: Generates k-mer sketches from raw reads using bbmap sendsketch and compares them against NCBI RefSeq to predict taxonomy.
  3. Automated Reference Selection:
     • Downloads metadata for relevant assemblies based on predicted taxonomy.
     • Applies a rule-based system to filter references based on assembly quality (e.g., type strain status, RefSeq inclusion, contig L50).
     • Uses Average Nucleotide Identity (ANI) to refine selection, distinguishing between contextual references (for phylogeny) and mapping references (for SNP calling).
     • Implements an iterative binning algorithm to select a minimal set of references that cover the diversity of the sample group.
  4. Dual-Branch Processing:
     • Prokaryotes: Annotates genomes, identifies ortholog clusters, constructs core-genome phylogenies, and performs SNP calling/mapping using a pangenome graph approach for closely related samples.
     • Eukaryotes: Skips automated annotation (due to taxonomic complexity) and instead constructs phylogenies based on Benchmarking Universal Single-Copy Orthologs (BUSCO).
  5. Variant Calling: For samples mapped to the same reference, it calls SNPs and generates Minimum Spanning Networks (MSNs) to visualize clonal lineages.

• Output & Reporting:

  • Generates an interactive HTML report containing sunburst plots (taxonomy), heatmaps (ANI/POCP), phylogenetic trees, and MSNs.
  • Organizes all intermediate files (assemblies, annotations, BAM/VCF files) in a hierarchical, human-readable, and machine-parseable directory structure.
  • Includes comprehensive Quality Control (QC) reports (FastQC, NanoPlot, Quast, BUSCO) compiled via MultiQC.

• Optimization Features:

  • Caching: Allows pipeline resumption (-resume) and rapid re-analysis with modified parameters without re-running the entire workflow.
  • Error Recovery: Automatically retries failed processes with increased resource allocation.

3. Key Contributions

• Automation of Reference Selection: The pipeline introduces a novel, rule-based system that automatically identifies, retrieves, and optimizes reference genomes based on sample data, removing the need for manual curation.
• Hybrid Taxonomic Support: It is one of the few pipelines capable of simultaneously analyzing mixed datasets containing both prokaryotic and eukaryotic organisms within a single run.
• Multi-Resolution Phylogenetics: Integrates k-mer based identification, multigene phylogenies (core genome/BUSCO), and variant-based (SNP) analyses to provide identification at species, subspecies, and clonal levels.
• User Accessibility: Designed to lower the barrier to entry for non-bioinformaticians, requiring only a single command to execute complex population genomic analyses.
• Open Source & Community Driven: Released under the MIT license via nf-core, ensuring community peer review and continuous improvement.

4. Results & Validation

The pipeline was validated using several datasets:

• Serratia Dataset (302 samples): Compared against a previously published study, PathogenSurveillance produced phylogenetic trees with topology concordant to the core gene phylogeny, accurately assigning species and lineages with only minor discrepancies in clonal placements.
• Honeybee Microbiota (Gilliamella apicola):
  • Successfully identified G. apicola strains from Nanopore data without prior knowledge of the specific strain.
  • Demonstrated robust reference selection, choosing the most similar strain for mapping based on ANI rather than just metadata.
  • Simulation Test: Even when introducing up to 10% sequence divergence, the pipeline consistently selected appropriate references, showing high robustness.
• Performance Benchmarks:
  • Scalability: Runtime and RAM usage increased linearly with sample size (e.g., 200 samples took ~11.7 hours and 12.2 GB RAM).
  • Genome Size Impact: Prokaryotic analyses were fast (<1 hour) regardless of genome size. Eukaryotic analyses scaled linearly with genome size (up to 13.6 hours for a 224.8 Mb genome).
  • Resource Efficiency: The pipeline effectively managed memory and utilized parallelization on a standard 16-core desktop.

5. Significance

PathogenSurveillance represents a significant advancement in biosurveillance capabilities:

• Rapid Response: It enables the rapid detection and monitoring of novel pathogens and pests, crucial for preventing epidemics and managing antimicrobial resistance.
• One Health Integration: By supporting both prokaryotes and eukaryotes, it aligns with "One Health" frameworks that track pathogens across human, animal, and environmental domains.
• Democratization of Genomics: By automating complex steps like reference selection and phylogeny construction, it empowers field researchers and diagnosticians without deep bioinformatics training to perform high-resolution genomic surveillance.
• Future-Proofing: The pipeline's modular design and caching capabilities allow for iterative refinement of analyses as new data or references become available, making it a sustainable tool for long-term surveillance programs.

Limitations Noted: The pipeline currently does not support viruses (requiring specialized metagenomic workflows), relies on the accuracy of public taxonomic databases, and is optimized for cultured organisms rather than complex metagenomes (though it showed promise with low-complexity metagenomes). It also requires significant storage for intermediate files (approx. 5x input size), though caching mitigates redundancy.