A comprehensive assessment of tandem repeat genotyping methods for Nanopore long-read genomes

This study systematically benchmarks seven actively maintained Nanopore long-read tandem repeat genotyping tools across accuracy, usability, and diverse genomic contexts, revealing that no single tool is universally superior and that sequence-level evaluation is essential for selecting the right method for population studies and clinical diagnostics.

The Gist

Imagine your genome as a massive, intricate library of instruction manuals for building a human being. Most of these manuals are written in clear, unique sentences. But scattered throughout are pages filled with repetitive phrases, like "ATCG ATCG ATCG..." repeated over and over. These are called Tandem Repeats.

Sometimes, these repeats are harmless. Other times, they go haywire. If the phrase "ATCG" gets repeated too many times (an "expansion"), it can cause serious diseases like Huntington's or Fragile X syndrome.

For years, scientists have struggled to count these repeats accurately. It's like trying to count the pages in a book where the text is smudged and the pages are stuck together. Short-read sequencing (the old method) was like trying to guess the length of a long rope by looking at tiny, blurry snippets of it. You could guess, but you often got the length wrong or missed important details, like a knot in the rope.

Enter the "Long-Read" Revolution.
New technology (Oxford Nanopore) allows us to read the entire rope in one go. This is a game-changer. But now, we have a new problem: We have many different tools (software) trying to count these ropes, but no one knows which tool is the best.

This paper is the ultimate "Consumer Reports" or "Car Test Drive" for these software tools. The authors didn't just pick one; they put seven different counting tools through a rigorous, head-to-head race to see which one is the most accurate, the fastest, and the easiest to use.

The Race Track: How They Tested the Tools

Since there is no "perfect truth" to compare against (we can't always see the exact rope length with our own eyes), the researchers used four clever strategies to judge the tools:

1. The "Gold Standard" Assembly: They compared the tools' answers against a super-detailed, high-definition map of the genome (like comparing a sketch to a 3D model).
2. The Family Tree Test (Mendelian Consistency): They checked if the tools made sense in families. If a child has a certain number of repeats, they must have inherited them from their parents. If a tool says the child has a number that couldn't possibly come from the parents, that tool made a mistake.
3. The "Crowd Consensus": They checked if the tools agreed with each other. If six tools say "50 repeats" and one says "100," the outlier is likely wrong.
4. The "Sick Patient" Test: They tested the tools on real patients known to have dangerous expansions. Did the tool spot the disease? This is the most critical test for doctors.

The Results: No Single Winner, But Some Stars Shone

The big takeaway? There is no single "best" tool. It depends entirely on what you need to do.

• The All-Rounders: Tools like LongTR and ATaRVa were like the reliable sedans. They were very accurate across the board, especially for standard-sized repeats.
• The Homopolymer Heroes: Some repeats are just one letter repeated (like "AAAAA"). These are notoriously hard to read (like trying to count identical bricks in a wall). Medaka Tandem was the champion here, seeing clearly where others stumbled.
• The Pathogenic Detectives: When it came to finding the dangerous, very long expansions that cause disease, STRdust was the most sensitive detective, finding the most cases, even though it was a bit messier with the details.
• The Speedsters: Vamos was incredibly fast and light on computer memory, making it great for analyzing thousands of people at once.
• The Strugglers: Some tools, like Straglr (which was popular in other workflows), were found to be less accurate and didn't provide enough detail about the sequence of the repeats, just the length. The authors suggest it might need to be retired from top-tier clinical use until it improves.

The "User Experience" Reality Check

The authors also highlighted a major frustration: These tools are often hard to use.
Imagine buying a high-tech car that comes with a manual written in a foreign language, missing parts, and requires you to build the engine yourself before you can drive it. That's what using these tools feels like for many scientists.

• They often crash or give confusing error messages.
• The instructions are outdated or missing.
• Installing them is a nightmare of technical dependencies.

The authors argue that for these tools to be useful in hospitals for diagnosing patients, they need to be as easy to install and use as a smartphone app.

The Bottom Line

This paper is a massive step forward. It tells us that while we have powerful new technology to read our DNA, we still need to be careful about how we interpret it.

• For researchers: Don't just trust one tool. Pick the one that fits your specific question (e.g., use Medaka for short repeats, STRdust for finding disease).
• For developers: You need to make your tools easier to use and document them better.
• For the future: We need a "perfect" tool that is fast, accurate, easy to install, and gives us the full story of the DNA, not just a guess.

In short: We have the engine (the sequencing tech), but we are still tuning the dashboard (the software) to make sure we don't get lost on the road to understanding human health.

Technical Summary

1. Problem Statement

Tandem repeats (TRs), including Short Tandem Repeats (STRs) and Variable Number Tandem Repeats (VNTRs), are critical for understanding human disease and phenotypic diversity. However, they remain among the most difficult genomic variations to measure accurately.

• Limitations of Short-Read Sequencing (SRS): While SRS can identify some expansions, it often fails to determine exact repeat lengths or sequence composition (e.g., motif interruptions) for long or complex alleles.
• Limitations of Long-Read Sequencing (LRS): Although Oxford Nanopore Technologies (ONT) LRS can span long repeats and resolve complex structures, the field lacks a systematic evaluation of the available bioinformatic tools.
• The Gap: Existing benchmarking efforts are often developer-led, focus primarily on allele length rather than full sequence, and lack real-world usability assessments. There is no consensus on which tools are best for population studies versus clinical diagnostics, particularly regarding sequence-level accuracy and performance on challenging motifs (e.g., homopolymers).

2. Methodology

The authors conducted a large-scale, independent benchmark of seven actively maintained ONT TR genotyping tools: STRkit, LongTR, Straglr, STRdust, vamos, ATaRVa, and Medaka Tandem. (TRsv was excluded due to installation failures; TRGT was excluded as it does not support ONT data).

Data Sources:

• Genomes: Public ONT whole-genome sequencing data from >100 individuals, including Genome in a Bottle (GIAB) trios and Human Pangenome Reference Consortium (HPRC) samples.
• Chemistry: Data from both R9.4.1 and R10.4.1 flow cells.
• Catalog: A curated benchmark catalog of 43,009 TR loci sampled from the TRExplorer catalog, designed to represent a wide range of motif lengths, repeat complexities, and genomic contexts.
• Pathogenic Cohort: Targeted adaptive sampling data from 27 individuals with known pathogenic TR expansions (e.g., FMR1, C9orf72, RFC1) confirmed by orthogonal methods (RP-PCR, Southern blot).

Evaluation Strategies (Four-Pronged Approach):
Since no single "ground truth" exists for genome-wide TRs, the authors used four complementary metrics:

1. Assembly Concordance: Comparing tool calls against high-quality, haplotype-resolved HPRC assemblies (hybrid ONT/HiFi).
2. Mendelian Consistency: Assessing inheritance patterns in GIAB trios (parent-offspring) to identify apparent de novo mutations (errors).
3. Cross-Tool Consistency: Measuring agreement between different genotypers on shared loci.
4. Sensitivity to Pathogenic Expansions: Evaluating the ability to detect known disease-causing alleles.

Metrics:

• Length Accuracy: Absolute deviation in base pairs.
• Sequence Accuracy: Levenshtein distance (edit distance) between the called allele sequence and the reference/assembly sequence.
• Usability: Installation ease, documentation quality, computational resource usage (RAM/CPU), and output format standardization (VCF).

3. Key Results

A. Tool Performance & Accuracy

• No Single "Best" Tool: No genotyper was universally optimal across all metrics. Performance varied significantly based on the specific use case (e.g., homopolymers vs. long expansions).
• Assembly Concordance:
  • Top Performers: LongTR, Medaka Tandem, and ATaRVa achieved >98% accuracy (within 1bp) on R10 data.
  • Chemistry Impact: All tools performed better on R10.4.1 data than R9.4.1.
  • Homopolymers: A major failure point for most tools. Medaka Tandem was the clear outlier, achieving ~89% accuracy on homopolymers (vs. ~11–79% for others), likely due to its per-sample error modeling.
  • Long Alleles: Accuracy generally declined as allele length increased (>1000 bp). LongTR performed best for very long expansions (>1000 bp).
• Sequence-Level Accuracy:
  • Length concordance often masked sequence errors. When analyzing only length-matched calls, LongTR showed the highest sequence fidelity (97.2% sequence accuracy), while ATaRVa and STRdust showed larger sequence deviations.
• Mendelian Consistency:
  • Medaka Tandem and LongTR showed the highest consistency in trios.
  • Straglr performed competitively in consistency despite being a length-only tool, but its lack of sequence output limits its utility for clinical interpretation.
  • STRdust showed the lowest consistency, particularly at compound heterozygous loci.
• Pathogenic Sensitivity:
  • STRdust demonstrated the highest sensitivity (96% with adjusted parameters) for detecting pathogenic expansions, likely because it was specifically optimized for clinical ONT data.
  • LongTR required parameter tuning (lowering quality thresholds) to achieve high sensitivity on older R9 data; in default settings, it missed all pathogenic loci in the test set.
  • C9orf72: All tools failed to detect the C9orf72 expansion due to alignment soft-clipping issues, highlighting a limitation in current aligners for extreme expansions.

B. Usability & Computational Resources

• Installation & Bugs: Significant barriers were encountered, including dependency conflicts, coordinate system mismatches (0-based vs. 1-based), and software bugs (e.g., Medaka failing on specific CRAM headers, vamos crashing on overlapping loci).
• Memory & Runtime:
  • Straglr was the most resource-intensive (mean 37.3 GB RAM, ~13.5 hours runtime).
  • Vamos was the fastest (mean 73 mins) and most memory-efficient.
  • Most other tools required 2.5–3.5 GB RAM and ~2–3 hours runtime.
• Output Formats: VCF output formats varied significantly. While most tools output full sequences in REF/ALT fields, Straglr only outputs lengths, and vamos uses non-standard formatting requiring post-processing.

4. Key Contributions

1. First Sequence-Level Benchmark: This is the first large-scale evaluation of ONT TR genotypers that prioritizes full allele sequence accuracy (interruptions, motifs) over simple length counts.
2. Multi-Metric Framework: The study establishes a robust framework for evaluating TR tools using assembly concordance, Mendelian consistency, cross-tool agreement, and clinical sensitivity.
3. Usability Audit: It provides a critical assessment of the "real-world" experience, highlighting that installation difficulties and poor documentation are major barriers to adoption.
4. Reproducible Workflows: The authors released a complete Nextflow pipeline and benchmarking catalog to support community reuse and future method development.
5. Tool Selection Guide: The paper provides a decision matrix for researchers:
   • Clinical Diagnostics: STRdust (sensitivity) or Medaka Tandem (homopolymers).
   • Population Studies: LongTR or ATaRVa (balance of speed and accuracy).
   • Sequence Analysis: LongTR (best sequence fidelity).

5. Significance

This work fundamentally shifts the paradigm for TR genotyping evaluation. It demonstrates that length accuracy alone is insufficient for clinical and population applications; sequence-level fidelity is essential for identifying pathogenic interruptions and complex structures.

The findings highlight that while LRS technology has matured, the bioinformatic ecosystem for TRs requires significant improvement in:

• Standardization: Unified VCF output formats and coordinate systems.
• Robustness: Better handling of homopolymers and extreme expansions.
• Documentation: Clearer guidance on parameter tuning for specific sequencing chemistries and disease contexts.

Ultimately, this study provides the necessary foundation for the 1000 Genomes Long-Read Sequencing Consortium and the broader genomics community to move toward accurate, scalable, and clinically actionable tandem repeat analysis.