Open-source, Hardware-Independent GPU Acceleration for Scalable Nanopore Basecalling with Slorado and Openfish

This paper introduces Openfish, an open-source GPU-accelerated decoding library, and Slorado, a fully open-source basecalling framework, to provide a hardware-independent, scalable, and accurate alternative to Oxford Nanopore Technologies' proprietary Dorado software, thereby eliminating hardware lock-in and enhancing accessibility for nanopore sequencing.

The Gist

The Big Picture: Unlocking the "Black Box"

Imagine Nanopore sequencing as a high-tech translator. It reads the electrical signals coming from DNA or RNA strands and translates them into the "language" of life (the letters A, C, T, and G). This translation process is called Basecalling.

Currently, the gold standard for this translator is a software called Dorado, made by Oxford Nanopore Technologies. However, there's a catch: Dorado comes with a "black box" inside it called Koi.

• The Problem: The Koi box is like a secret recipe locked in a vault. It only works if you have a very specific, expensive, high-end NVIDIA GPU (a powerful computer chip). If you don't have that specific chip, or if you try to use a different brand of chip (like AMD), the translator becomes incredibly slow—so slow that it's practically useless. It's like having a Ferrari engine that only runs on a specific type of premium fuel found in only one gas station.

The Solution: The authors of this paper built two new tools, Openfish and Slorado, to break open that vault. They created a fully open-source, hardware-independent translator that works just as fast as the secret recipe but runs on any modern graphics card, whether it's made by NVIDIA, AMD, or even built into a laptop.

---

The Cast of Characters

1. Dorado (The Old Guard): The current industry leader. It's fast, but only if you have the "right" expensive hardware. It relies on the closed-source Koi library to do its heavy lifting.
2. Koi (The Secret Sauce): A closed-source library that speeds up the decoding process. Without it, Dorado is like a car trying to drive through mud.
3. Openfish (The Engine): This is the new, open-source engine. It does the heavy lifting of decoding the DNA signals directly on the graphics card (GPU), just like Koi does, but it's free and works on many different types of cards.
4. Slorado (The Car): This is the complete software package that uses Openfish. It's the full vehicle you drive to get your DNA translated. It's designed to be easy to use and works on everything from massive supercomputers to small, cheap laptops.

---

The Analogy: The Assembly Line

To understand why this matters, imagine a factory assembly line that turns raw materials into a finished product.

The Old Way (Dorado with Koi):

• Step 1: A robot (the GPU) does the heavy lifting of sorting the raw materials.
• Step 2: The robot has to stop, walk over to a human worker (the CPU), hand over a massive stack of papers (data), and wait for the human to sort them.
• Step 3: The human hands the papers back.
• The Bottleneck: The time spent walking back and forth (transferring data) is so slow that the whole factory grinds to a halt. The human worker is the bottleneck.

The New Way (Slorado with Openfish):

• Step 1: The robot (GPU) does the sorting.
• Step 2: Instead of walking the papers to the human, the robot has a second, smarter arm that does the sorting right there on the assembly line.
• The Result: No walking, no waiting. The factory runs at full speed, regardless of whether the robot is made by Company A (NVIDIA) or Company B (AMD).

---

What They Actually Did (The Results)

The researchers tested their new tools against the old ones and found some amazing things:

1. Speed: Without the secret "Koi" box, the old software was painfully slow (taking days to process a human genome). With Openfish, the speed jumped up to match the secret box. It went from "impractical" to "lightning fast."
2. Accuracy: They worried that making it open-source might make it less accurate. It didn't. The results were identical to the expensive, proprietary software.
3. Hardware Freedom: This is the biggest win.
   • AMD Chips: They ran the software on AMD chips (which Dorado doesn't support at all) and it worked perfectly.
   • Cheap Laptops: They ran it on a consumer gaming laptop and even a small, low-power device (like a Raspberry Pi but more powerful).
   • Real-Time: They proved it could keep up with a live DNA sequencing machine. As the machine spits out data, Slorado translates it instantly, allowing scientists to see results while the experiment is happening, not days later.

Why This Matters to Everyone

Think of this as democratizing the future of biology.

• For Universities: They can use their existing, cheaper supercomputers (which might have AMD chips) instead of buying millions of dollars worth of new NVIDIA hardware.
• For Remote Clinics: Imagine a portable DNA sequencer in a remote village. With Slorado, that device could run on a cheap, low-power laptop, allowing for instant disease diagnosis without needing an internet connection to a massive data center.
• For Innovation: Because the code is open, anyone can look at it, improve it, or adapt it. It stops a single company from holding the keys to the future of genomics.

The Bottom Line

The authors built a universal translator for DNA. They took a process that was locked behind expensive, proprietary walls and opened the gates. Now, scientists can use whatever computer hardware they have—whether it's a massive supercomputer, a gaming PC, or a portable device—to decode DNA quickly, accurately, and for free.

It's like taking a Ferrari that only runs on a specific track and turning it into a rugged, all-terrain vehicle that can go anywhere, anytime, without needing a special ticket.

Technical Summary

1. Problem Statement

Nanopore sequencing (e.g., Oxford Nanopore Technologies, ONT) generates raw electrical signals that must be converted into nucleotide sequences via a process called basecalling. While the state-of-the-art basecaller, Dorado, is open-source, its high-performance execution relies on a closed-source, proprietary library called Koi.

• Hardware Lock-in: Koi contains highly optimized, platform-specific GPU kernels (NVIDIA only) for the decoding step. Without Koi, Dorado's performance degrades drastically (by >14x), making it impractical for real-world use on non-NVIDIA hardware or even on NVIDIA hardware without the proprietary library.
• Bottleneck: The primary performance bottleneck in the open-source Dorado execution path is the CPU-GPU data transfer. The inference step runs on the GPU, but the decoding (beam search) runs on the CPU, requiring massive data transfer of neural network scores, which consumes ~50% of total runtime.
• Limitations: This dependence restricts accessibility to specific NVIDIA GPUs, hinders deployment on portable/low-power devices (e.g., embedded systems), and prevents the use of alternative hardware ecosystems like AMD or Apple Silicon for high-speed basecalling.

2. Methodology

The authors developed two interconnected components to solve these issues: Openfish and Slorado.

A. Openfish (The Core Acceleration Library)

Openfish is an open-source, GPU-accelerated decoding library that replaces the closed-source Koi library.

• GPU-Based Decoding: Instead of transferring scores to the CPU, Openfish performs the entire beam search decoding algorithm directly on the GPU. This eliminates the massive CPU-GPU data transfer bottleneck.
• Algorithmic Adaptation: The original CPU beam search (Algorithm 2) is sequential and thread-bound. Openfish re-engineers this for massive parallelism (Algorithm 4):
  • It utilizes CUDA C for NVIDIA GPUs and HIP C for AMD GPUs.
  • It employs static memory allocation to avoid expensive dynamic memory allocation on the GPU.
  • It uses GPU Shared Memory (SM) for high-speed read/write operations during beam candidate scoring, synchronizing threads at specific steps to maintain beam integrity.
  • It supports both RNN-based models (LSTM, used in FAST/HAC) and Transformer-based models (used in SUP).
• Fused Layers: Openfish includes fused implementations of neural network layers (e.g., rotary embeddings, RMSNorm) to optimize the inference step as well.

B. Slorado (The End-to-End Framework)

Slorado is a fully open-source basecalling framework that integrates Openfish.

• Input Format: Unlike Dorado, which uses the proprietary POD5 format, Slorado accepts SLOW5/BLOW5 formats (via the slow5lib), increasing compatibility with open tools.
• Architecture: It performs inference on the GPU via LibTorch and immediately invokes Openfish for decoding within the same GPU context.
• Hardware Agnosticism: It is designed to run on heterogeneous environments, from consumer-grade desktop GPUs to datacenter clusters and embedded edge devices.

3. Key Contributions

1. Openfish Library: The first open-source, high-performance GPU decoding library for nanopore basecalling that supports both NVIDIA and AMD architectures, achieving performance competitive with the proprietary Koi library.
2. Slorado Framework: A fully open-source, end-to-end basecalling pipeline that removes the hardware lock-in of ONT's Dorado.
3. Hardware Independence: Demonstrated successful high-speed basecalling on AMD Instinct™ GPUs (MI250X, MI300X), which are currently unsupported by Dorado.
4. Scalability: Proven ability to scale from single consumer GPUs (e.g., AMD RX 7900 XTX) to massive HPC clusters (e.g., 33 parallel jobs on 192 AMD GPUs).
5. Live Basecalling: Demonstrated the feasibility of real-time basecalling on low-cost, consumer-grade hardware, keeping pace with the sequencing rate of a PromethION flow cell.

4. Key Results

Performance & Speed

• Removing Koi: Without Koi, Dorado on an NVIDIA A100 took ~2 hours for a small dataset (SUP model). With Openfish, the time dropped to ~7 minutes, comparable to Koi (only ~6% slower on average).
• AMD Support: Slorado successfully ran on AMD MI250X and MI300X GPUs. On a single node with 8x AMD MI300X, it achieved 2,196 gigabases/day (FAST model), a capability entirely absent in Dorado.
• Scaling: Slorado showed near-linear scaling with GPU count. Using 8 AMD MI300X GPUs provided a 6.4x speedup over a single GPU for the FAST model.
• Multi-Node: On the Pawsey supercomputer, Slorado processed a 16M read dataset in 1.2 hours using 33 parallel jobs, a 12.5x improvement over single-node processing.

Accuracy

• Equivalence: Slorado produced basecalled reads with equivalent accuracy to Dorado.
• Metrics: Median identity scores were nearly identical (e.g., SUP model: Slorado 0.988923 vs. Dorado 0.988889). Minor variations were attributed to floating-point precision differences, not algorithmic flaws.
• RNA Support: The framework successfully extended to RNA basecalling with the same accuracy profile as DNA.

Hardware Versatility

• Embedded/Edge: Successfully ran FAST basecalling on an NVIDIA Jetson AGX Xavier and an AMD 760M integrated GPU, devices where Dorado either fails or is unsupported.
• Live Streaming: A single AMD RX 7900 XTX (consumer grade) kept up with a 2-day PromethION run in real-time.

5. Significance

• Democratization of Genomics: By removing the dependency on proprietary NVIDIA-only libraries, Slorado and Openfish allow researchers to utilize diverse, often cheaper, or more accessible hardware (including AMD and embedded systems) for high-performance genomics.
• Portability & Reproducibility: The open-source nature ensures that basecalling pipelines are reproducible across different computing environments, from local laptops to national supercomputers.
• Future-Proofing: The framework is designed to adapt to evolving neural network architectures (RNNs to Transformers) and can potentially be extended to other accelerators like FPGAs or Intel GPUs.
• Decentralized Sequencing: The ability to run live basecalling on low-power, portable hardware supports the vision of decentralized, point-of-care nanopore sequencing applications.

In conclusion, this work successfully breaks the hardware monopoly of ONT's Dorado by providing a robust, open-source alternative that matches the speed and accuracy of the proprietary solution while vastly expanding the range of supported hardware.