Variable performance of widely used bisulfite sequencing methods and read mapping software for DNA methylation

This study evaluates the performance of bisulfite sequencing methods (RRBS and WGBS) and mapping software in genetically diverse stickleback populations, revealing significant variations in methylation detection and tool accuracy to provide methodological recommendations for improving the reliability of DNA methylation profiles in non-model organisms.

The Gist

Imagine your DNA is a massive, ancient library containing the instruction manual for building and running a living organism. DNA methylation is like a system of sticky notes or highlighters that the cell uses to mark certain pages. Some pages get highlighted to say, "Read this!" (active genes), while others get marked with a "Do Not Disturb" sign to say, "Ignore this" (silenced genes).

Scientists want to read these sticky notes to understand how animals adapt to their environments, how they evolve, and how they react to stress. To do this, they use a technique called Bisulfite Sequencing. Think of this as a chemical process that turns all the "unmarked" pages into a different color, so when you scan the library, you can easily see which pages still have their original "Do Not Disturb" signs (methylated) and which ones have been turned into the new color (unmethylated).

This paper is essentially a consumer report for scientists who are trying to read these sticky notes in wild animals (like fish, birds, and insects) rather than in controlled lab mice. The authors, Emily Kerns and Jesse Weber, tested two main ways to scan the library and four different types of "scanning software" to see which combination gives the most accurate picture.

Here is the breakdown of their findings using simple analogies:

1. The Two Scanning Methods: The "Wide-Angle" vs. The "Zoom Lens"

The researchers compared two ways to scan the DNA library:

• WGBS (Whole Genome Bisulfite Sequencing): This is like taking a wide-angle photo of the entire library. You see every single book, every shelf, and every corner.
  • Pros: You get the whole picture. You find sticky notes in the back rooms (introns) and the dusty basements (intergenic regions).
  • Cons: Because you are trying to photograph the whole library, the photo is a bit blurry. You don't get a high enough resolution to read the text clearly on every single page unless you spend a fortune on a massive camera.
• RRBS (Reduced Representation Bisulfite Sequencing): This is like using a powerful zoom lens that only focuses on the most important sections of the library (the "CpG islands," which are like the main reading rooms and the front covers of books).
  • Pros: You get a super-sharp, high-definition view of the most important pages. You can take many more photos (samples) for the same price.
  • Cons: You miss the back rooms and the basements. You only see a small fraction of the library.

The Finding: If you are a scientist studying wild animals and want to know how they are reacting to their environment right now, the Zoom Lens (RRBS) is usually better. It gives you a clearer, more detailed look at the functional parts of the genome (promoters and exons) where the action happens. The Wide-Angle lens (WGBS) is great for finding where in the genome things are, but it's often too expensive and blurry to give you precise details on specific animals in the wild.

2. The Scanning Software: The "Old Map" vs. The "New GPS"

Once you have the photos (data), you need software to stitch them together and tell you which pages are highlighted. The authors tested four different software programs: Bismark (the old, popular standard), BWA meth, BiSulfite Bolt, and Biscuit (the newer, faster tools).

• The Old Map (Bismark): This has been the most popular tool for years. It's like using a paper map from 1990. It works, but it's slow, and it often gets lost. The authors found that Bismark missed a huge chunk of the data (low "mapping efficiency"). It was like trying to navigate a city with a map that only shows 50% of the streets.
• The New GPS (BWA meth, BiSulfite Bolt, Biscuit): These are like modern GPS apps. They are faster and, crucially, they found more of the data. They successfully mapped almost all the reads to the reference genome.

The Big Surprise:
While the new GPS tools were better at finding the data, they seemed to misinterpret the sticky notes.

• The Old Map (Bismark) and the slightly older BWA meth showed a realistic picture: most pages were either clearly "Do Not Disturb" (methylated) or clearly "Open for Business" (unmethylated).
• The New GPS tools (Biscuit, BiSulfite Bolt) seemed to get confused. They reported that almost every page had a "half-methylated" sticky note. They overestimated how many pages were in a "maybe" state.

The Analogy: Imagine you are counting red and blue marbles in a jar.

• Bismark/BWA meth say: "There are mostly red marbles and mostly blue marbles."
• Biscuit/BiSulfite Bolt say: "Actually, almost every marble is a weird purple color!"
  The authors suggest that for wild animals, the "Old Map" (or the slightly older BWA meth) might actually be giving a more honest, realistic picture of the biology, even if it misses a few marbles.

3. The "Wild" Factor: Why This Matters for Nature

Most of these software tools were built and tested on lab mice or humans, which are like clones of each other genetically. They are very predictable.

But wild animals (like the stickleback fish the authors studied) are genetically diverse. They are all different individuals.

• When you scan a wild animal, the software has to deal with "typos" in the DNA (genetic variations) that look like sticky notes.
• The authors found that the newer, faster software tools struggled with this diversity, often mistaking genetic differences for methylation marks.
• They also found that the "Zoom Lens" (RRBS) was surprisingly good at filtering out the noise and focusing on the functional parts of the genome, making it a better choice for ecological studies where you care about function (how the animal survives) rather than just location (where the DNA is).

The Bottom Line for Scientists

If you are a researcher studying wild animals and want to understand how they adapt to their environment:

1. Don't just use the default software (Bismark). It's the most popular, but it's often the least efficient. It's like using a slow, old car when a sports car is available.
2. Consider the "Zoom Lens" (RRBS). If you have a limited budget and need to study many different animals, RRBS gives you a sharper, more useful picture of the important parts of the genome.
3. Be careful with the "New GPS" tools. While they are fast and find more data, they might be telling you that everything is "half-methylated" when it's actually just a mix of clear red and blue. You need to test these tools carefully before trusting them with wild data.

In short: The paper is a warning to scientists not to blindly trust the "standard" tools. Just because a method worked on a lab mouse doesn't mean it works on a wild fish. To get the truth about nature, you need to choose your tools (both the camera and the software) very carefully.

Technical Summary

1. Problem Statement

DNA methylation (DNAm) is a critical marker in ecological epigenetics, yet the performance of library preparation strategies (Reduced Representation Bisulfite Sequencing [RRBS] vs. Whole Genome Bisulfite Sequencing [WGBS]) and bioinformatic pipelines is rarely assessed in genetically variable, non-model natural populations.

• The Gap: Most methylation profiling tools were developed for inbred laboratory model organisms (e.g., humans, mice). Their performance in wild populations with high genetic diversity (SNPs) and unknown genotypes is uncertain.
• The Specific Issues:
  • Bioinformatics: Bismark is the most widely used aligner but relies on Bowtie2 (global alignment), which may have lower mapping efficiency than BWA mem-based tools (local alignment). Newer tools (Biscuit, BiSulfite Bolt) exist but lack comparative validation in wild systems.
  • Methodology: RRBS enriches for CpG islands (functional regions) but sacrifices genomic breadth, while WGBS offers breadth but lower read depth per site. The extent to which these methods yield concordant biological inferences in natural populations is unclear.

2. Methodology

The authors conducted a comprehensive comparative study using both generated data and public datasets.

• Biological System:
  • Primary Data: Threespine stickleback (Gasterosteus aculeatus) liver tissue from five ecologically divergent populations.
  • Design: Four individuals were profiled using both RRBS and WGBS (technical replicates). A total of 34 RRBS samples and 4 WGBS samples were generated.
  • Public Data: RRBS and WGBS data from five other diverse taxa (cichlid, purple sea urchin, great tit, coral, stick insect) were integrated to test pipeline robustness across species.
• Bioinformatic Pipelines:
  • Alignment: Four tools were compared: Bismark (default global and "Local" mode), BWA meth, BiSulfite Bolt, and Biscuit.
  • Methylation Calling: MethylDackel was used for BWA meth, BiSulfite Bolt, and Biscuit outputs. Bismark's native caller was used for its output.
  • Filtering: A lenient SNP filter (--maxVariantFrac 0.8) was applied to account for unknown genotypes in wild populations. Sites required a minimum depth of 5x or 10x.
• Statistical Analysis:
  • Mapping efficiency was compared using Friedman tests and Wilcoxon signed-rank tests.
  • Concordance between pipelines was assessed via linear models (slope/intercept analysis) and residual analysis against sequencing depth.
  • Genomic annotations (promoters, exons, introns, intergenic, CpG islands) were performed using genomation and TaJoCGI.

3. Key Contributions & Results

A. Read Mapping Performance

• Bismark Underperformance: Even after optimizing parameters (switching to local alignment and non-directional mode), Bismark consistently showed the lowest mapping efficiency compared to BWA mem-based tools.
  • Example: In RRBS stickleback data, Bismark mapped ~54% of reads, whereas Biscuit and BWA meth mapped >85–99%.
• Newer Tools: Biscuit and BiSulfite Bolt generally achieved the highest mapping efficiencies across all species and library types.
• Batch Effects: While Bismark showed no batch effects, other aligners showed minor but significant batch effects, suggesting sensitivity to sequencing runs.

B. Methylation Profile Discrepancies

• Old vs. New Tools: There was a significant divergence in methylation estimates between older tools (Bismark, BWA meth) and newer tools (Biscuit, BiSulfite Bolt).
  • Older Tools: Recovered a distribution dominated by unmethylated (0–10%) and fully methylated (>90%) sites, consistent with expected bimodal distributions.
  • Newer Tools: Systematically overrepresented intermediate methylation (10–90%) and underrepresented unmethylated sites.
  • Implication: Newer tools may be misclassifying unmethylated cytosines or failing to filter PCR duplicates/SNPs correctly in wild populations.
• Sequencing Depth: Agreement between pipelines increased with sequencing depth but plateaued at approximately 20 million reads per sample. Beyond this, additional sequencing provided diminishing returns for concordance.

C. RRBS vs. WGBS Comparison

• Genomic Breadth:
  • WGBS: Captured ~96% of CpG sites unique to itself, predominantly in introns and intergenic regions. It recovered nearly all CpG islands, promoters, and exons present in the genome.
  • RRBS: Captured a highly specific subset. Sites unique to RRBS were enriched in CpG islands (50%) and promoters (37%), but captured <15% of exons/introns.
• Shared Sites: Very few CpG sites were shared between RRBS and WGBS for the same individual (<1% at 10x depth; ~2.25% at 5x depth).
• Intermediate Methylation: WGBS detected nearly twice as many intermediately methylated sites (55.6%) compared to RRBS (28.6%). This suggests RRBS may miss biologically relevant "partial" methylation states or that the method inherently filters them out due to lower coverage in those specific regions.
• Population Variability: The overlap of CpG sites between biological replicates in RRBS varied significantly by population (e.g., Wik Lake vs. Watson Lake), likely due to genetic variation affecting MspI restriction enzyme cut sites. This was not observed in WGBS.

4. Significance and Recommendations

• For Bioinformatics:

  • Researchers should consider BWA meth, Biscuit, or BiSulfite Bolt over Bismark for higher mapping efficiency in wild populations.
  • However, caution is advised with newer tools (Biscuit/BiSulfite Bolt) as they appear to inflate intermediate methylation calls. Validation of these tools against known ground truths in non-model organisms is urgently needed.
  • Local alignment (soft-clipping) is superior to global alignment for bisulfite data in diverse genomes.

• For Experimental Design (RRBS vs. WGBS):

  • RRBS is recommended for studies focused on functional, regulatory regions (promoters, CpG islands) and studies requiring large sample sizes with high per-site depth. It is cost-effective for detecting relative changes in functionally relevant methylation.
  • WGBS is necessary for capturing genome-wide patterns, including intergenic and intronic regions, and for detecting intermediate methylation states that RRBS may miss.
  • Sequencing Depth: For RRBS in stickleback, 20 million reads/sample is the optimal trade-off between cost and accuracy.

• General Impact:

  • The study highlights that methodological choices (library prep and software) can drastically alter biological inferences in ecological epigenetics.
  • It provides open-source code and a benchmarking framework to help researchers optimize pipelines for genetically diverse, non-model organisms.

Conclusion

The paper demonstrates that while RRBS is efficient for functional regions, it misses a significant portion of the methylome (intermediate sites, intergenic regions) compared to WGBS. Furthermore, the choice of bioinformatic aligner significantly impacts the distribution of methylation calls, with newer tools potentially introducing bias toward intermediate methylation. The authors advocate for careful method selection based on specific research questions and the genetic diversity of the study system.