🧬 biology

LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling

LDARNet is a 120M-parameter hierarchical genomic foundation model that introduces unsupervised, learnable tokenization via dynamic chunking and learned routing, achieving state-of-the-art performance on histone modification tasks and outperforming models up to 20 times larger by aligning adaptive sequence boundaries with biologically relevant structures like promoter motifs and splice junctions.

Original authors: Daria Ledneva, Denis Kuznetsov

Published 2026-06-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Daria Ledneva, Denis Kuznetsov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to teach a computer to understand the language of life: DNA. DNA is a long string of letters (A, C, G, T) that tells cells how to build and run a living organism.

For a long time, computers have tried to read this "language" by chopping the DNA into fixed-size chunks, like cutting a long sentence into groups of exactly three letters every time, regardless of what the words actually mean. This is like trying to read a book by cutting every page into strips of 10 inches, even if a sentence ends in the middle of a strip. It works, but it's messy and misses the natural structure of the story.

Enter LDARNet: The Smart Reader

The paper introduces a new AI model called LDARNet. Instead of using a rigid ruler to cut the DNA, LDARNet learns to find the natural places to break the text, just like a human reader naturally pauses at the end of a sentence or a paragraph.

Here is how it works, using some simple analogies:

1. The "Smart Highlighter" (Learnable Tokenization)

Most DNA models use a "fixed grid." Imagine a conveyor belt where a machine chops a long loaf of bread into slices that are always 2 inches thick. Sometimes a slice cuts right through a delicious filling (a biological signal), and sometimes it leaves a whole filling stuck to the crust.

LDARNet is different. It has a smart highlighter that scans the DNA and decides, "Okay, this group of letters forms a meaningful unit, so I'll stop here. This next part is different, so I'll start a new chunk." It learns where the natural "boundaries" are, rather than forcing them.

2. The "Two-Way Street" (Bidirectional Reading)

In the human body, DNA is read from both directions (forward and backward) to understand how genes work. Older models often read like a one-way street, only looking ahead. LDARNet is built like a two-way street. It looks at the context from both the left and the right simultaneously. This helps it understand the full picture of a gene, not just what comes next.

3. The "Compression Trick" (Efficiency)

DNA is incredibly long. Reading every single letter is slow and expensive for a computer.

Old way: Read every letter, one by one.
LDARNet way: It uses its "smart highlighter" to group letters into chunks. It then processes these chunks as single units.

Think of it like reading a summary of a book instead of reading every single word. LDARNet compresses the information so it can process it much faster, but because it learned where to compress, it doesn't lose the important details.

What Did They Find?

The researchers tested LDARNet against other top models, including some that are 20 times larger (with much more memory and computing power).

The Underdog Wins: Even though LDARNet is small (only 120 million "parameters," or brain cells), it beat the giant models on many tasks. It won 11 out of 18 major challenges in the "Nucleotide Transformer" competition.
The Histone Special: It was especially good at predicting "histone modifications." Think of histones as the spools that DNA wraps around. When the spool changes shape, it turns genes on or off. LDARNet was the best at predicting these changes, outperforming models that were 20 times bigger.
The "Why" Experiment: To prove it wasn't just luck, they ran a controlled test. They took a version of the model and forced it to use the old "fixed ruler" method (cutting every 4 letters).
- Result: The model with the learned boundaries (the smart highlighter) was significantly better at finding biological signals than the one with the fixed ruler. The paper claims that up to 14 percentage points of the improvement came simply from learning where to cut the text, not from having more computing power.

The Biological "Aha!" Moment

The most exciting part is that the computer didn't just guess randomly. When the researchers looked at where LDARNet decided to make its cuts, they found it was landing exactly on real biological landmarks.

It stopped cutting right at promoters (the "start" buttons for genes).
It stopped cutting right at splice junctions (where the cell edits the genetic message).

The model figured out the biological rules of the game without anyone explicitly teaching it those rules. It learned to see the "words" of life on its own.

Summary

LDARNet is a small, efficient AI that reads DNA by learning to find its own natural sentence breaks, rather than following a rigid, pre-set pattern. By doing this, it understands the biological story better than much larger, more expensive models, proving that knowing how to read is more important than just having a bigger brain.

Technical Summary: LDARNet

Problem Statement

Genomic foundation models increasingly adopt Large Language Model (LLM) architectures but predominantly rely on fixed tokenization schemes (e.g., k-mers, Byte-Pair Encoding, or single nucleotides). These fixed schemes impose arbitrary sequence boundaries that may obscure biologically relevant structures, as they lack clear biological grounding. While recent work like H-Net demonstrated that adaptive, dynamic tokenization is feasible for autoregressive (AR) generation, it remains unclear whether such adaptive tokenization yields competitive embeddings under the Masked Language Modeling (MLM) paradigm, which is standard for genomic foundation models. Furthermore, existing adaptive approaches often fail to validate learned boundaries against canonical genomic landmarks or demonstrate causal links between learned routing and performance gains.

Methodology

The authors present LDARNet (Learnable DNA Adaptive Representation Network), a 120M-parameter hierarchical genomic foundation model that adapts the H-Net architecture from AR generation to MLM pretraining.

Architecture

LDARNet retains the encoder–backbone–decoder hierarchy of H-Net but introduces four key architectural modifications to suit bidirectional genomic modeling:

BiMamba-2 Backbone: Causal Mamba layers are replaced with BiMamba-2 blocks (extending the BiMamba design of Caduceus to Mamba-2). These blocks utilize shared weights for forward and reverse passes to enable full-context conditioning without doubling parameters.
Non-Causal Attention: All attention mechanisms are non-causal to support bidirectional context.
Local Attention: The encoder includes a single local attention layer to aid fine-grained pattern recognition, while the backbone and decoder rely entirely on BiMamba-2.
Bidirectional Routing and Dechunking:
- Bidirectional Routing: Unlike H-Net's unidirectional router, LDARNet estimates boundary evidence using both forward and backward orientations of adjacent pairs. This employs a bidirectional similarity metric to detect boundaries symmetrically, making the process robust to the 15% masking used in MLM.
- Bidirectional Dechunking: The dechunker incorporates a bidirectional Exponential Moving Average (EMA) smoother. This propagates information from both directions into masked positions, ensuring stable reconstruction of variable-length spans.

Training Objective

The model is optimized using an MLM loss combined with a ratio-based regularizer ( $L_{ratio}$ ). This regularizer prevents degenerate compression solutions (e.g., trivial compression) by enforcing a target compression ratio ( $N=4$ ) across stages. The training corpus combines the human reference genome with multispecies data from the Nucleotide Transformer, utilizing reverse-complement augmentation to enforce biological symmetry.

Key Contributions

The paper outlines four primary contributions:

Architectural Adaptation: Successfully adapting the H-Net dynamic tokenization framework from autoregressive generation to masked language modeling.
Performance Efficiency: Demonstrating that a compact 120M-parameter model with dynamic boundary prediction can match or surpass baselines 4–20 times larger (up to 2.5B parameters), achieving state-of-the-art results on 5 histone modification tasks.
Cross-Species Generalization: Establishing that architectural generality through learnable multi-scale compression preserves cross-species transferability better than domain-specific optimizations. LDARNet achieves 11/18 wins on the Nucleotide Transformer benchmark, outperforming human-genome-specialized models of comparable scale.
Causal Evidence for Learned Boundaries: Providing direct evidence that learned routing (not just hierarchical scaffolding) drives performance gains. A FLOPs-matched controlled experiment shows that learned boundaries outperform fixed-grid boundaries by up to 14.3 percentage points on histone tasks. Furthermore, nucleotide-resolution analysis reveals that learned boundaries align with canonical promoter motifs and splice junctions without supervision.

Results

Nucleotide Transformer (NT) Benchmark

On 18 diverse tasks, LDARNet achieved 11 wins among compact models (<300M parameters), significantly outperforming other compact models (which achieved 1–2 wins each). Notably, it secured the best overall results on 5 histone modification tasks (H3K4me1, H3K4me2, H3K4me3, H3K79me3, H4ac), outperforming competitors up to 2.5B parameters.

Genomic Benchmarks (GB)

On 9 regulatory genomics tasks, LDARNet tied with DNABERT-2 for the best compact-model performance (3 wins each). While GB tasks exhibit a ceiling effect due to high baseline accuracies, LDARNet achieved the highest accuracy (96.3%) on the "Human non-TATA Promoters" task, surpassing larger models like Generator-1.2B.

Controlled Experiments

A FLOPs-matched experiment comparing LDARNet (learned routing) against a variant with fixed boundaries every 4 nucleotides ( $N=4$ ) isolated the source of gains:

Histone Tasks: LDARNet outperformed the fixed-boundary variant by +14.3pp on H3K4me1 and +6.7pp on H3K36me3. The fixed-boundary model also exhibited significantly higher variance on these tasks.
Splice Sites: The trend reversed for splice-site classification, where fixed boundaries (0.961) outperformed learned routing (0.901). The authors attribute this to the local nature of splice motifs (GT/AG), where fixed grids preserve uniform positional information better than learned routing, which is not explicitly rewarded for landing on every junction.

Significance

The paper claims that LDARNet provides a biological interpretation for adaptive tokenization in genomic foundation models. By learning boundaries that align with functional genomic elements (promoters, splice junctions) without supervision, the model demonstrates that adaptive tokenization can capture functional signals more faithfully than fixed schemes. The work highlights that learned routing is the critical factor enabling compact models to rival much larger architectures, particularly for long-range epigenetic modeling tasks like histone modification prediction.