PlantCAD2: a DNA foundation model for interpreting genomes across flowering plants

PlantCAD2 is a parameter-efficient, plant-specific DNA foundation model pretrained on 65 angiosperm genomes that, despite having fewer parameters than larger generalist models, outperforms them in capturing evolutionary conservation and predicting genomic functions across diverse flowering plant species.

The Gist

Imagine you have a library containing the instruction manuals for over 100,000 different types of plants. These manuals are written in a secret code made of just four letters: A, C, G, and T (the DNA alphabet). For decades, scientists have been trying to read these manuals to understand how plants grow, fight disease, and produce food. But the manuals are huge, messy, and written in thousands of different dialects.

Enter PlantCAD2. Think of it as a super-smart, plant-loving "translator" or "detective" that has read thousands of these manuals and learned the underlying rules of the language.

Here is a simple breakdown of what this paper is about, using everyday analogies:

1. The Problem: Too Many Manuals, Too Little Time

Flowering plants (like roses, corn, and oak trees) are the most diverse group of life on land. They have massive, complex instruction books.

• The Challenge: Scientists used to have to hire a new translator for every single plant species. If they wanted to understand a new type of corn, they had to start from scratch.
• The Old Way: Previous AI models were like students who only read a few short sentences (512 letters long) or only read books about a specific family of plants. They missed the big picture and couldn't connect distant parts of the manual.

2. The Solution: PlantCAD2 (The "Super-Reader")

The researchers built PlantCAD2, a new AI model designed specifically for plants. Here's what makes it special:

• It Reads the Whole Page, Not Just a Word:
  Imagine trying to understand a joke by reading only the punchline. You'd miss the setup! Previous models could only read short snippets of DNA. PlantCAD2 has a "long memory" (an 8,192-letter window). It can read a whole paragraph of the DNA manual at once, allowing it to see how a gene far away on the page controls a trait nearby.
• It Learned from a Diverse Crowd:
  Instead of just reading about corn or rice, PlantCAD2 was trained on 65 different flowering plants from all over the evolutionary tree. It's like a student who studied biology in the rainforest, the desert, and the tundra. Because it saw so many different "dialects," it learned the universal rules of plant language, not just the slang of one species.
• It's Efficient:
  There are other massive AI models (like "Evo2") that read every living thing on Earth, from bacteria to humans. But they are so huge and heavy that they are slow and expensive to run. PlantCAD2 is like a sports car: it's smaller and lighter than the giant trucks, but because it's built specifically for the track (plants), it actually drives faster and better on plant DNA than the giant trucks do.

3. What Can It Do? (The Magic Tricks)

The paper shows PlantCAD2 performing amazing "zero-shot" tricks. This means it can guess the answer to a question it has never seen before, just by using what it learned during training.

• The Conservation Detective:
  If you show PlantCAD2 a piece of DNA from a plant it has never seen (like a specific type of tomato), it can instantly tell you which parts of the code are "important" and which are "junk." It does this better than models that are 10 times bigger.
  • Analogy: It's like looking at a sentence in a foreign language and instantly knowing which words are the most critical to the meaning, even if you've never heard that specific sentence before.
• The Junction Finder:
  DNA has "start" and "stop" signs (like traffic lights) that tell the cell when to begin building a protein. PlantCAD2 is incredibly good at finding these signs, even in complex DNA structures where other models get confused.
• The Future Predictor:
  When scientists "fine-tune" PlantCAD2 (give it a little extra homework on a specific task), it becomes a master at predicting:
  • Gene Expression: Will this gene turn on or off in a leaf?
  • Protein Production: How much protein will be made?
  • Chromatin Access: Is the DNA open and ready to be read, or is it tightly packed away?
  • Analogy: It's like giving a chef a new recipe book. After a little practice, the chef can predict exactly how a new dish will taste, even if they've never cooked that specific dish before.

4. Why Does This Matter?

This isn't just about computer science; it's about the future of food and nature.

• Breeding Better Crops: Farmers need crops that can survive droughts or pests. PlantCAD2 can help scientists scan the DNA of wild plants to find the "secret ingredients" for survival and transfer that knowledge to our food crops.
• Saving Time: Instead of running expensive lab experiments to test every single gene, scientists can use PlantCAD2 to simulate the results first. It's like using a flight simulator before building a real plane.
• Unlocking the Unknown: There are thousands of plant species we know very little about. PlantCAD2 gives us a "Rosetta Stone" to finally read their instruction manuals and understand how they work.

The Bottom Line

PlantCAD2 is a specialized, high-speed AI that learned the universal language of flowering plants. It is faster, smarter, and more efficient at understanding plant DNA than previous giants, proving that you don't need to be the biggest model to be the best—you just need to be the right one for the job. It's a powerful new tool that could help us feed the world and protect our planet.

Technical Summary

1. Problem Statement

The fundamental challenge in plant genomics is deciphering how DNA sequences encode biological functions, phenotypes, and fitness. While large-scale sequencing initiatives (e.g., Earth BioGenome Project) are generating vast amounts of genomic data, functional annotations lag significantly behind, particularly for non-model flowering plants (angiosperms).

• Unique Challenges: Angiosperms exhibit extreme species diversity (>300,000 species), massive variation in genome size (~200-fold), and high rates of transposable element (TE) turnover, which obscure orthologous relationships.
• Limitations of Existing Models:
  • Generalist Models: Models like Evo2 (40B parameters) are trained across the entire tree of life. While powerful, they are computationally prohibitive for widespread plant genomics use and may dilute lineage-specific regulatory signals.
  • Specialist Models: Previous plant-specific models (e.g., PlantCAD, AgroNT) suffer from limited context windows (512 bp), preventing the modeling of distal regulatory interactions (enhancers/promoters) that often span kilobases. AgroNT also sacrifices single-nucleotide resolution via k-mer tokenization.
  • Data Scarcity: There is a lack of labeled data for most plant species, necessitating models that can learn from raw sequences and transfer knowledge across species (zero-shot or few-shot learning).

2. Methodology

The authors introduce PlantCAD2, a DNA language model specifically designed for angiosperms.

• Architecture:

  • Base: Built on the Mamba2 architecture (a State Space Model), which offers linear scaling with sequence length compared to the quadratic scaling of Transformers. This enables efficient processing of long sequences.
  • Design: Retains the Caduceus design principles: bidirectional processing, reverse-complement (RC) equivariance (ensuring the model treats forward and reverse strands symmetrically), and parameter-efficient implementation.
  • Context Window: Expanded to 8,192 base pairs (bp), a 16-fold increase over the previous PlantCAD (512 bp), allowing the capture of distal regulatory elements.
  • Tokenization: Uses single-nucleotide tokenization (A, C, G, T), preserving full resolution unlike k-mer based models.

• Pre-training Strategy:

  • Dataset: Curated from 65 angiosperm genomes representing one species per genus to maximize phylogenetic diversity while minimizing bias toward overrepresented families (e.g., grasses).
  • Objective: Masked Language Modeling (MLM) with a 15% masking rate.
  • Bias Mitigation: Applied loss re-weighting to down-weight repetitive sequences (which are easy to predict but less informative for function) and emphasize coding/regulatory regions.
  • Model Scales: Three variants were trained: Small (88M), Medium (311M), and Large (676M parameters).

• Evaluation Framework:

  • Zero-Shot: Evaluated without fine-tuning on tasks like evolutionary conservation, junction recovery, and structural variant impact.
  • Fine-Tuning: Used Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning on downstream tasks (chromatin accessibility, gene expression, protein translation).
  • Benchmarks: Compared against predecessors (PlantCAD, GPN), generalist models (Evo2, NTv3), and other plant models (AgroNT).

3. Key Contributions

1. PlantCAD2 Model: The first plant-specific DNA foundation model with an 8kb context window and single-nucleotide resolution, built on the efficient Mamba2 architecture.
2. Phylogenetic Curation: A deliberate strategy to pre-train on 65 diverse angiosperm genomes to capture shared regulatory grammar without the noise of non-plant lineages.
3. Comprehensive Benchmarking: Established a suite of 12 zero-shot and 7 fine-tuned benchmarks covering conservation, regulatory motifs, chromatin accessibility, gene expression, and translation.
4. Demonstration of Efficiency: Proved that a 676M-parameter plant-specific model can outperform a 7B-parameter generalist model (Evo2) on plant-specific tasks, highlighting the value of domain-specific pre-training over sheer parameter scale.

4. Key Results

A. Zero-Shot Performance (No Fine-Tuning)

• Evolutionary Conservation: PlantCAD2 (676M) surpassed the 7B-parameter Evo2 in 10 out of 12 conservation tasks.
  • Example: In predicting conserved sites within the Andropogoneae tribe, PlantCAD2-L achieved an AUROC of 0.725 vs. Evo2's 0.691.
  • TIS Prediction: PlantCAD2 showed robust performance at Translation Initiation Sites (AUROC ~0.67), whereas Evo2 dropped to near-random (0.534), likely due to Evo2's unidirectional nature failing to capture bidirectional context required for start codons.
• Junction Recovery: PlantCAD2 accurately recovered masked motifs for splice donors/acceptors and translation start/stop sites in both maize (training species) and tomato (unseen species), outperforming all baselines.
• Structural Variants: PlantCAD2 successfully predicted the functional impact of deletions (indels) using zero-shot ΔlogP scores, outperforming Evo2 even with a much smaller parameter count.
• Cross-Lineage Generalization: The model performed well on Solanaceae (potato/tomato), a family excluded from pre-training, demonstrating it learned fundamental angiosperm constraints rather than memorizing specific sequences.

B. Fine-Tuned Performance (Downstream Tasks)

• Chromatin Accessibility:
  • Fine-tuned PlantCAD2 significantly outperformed supervised baselines (CNN+LSTM) and AgroNT in predicting ATAC-seq peaks across 10 species.
  • Context Length Impact: Increasing the input window from 600 bp to 4,600 bp improved AUPRC for maize from 0.587 to 0.711, proving the critical importance of long-range context for distal regulation.
  • Cell-Type Specificity: Successfully predicted cell-type-specific accessibility in maize (92 cell types) with high accuracy (AUPRC 0.662), outperforming baselines.
• Gene Expression & Translation:
  • Outperformed AgroNT (1B params) and supervised models in cross-species gene expression prediction (Arabidopsis → Maize).
  • Showed strong transferability for protein translation (binary classification AUROC 0.692), suggesting translational control is more evolutionarily conserved than transcriptional regulation.

5. Significance and Impact

• Paradigm Shift: PlantCAD2 establishes a new standard for plant genomics, moving from task-specific models to a unified foundation model that can be adapted to diverse species and tasks with minimal supervision.
• Accessibility: By demonstrating that a ~676M parameter model outperforms 7B-40B generalist models, PlantCAD2 makes advanced genomic interpretation feasible for standard plant genomics laboratories without requiring massive compute resources.
• Biological Insight: The model's success with long context windows validates the biological hypothesis that distal regulatory interactions (enhancers) are crucial for plant gene regulation, a feature previously missed by short-window models.
• Applications: The model enables:
  • Accurate genome annotation for non-model crops.
  • Prioritization of causal variants in GWAS.
  • Interpretation of structural variants in non-coding regions.
  • Synthetic biology (designing promoters with desired expression patterns).

In conclusion, PlantCAD2 represents a significant leap forward in plant genomics, offering a powerful, efficient, and versatile tool to bridge the gap between the explosion of plant genomic data and the need for functional understanding.