HViLM: A Foundation Model for Viral Genomics Enables Multi-Task Prediction of Pathogenicity, Transmissibility, and Host Tropism

The paper introduces HViLM, the first foundation model for pan-viral genomic analysis, which achieves state-of-the-art performance in predicting pathogenicity, host tropism, and transmissibility across diverse viral families through continued pre-training on 5 million non-redundant viral sequences and parameter-efficient fine-tuning.

The Gist

Imagine the world of viruses as a massive, chaotic library containing millions of books written in a secret language (DNA and RNA). For a long time, scientists had to read every single book one by one to figure out if a new virus was dangerous, who it could infect, and how fast it could spread. This was slow, expensive, and often too late to stop an outbreak.

The paper you shared introduces HViLM (Human Virome Language Model), a new "super-reader" AI designed to solve this problem. Here is how it works, explained simply:

1. The Problem: Too Many New Books, Too Little Time

Every time a new virus appears (like a new edition of a scary book), scientists usually have to start from scratch to understand it. Old methods are like trying to find a specific word in a dictionary by looking at every single page. They are slow and often fail when the virus is completely new.

2. The Solution: HViLM, the "Super-Reader"

The researchers built HViLM, which is like a genius librarian who has read the entire library of viral history.

• The Training: Instead of just reading a few books, they fed this AI 5 million different viral sequences (chunks of genetic code) from a massive database called VIRION.
• The "Continued Pre-training": Think of existing AI models (like DNABERT-2) as students who studied general biology. The researchers took these students and gave them a specialized boot camp focused entirely on viruses. This allowed the AI to learn the specific "dialect" and "grammar" of viruses, not just general biology.

3. The Three Superpowers

Once trained, HViLM can look at a new virus and instantly answer three critical questions:

• Is it Dangerous? (Pathogenicity)

  • Analogy: Imagine a security guard checking a guest list. HViLM can tell if a virus is a "peaceful tourist" or a "dangerous criminal" just by reading its genetic code.
  • Result: It got this right 95% of the time, beating all previous methods.

• Who can it infect? (Host Tropism)

  • Analogy: Think of a virus as a key and a human cell as a lock. HViLM can look at the key and say, "This key fits human locks, but not cat locks," or "This key only fits bat locks."
  • Result: It correctly identified human-infecting viruses 96% of the time.

• How fast will it spread? (Transmissibility)

  • Analogy: This is like predicting if a rumor will stay in one room or spread to the whole city. HViLM predicts if the virus will cause a small, contained outbreak or a global pandemic.
  • Result: It predicted this with 97% accuracy.

4. The "Magic Glasses": How It Thinks

The most exciting part isn't just that the AI is fast; it's that we can see how it thinks. Usually, AI is a "black box"—you put data in, and it gives an answer, but you don't know why.

The researchers put on "magic glasses" (called Attention Analysis) to see what parts of the virus the AI was focusing on. They discovered something fascinating:

• The Virus is a Master of Disguise: The AI found that dangerous viruses have tiny genetic "stubs" or patterns that look exactly like human body signals.
• The Heist: It's like a burglar who doesn't just break the door; they wear a uniform that looks exactly like the police officer so the real police let them in.
  • The AI found that viruses mimic human signals that control the immune system (specifically a signal called Irf1). By copying these signals, the virus tricks the body into thinking, "Oh, this is a friend, don't attack it!"
  • It also found viruses mimicking signals that control lung cells (Foxq1), helping the virus sneak into the lungs.

5. Why This Matters

• Speed: In the past, characterizing a new virus took months. With HViLM, it could take minutes.
• Preparedness: If a new virus jumps from a bat to a human, HViLM can immediately tell us: "This one is dangerous, it can infect humans, and it spreads fast." This gives public health officials a head start.
• New Cures: By understanding exactly how the virus disguises itself (the specific genetic patterns it copies), scientists can design drugs to block those specific disguises.

Summary

HViLM is a highly trained AI librarian that has read millions of viral books. It can instantly tell us if a new virus is dangerous, who it can infect, and how fast it will spread. Even better, it acts like a detective, revealing the secret "disguises" viruses use to trick our immune systems, helping us fight back faster and smarter.

The best part? The researchers made the "library" and the "librarian" available for free so other scientists can use them to prepare for the next pandemic.

Technical Summary

1. Problem Statement

The emergence of novel viral pathogens poses critical threats to global health. Current computational approaches for viral risk assessment suffer from significant limitations:

• Virus-Specificity: Existing methods are often tailored to specific viruses, requiring extensive retraining for new threats.
• Lack of Generalization: Traditional tools (e.g., BLAST, HMMER) and k-mer-based classifiers struggle with computational efficiency, sensitivity to novel pathogens, and generalization across diverse viral families.
• Single-Task Focus: Most existing genomic foundation models (e.g., DNABERT, Nucleotide Transformer) are pre-trained primarily on prokaryotic genomes or human microbiomes, lacking comprehensive benchmarks for multi-task viral phenotype prediction (pathogenicity, host tropism, transmissibility) essential for pandemic preparedness.

There is an urgent need for a unified, scalable foundation model capable of rapid, multi-dimensional characterization of emerging viruses to guide public health responses.

2. Methodology

A. Data Curation and Pre-training

• Source Data: The model utilizes the VIRION database, a comprehensive resource containing ~476,000 virus-host interactions across 9,000 viral species and 3,767 vertebrate hosts.
• Sequence Processing:
  • Retrieved complete viral genomes and segmented them into non-overlapping 1,000 bp chunks.
  • Applied MMseqs2 clustering at an 80% identity threshold to remove redundancy while preserving diversity.
  • Final Corpus: 5 million unique, non-redundant viral sequences spanning 45+ viral families (covering all Baltimore classification groups).
• Base Architecture: Built upon DNABERT-2 (117M parameters, 12-layer transformer), which was originally pre-trained on prokaryotic and viral genomes.
• Continued Pre-training: The authors performed domain-adaptive continued pre-training on the 5 million viral chunks using a Masked Language Modeling (MLM) objective.
  • Masking Strategy: 15% of tokens masked (80% [MASK], 10% random, 10% unchanged).
  • Training: 10 epochs on 4 NVIDIA A100 GPUs, achieving 94.2% MLM accuracy on a held-out validation set.

B. The HVUE Benchmark

To rigorously evaluate the model, the authors introduced the Human Virome Understanding Evaluation (HVUE) benchmark, comprising seven curated datasets across three critical tasks:

1. Pathogenicity Classification: Distinguishing disease-causing strains from benign ones (Datasets: CINI, BVBRC-CoV, BVBRC-Calici).
2. Host Tropism Prediction: Identifying human-infecting vs. non-human-infecting viruses (Dataset: VHDB, 30 viral families).
3. Transmissibility Assessment: Evaluating epidemic potential via $R_0$ classification ($R_0 < 1$ vs. $R_0 \ge 1$) (Datasets: Coronaviridae, Orthomyxoviridae, Caliciviridae).

• Data Leakage Prevention: Strict overlap analysis ensured no sequences in the HVUE test sets were present in the pre-training corpus.

C. Fine-Tuning and Efficiency

• Parameter-Efficient Fine-Tuning (PEFT): The authors employed Low-Rank Adaptation (LoRA) to adapt the 117M parameter base model to specific tasks.
  • LoRA was applied to query and value projection matrices in all 12 attention layers (Rank $r=8$, scaling $\alpha=16$).
  • Parameter Efficiency: Only 0.3 million trainable parameters (0.26% of total) were added per task, preventing catastrophic forgetting and reducing computational costs.
• Training: Fine-tuning took <6 hours per task on a single GPU, compared to estimated 200–300 GPU-hours for training from scratch.

D. Interpretability Framework

A systematic pipeline was developed to link sequence representations to biological mechanisms:

1. Attention Analysis: Extracting attention weights from the final transformer layer to identify high-attention genomic regions.
2. Motif Discovery: Using MEME-ChIP to identify conserved sequence patterns in high-attention regions.
3. TF Mapping: Matching discovered motifs against the JASPAR vertebrate transcription factor database using TOMTOM.
4. Statistical Validation: Permutation testing to validate local and global enrichment of motifs in pathogenic strains.

3. Key Contributions

1. HViLM (Human Virome Language Model): The first foundation model specifically designed for pan-viral genomic analysis via continued pre-training on 5 million viral sequences.
2. HVUE Benchmark: The first systematic multi-task evaluation framework for viral genomics, covering pathogenicity, host tropism, and transmissibility across diverse viral families.
3. State-of-the-Art Performance: Demonstrated that viral-specialized pre-training significantly outperforms general genomic models and sequence-similarity baselines.
4. Mechanistic Interpretability: Provided evidence that the model learns biologically meaningful features, specifically identifying molecular mimicry of host regulatory elements (transcription factors) as a core mechanism of viral pathogenicity.

4. Results

Predictive Performance

HViLM achieved state-of-the-art results across all tasks, substantially outperforming baselines like Nucleotide Transformer, GENA-LM, and DNABERT-MB:

• Pathogenicity: 95.32% average accuracy.
• Host Tropism: 96.25% average accuracy.
• Transmissibility: 97.36% average accuracy.
• Generalization: The model demonstrated robust cross-family generalization, maintaining high performance even on families not heavily represented in the training data, whereas general genomic models showed significant performance drops.

Interpretability Findings

Attention-based analysis revealed that HViLM captures specific biological determinants of pathogenicity:

• 42 Conserved Motifs: Identified 42 distinct viral motifs (14–20 bp) associated with pathogenicity.
• Host Mimicry: These motifs matched binding sites for 10 distinct vertebrate transcription factors.
• Convergent Evolution:
  • Irf1 (Interferon Regulatory Factor 1): Eight independent viral motifs converged on mimicking Irf1 binding sites, indicating strong positive selection for immune evasion.
  • Foxq1: Multiple motifs targeted Foxq1, a regulator of epithelial differentiation, suggesting a strategy for epithelial tropism.
  • ZNF354A: Motifs matched this factor, implicating chromatin regulation hijacking.
• Significance: Unlike alignment-based methods that miss dispersed non-homologous motifs, HViLM's transformer architecture successfully identified these functional elements, revealing coordinated multi-target genomic strategies used by viruses to hijack host machinery.

5. Significance

• Pandemic Preparedness: HViLM provides a scalable, rapid computational tool for characterizing emerging viral threats across multiple epidemiologically relevant dimensions without requiring task-specific retraining from scratch.
• Therapeutic Discovery: By identifying specific host transcription factors mimicked by viruses (e.g., Irf1, Foxq1), the model offers novel targets for antiviral drug development and understanding viral immune evasion mechanisms.
• Resource Availability: The authors have open-sourced the HVUE benchmark, training scripts, and pre-trained model weights (HViLM-base and task-specific variants) on GitHub and Hugging Face, fostering reproducibility and further research in viral genomics.
• Computational Efficiency: The use of LoRA makes the model practical for deployment in resource-constrained environments during outbreaks, achieving 30–50× computational savings compared to training from scratch.

In conclusion, HViLM represents a paradigm shift from single-task, virus-specific tools to a unified, interpretable foundation model that not only predicts viral risk with high accuracy but also elucidates the underlying molecular mechanisms of pathogenicity.