DVPNet: A New XAI-Based Interpretable Genetic Profiling Framework Using Nucleotide Transformer and Probabilistic Circuits

This paper introduces DVPNet, an interpretable XAI framework that integrates the Nucleotide Transformer with probabilistic circuits to quantify gene importance and uncover biologically significant pathways distinguishing cancer cells from normal cells, revealing insights that transcend traditional statistical analyses.

The Gist

The Big Picture: Finding the "Smoking Gun" in a Sea of Clues

Imagine you are a detective trying to solve a mystery: How do we tell a "Cancer Cell" apart from a "Normal Cell"?

In the past, scientists tried to solve this by counting how often certain suspects (genes) appeared in the crime scene. If a specific gene showed up 100 times in cancer cells and only 5 times in normal cells, they assumed that gene was the "bad guy." This is like saying, "The guy wearing a red hat must be the thief because I saw him at the bank 10 times."

But this paper argues that counting isn't enough. Sometimes, a gene might appear often just by chance, or a rare gene might be the real mastermind behind the cancer.

The author, Taishi Kusumoto, built a new digital detective called DVPNet. Instead of just counting, this detective reads the "story" inside the DNA to understand what the genes are actually doing.

---

The Two Main Tools in the Detective's Kit

To solve this mystery, the detective uses two high-tech tools:

1. The "DNA Translator" (Nucleotide Transformer)

Think of DNA as a book written in a complex, ancient language.

• Old way: Scientists just looked at the page numbers (how many times a word appeared).
• New way (The Transformer): This tool is like a super-smart translator that has read millions of DNA books. It doesn't just count words; it understands the context. It knows that the word "apple" means something different in a recipe than it does in a tech company's name.
• In the paper: The model reads the DNA sequence of a gene (from the start of the gene to a bit before and after) and turns it into a "meaning vector." It captures the biological function of the gene, not just its frequency.

2. The "Glass Box" Judge (Probabilistic Circuits)

Most modern AI models are "Black Boxes." You put data in, and they give an answer, but you have no idea why they decided that. It's like a judge who says, "Guilty," but won't tell you the evidence.

• The Problem: If you can't see the evidence, you can't trust the verdict in biology.
• The Solution (DVPNet): This model is a "Glass Box." It is built using Probabilistic Circuits. Imagine a courtroom where every piece of evidence (every gene) is weighed individually. The model calculates: "How much does this specific gene contribute to the verdict of 'Cancer'?"
• The Result: It gives a score for every single gene, explaining exactly how much that gene pushed the decision toward "Cancer" or "Normal."

---

The Experiment: The Great Mix-Up

The researcher tested this on a massive dataset of lung cells (from the GSE131907 atlas).

1. The Setup: They took 900 random genes from each cell. They didn't pick the "loudest" genes (the ones with the most activity); they picked them randomly to be fair.
2. The Training: They taught the model to distinguish between cancer and normal cells.
3. The Surprise: The model didn't just rely on which genes appeared most often. It found 1,524 genes that were "contradictory."

What does "Contradictory" mean?
Imagine a gene that appears rarely in cancer cells (only 5 times) but often in normal cells (20 times).

• Old Logic: "This gene is rare in cancer, so it must be a 'Normal' gene. It shouldn't help identify cancer."
• DVPNet Logic: "Wait! Even though this gene is rare, the way its DNA is written suggests it is actually a key player in the cancer process. It's a 'sleeper agent'!"

The model gave these rare genes high "Cancer Scores" because the DNA Translator understood their hidden biological function, overriding the simple count.

The Results: New Clues for Scientists

The study found that the model prioritized genes that are already famous in cancer research (like ITGA5 and TP73), proving it works. But more importantly, it highlighted genes that traditional statistics missed.

• The Network: The researchers grouped these genes into "neighborhoods" (modules). Some neighborhoods were full of immune system genes, suggesting that the difference between cancer and normal cells isn't just about the cells themselves, but how the immune system interacts with them.
• The Insight: The model realized that the "Cancer" label wasn't just about the tumor cells; it was about the whole environment (the tumor microenvironment) fighting back.

Why This Matters (The "So What?")

1. Beyond Counting: It proves that biology is too complex to be solved by simple math (counting). You need to understand the story the DNA is telling.
2. Trustworthy AI: Because the model is "interpretable" (a Glass Box), scientists can actually look at the scores and say, "Ah, I see why the model thinks this gene is important." This builds trust.
3. New Discoveries: By finding genes that contradict simple statistics, this method acts like a spotlight, showing researchers new suspects to investigate that they might have ignored before.

Summary Analogy

If traditional genetic analysis is like counting how many people are wearing red hats to find a crowd, DVPNet is like a detective who reads the conversations of the people in the crowd.

Even if only one person is wearing a red hat, if that person is whispering a secret plan to start a riot, DVPNet will spot them immediately. It combines the power of a super-smart language translator (Nucleotide Transformer) with a transparent, logical judge (Probabilistic Circuits) to find the true biological drivers of cancer, not just the most common ones.

Technical Summary

1. Problem Statement

Traditional genetic analysis often relies on gene co-expression networks derived from RNA sequencing data. These networks are built on statistical correlations of expression levels, which have significant limitations:

• Lack of Causality: They indicate which genes are active simultaneously but do not distinguish between regulatory and regulated genes or provide causal insights.
• Context Blindness: Genes in the same biological pathway often do not show similar expression patterns, a nuance missed by correlation-based methods.
• Black-Box AI: While deep learning models (CNNs, Transformers) offer strong feature extraction, their "black-box" nature prevents the interpretation of why a specific gene contributes to a classification decision (e.g., cancer vs. normal).

The paper addresses the need for a framework that combines the feature extraction power of foundation models with probabilistic interpretability to uncover biological insights beyond simple statistical frequency or expression correlation.

2. Methodology: The DVPNet Framework

The authors propose DVPNet, a novel Explainable AI (XAI) classification model adapted from VPNet (an image classification model). The workflow consists of four main stages:

A. Data Preparation & Encoding

• Dataset: The study utilizes the GSE131907 single-cell lung cancer atlas, comprising 208,506 cells from 44 patients (cancer vs. normal tissues).
• Gene Selection: To avoid bias toward highly expressed genes, 900 expressed genes are randomly sampled per cell, regardless of expression level.
• Sequence Extraction: For each gene, the model extracts a nucleotide sequence from -2000 bp upstream to +500 bp downstream of the Transcription Start Site (TSS). This preserves intronic and regulatory region information often lost in mature RNA analysis.
• Embedding: These sequences are processed by the Nucleotide Transformer (a foundation model trained on 3,202 human genomes). The model outputs 1,024-dimensional embedding vectors for each gene.

B. Model Architecture: Probabilistic Circuits

DVPNet replaces the Vision Transformer in VPNet with the Nucleotide Transformer and utilizes Probabilistic Circuits (PCs) for the classification head.

• Structure: The model treats each gene vector as an input to a tractable probabilistic circuit.
• Decomposability & Smoothness: The circuit structure ensures that the joint probability distribution over all 900 genes is factorizable (decomposable) because gene scopes are disjoint.
• Training Objective: The model optimizes a power posterior using Bayes' rule. It minimizes a loss function combining cross-entropy (for classification accuracy) and a Shannon-entropy regularizer (to prevent overconfidence).
• Key Innovation: Unlike standard neural networks, PCs allow the extraction of conditional probability distributions for every single gene given a class label ($P(G_i | \text{class})$) during inference.

C. Scoring Mechanism

The framework defines a probabilistic contribution score ($S(gene)$) for each gene:
$$S(gene) = S(gene | \text{cancer}) - S(gene | \text{normal})$$
Where $S(gene | \text{class})$ is the log-probability contribution of that gene to the specific class.

• Contradictory Filtering: A critical step involves identifying genes where the contribution score contradicts the raw occurrence frequency (e.g., a gene is less frequent in cancer samples but the model assigns it a high positive contribution to the cancer class). This isolates features driven by biological context rather than simple statistics.

3. Key Contributions

1. DVPNet Architecture: The first integration of the Nucleotide Transformer with Probabilistic Circuits for single-cell genetic profiling, enabling tractable, interpretable decision-making.
2. Beyond Statistics: The framework successfully decouples gene importance from raw expression frequency. It identifies genes that are biologically significant for classification despite having low or contradictory occurrence rates.
3. New Genetic Networks: The authors construct a WGCNA-based network using probabilistic contribution scores ($S(gene|sample)$) instead of expression correlations, revealing functional modules distinct from traditional co-expression networks.
4. Biological Validation: The model prioritizes known cancer-related genes (e.g., ITGA5, SIGLEC9, NOTUM, TP73) and immune pathways, validating its ability to capture relevant biological signals.

4. Results

• Classification Performance: The model achieved high accuracy on both training and test sets (AUROC > 0.97), demonstrating no significant overfitting or underfitting.
• Feature Representation: A moderate correlation ($r \approx 0.35$) was found between raw gene frequency differences and the learned contribution scores. This confirms the model relies heavily on the biological feature representations encoded by the Nucleotide Transformer, not just frequency statistics.
• Contradictory Genes: Out of 9,540 observed genes, 1,524 showed contradictory count-score pairs.
  • Top Ranked: Included well-known cancer targets like ITGA5 (integrin), SIGLEC9 (immune checkpoint), and TP73 (tumor suppressor).
  • Bottom Ranked: Included genes with high frequency in cancer but negative contribution scores, suggesting they may be bystanders or markers of the tumor microenvironment rather than drivers.
• Pathway Analysis:
  • Positive Contributors: Enriched in immunoglobulin complexes, complement activation, and humoral immune responses.
  • Negative Contributors: Enriched in metabolic processes, ion transport, and structural components.
  • Module Analysis: The WGCNA analysis identified 50 distinct gene modules. The "orange" module (highest cancer score) was linked to proton transport and macroautophagy, while the "royal blue" module (lowest score) was linked to methionine catabolism.

5. Significance and Limitations

Significance:

• Interpretability: DVPNet provides a mathematically rigorous way to visualize why a model classifies a cell as cancerous, moving beyond "black box" predictions.
• Novel Biological Insights: By leveraging the Nucleotide Transformer, the model captures regulatory and functional relationships encoded in DNA sequences that RNA expression levels alone miss.
• Complementary Workflow: This approach complements traditional differential expression analysis and co-expression networks, offering a new perspective on gene regulation in cancer.

Limitations:

• Dataset Specificity: The study focused solely on lung cancer (epithelial cells). Generalizing these findings to other cancer types requires broader datasets.
• Microenvironment Confounding: The model may be distinguishing between the tumor microenvironment (immune cells, stroma) and normal tissue rather than purely cancer cell vs. normal cell intrinsic biology, as evidenced by the strong immune-related GO terms.
• Validation: While the results align with existing literature, wet-lab experimental validation of the top-ranked novel candidates was not performed in this study.

Conclusion

DVPNet represents a significant step forward in computational biology by merging large-scale foundation models with interpretable probabilistic circuits. It demonstrates that AI can move beyond statistical correlations to identify biologically meaningful gene contributions, offering a powerful tool for discovering new therapeutic targets and understanding cancer mechanisms.