🤖 AI

Decoding Phenotypes: A Framework for Fusing Genomic Language Models and Neuroimaging

The paper proposes GeneFuse, a multimodal framework that integrates pre-trained Genomic Language Model embeddings with neuroimaging features via genotype-conditioned modulation and uncertainty-aware fusion to outperform existing methods in identifying early cognitive decline and dementia.

Original authors: Tianli Tao, Ziyang Wang, Emma Robinson, Rachel Sparks, Le Zhang

Published 2026-08-11

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Tianli Tao, Ziyang Wang, Emma Robinson, Rachel Sparks, Le Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to solve a massive, three-dimensional puzzle where the pieces are made of two very different materials: one set is a detailed, colorful map of a city (your brain scan), and the other set is a long, complex sentence written in a secret code (your DNA). For a long time, doctors have used both the map and the code to figure out why some people's brains start to fog up as they age, leading to conditions like Alzheimer's disease. The map shows the physical changes, like shrinking neighborhoods, while the code holds the instructions that might explain why those changes happen in the first place. But here's the tricky part: these two types of information don't naturally speak the same language. The map is visual and structural, while the code is a sequence of letters. Trying to mash them together has been like trying to mix oil and water; the computer programs used to combine them often just slapped the two pieces side-by-side, losing the subtle, local context of the genetic instructions in the process.

This is where a new approach called "GeneFuse" comes in, offering a smarter way to blend these two worlds. Instead of treating DNA as a simple list of facts, this method uses a "Genomic Language Model"—think of it as a super-smart translator that reads DNA like a story, understanding the context around specific genetic words. The researchers built a system that lets this genetic story gently nudge and adjust the brain map as the computer is looking at it, rather than just adding the story at the very end. They tested this on a group of 182 people, looking at their brain scans and DNA to see if they could spot early signs of memory decline or full-blown dementia. The results suggest that when the computer uses this "language-aware" genetic help, it gets better at spotting the disease, especially when the brain scan alone isn't 100% clear. It's like having a detective who can read the suspect's diary while looking at the crime scene, rather than just looking at the scene and then reading the diary separately.

The Problem: Oil and Water in the Brain

For years, scientists have known that to understand nervous system diseases, you need two things: a look at the brain's structure (neuroimaging) and a look at the person's genetic code (genomics). The brain scan is like a photograph of a house; it shows you if the roof is leaking or the walls are cracking. The genetic code is like the blueprint and the history of the family living there; it might tell you why the roof is leaking or if the family is prone to certain problems.

The problem is that these two sources of information are very different. Traditional methods tried to combine them by turning the genetic code into a simple list of numbers (like "0, 1, 2") and just sticking it next to the brain scan data. The authors of this paper argue that this is a mistake. It's like taking a complex novel and summarizing it as just "Chapter 1: Good, Chapter 2: Bad." You lose all the nuance, the context, and the specific details that make the story meaningful. When you lose the local context of the DNA, you lose the ability to see how specific genetic instructions interact with the brain's physical structure.

The Solution: GeneFuse

To fix this, the team created a new framework called GeneFuse. Think of it as a high-tech kitchen where a master chef (the computer) is trying to cook a perfect dish using two very different ingredients: a 3D brain scan and a long DNA sequence.

1. The Translator (Genomic Language Models)
First, instead of turning the DNA into a boring list of numbers, GeneFuse uses a pre-trained "Genomic Language Model" (GLM). Imagine the DNA sequence as a sentence in a foreign language. Older methods just counted the letters. The GLM, however, reads the sentence and understands the context around the important words. In this study, they focused on a specific part of the DNA called the APOE region, which is a well-known risk factor for Alzheimer's. They took a window of 1,024 base pairs (the "letters" of DNA) around this region and let the GLM turn it into a rich, dense "sentence" that captures the full meaning of that genetic spot.

2. The Nudge (Genotype-Conditioned Feature Modulation)
Next, the system needs to mix this genetic "sentence" with the brain scan. The authors introduced a module called GCFM. Imagine the brain scan is a painting being created by an artist. Usually, the artist just paints based on what they see. With GCFM, the genetic "sentence" acts like a gentle whisper in the artist's ear. It doesn't take over the painting; instead, it tells the artist, "Hey, pay a little more attention to this specific color here," or "Make this texture a bit sharper."

Technically, the system looks at different layers of the brain scan (from the fine details to the big picture) and uses the genetic data to tweak the importance of different features. It's not just adding the DNA at the end; it's letting the DNA guide the brain scan analysis from the very beginning.

3. The Safety Valve (Uncertainty-aware Genomic Residual Fusion)
Here is the cleverest part. Sometimes, the brain scan is so clear that you don't need the DNA at all. Other times, the scan is blurry or confusing, and the DNA might hold the missing clue. If you force the DNA into the mix when it's not needed, it might just add noise and confuse the computer.

To handle this, GeneFuse uses a module called U-GRF. This acts like a smart traffic light. First, the system checks how confident it is in the brain scan alone.

If the scan is very clear (low uncertainty): The traffic light turns red for the DNA. The system ignores the genetic data because the brain scan is doing a great job on its own.
If the scan is confusing (high uncertainty): The traffic light turns green. The system says, "Okay, the scan is struggling. Let's bring in the genetic data to help."

This ensures that the genetic information is only used when it's actually helpful, preventing it from messing up a good diagnosis.

What They Found

The researchers tested this new system on data from 182 people, including healthy controls, people with mild cognitive impairment (MCI), and people with Alzheimer's disease (AD). They asked two main questions:

Can we spot the early signs of cognitive decline (Normal vs. MCI)?
Can we screen for dementia (Normal vs. AD)?

The Results:

Better than before: When they used GeneFuse, the system got much better at spotting the disease compared to just using the brain scan alone. For spotting early decline (Normal vs. MCI), the accuracy score (AUROC) went up to 0.77. For spotting dementia (Normal vs. AD), it jumped to 0.83.
Beating the old ways: These scores were higher than using the old method of just sticking a simple DNA list next to the scan, and also better than other existing methods that try to combine genetics and imaging.
The "Whisper" works: The study showed that the "nudge" from the genetic data (GCFM) and the "safety valve" (U-GRF) both played important roles. The system worked best when it used the genetic data to guide the brain scan at multiple levels of detail, not just at the end.
More genes, more help: While they focused on the APOE gene, they also found that if they added other risk genes (Multi-locus), the system got even better, reaching an AUROC of 0.84 for dementia screening. This suggests that looking at more than just one genetic spot provides extra clues.

What This Means

The paper suggests that treating DNA like a language, rather than a simple list of numbers, allows computers to understand the connection between our genes and our brains much better. By letting the genetic data gently guide the analysis of brain scans—and only stepping in when the scans are unsure—the system can make more accurate predictions about Alzheimer's disease.

However, the authors are careful to note that this was tested on a single group of people (the ADNI dataset). While the results are promising, they suggest that future work needs to test this on other groups to make sure it works for everyone. They also plan to see if they can teach the genetic "translator" even more by training it on larger sets of brain and gene data. For now, though, GeneFuse offers a fresh, playful, and powerful way to decode the complex relationship between our genes and our brains.

=== SUMMARY ===

Technical Summary: GeneFuse – A Framework for Fusing Genomic Language Models and Neuroimaging

Problem Statement

The integration of neuroimaging and genetic data is critical for the precise diagnosis of nervous system diseases, such as Alzheimer's Disease (AD), as these modalities offer complementary information regarding macroscopic structural phenotypes and underlying disease processes. However, effective fusion remains challenging due to significant cross-modality heterogeneity.

Existing imaging-genetics approaches typically encode genetic information as hard-coded, discrete tabular labels (e.g., {0, 1, 2} for Single Nucleotide Polymorphisms). This representation loses the local sequence context surrounding disease-associated variants and fails to capture complex interactions between genomic factors and imaging phenotypes. Furthermore, standard fusion strategies face two fundamental challenges:

Semantic Misalignment: Unlike image-text pairs, gene-image pairs lack explicit semantic alignment.
Coupling Uncertainty: Genotype-phenotype relationships are often weak or indirect. Genetic features may not always manifest as observable imaging information due to genetic resilience or environmental factors. Consequently, incorporating noisy or loosely coupled genomic signals without adaptive mechanisms can degrade predictive performance when imaging features alone are highly informative.

Methodology

The authors propose GeneFuse, a multimodal learning framework designed to align representations from pre-trained Genomic Language Models (GLMs) with features extracted from volumetric neuroimaging. The framework consists of three primary components:

1. Unimodal Representation and Encoders

Volumetric Image Encoding: The framework utilizes a 3D TransUNet backbone to extract high-level anatomical representations from T1-weighted MRI scans. The encoder processes multi-scale feature maps at downsampling scales of 1/2, 1/4, and 1/8. These maps are aggregated into a latent imaging vector ( $z_{img}$ ).
Semantic Genomic Encoding: Instead of tabular SNP encoding, the framework encodes local genomic sequences using a pre-trained GLM (specifically the Nucleotide Transformer v2, NT-v2). For each subject, a patient-specific DNA sequence window (1024 base pairs) centered on a target variant (e.g., APOE) is extracted. This preserves the nucleotide context around disease-associated variants. The frozen GLM produces a dense genomic latent vector ( $z_{gene}$ ).

2. Cross-Modal Feature Fusion

GeneFuse introduces two novel modules to bridge the modalities:

Genotype-Conditioned Feature Modulation (GCFM): Inspired by Feature-wise Linear Modulation (FiLM), this module injects GLM-derived conditioning into the multi-scale spatial feature maps of the 3D volumetric image encoding.
- The genomic embedding ( $z_{gene}$ ) is processed by two Multi-Layer Perceptrons (MLPs) to predict channel-wise scale ( $\gamma$ ) and shift ( $\beta$ ) parameters.
- These parameters modulate the imaging feature maps ( $F'_{img} = \gamma \odot F_{img} + \beta$ ) at the 1/2, 1/4, and 1/8 encoder scales.
- This allows genomic information to recalibrate imaging features before final aggregation, rather than conditioning only the final representation.
Uncertainty-aware Genomic Residual Fusion (U-GRF): This module addresses the coupling uncertainty by adaptively gating the contribution of genotypic features based on the predictive uncertainty of the imaging modality.
- Uncertainty Estimation: Predictive entropy ( $E_{img}$ ) is calculated from the initial logits of the imaging branch alone. High entropy indicates diagnostic ambiguity in the visual modality.
- Gating Mechanism: A gating coefficient $\alpha$ is computed using a sigmoid function applied to an MLP that takes the concatenation of the imaging latent vector and the entropy ( $[z_{img} \oplus E_{img}]$ ).
- Fusion: The final representation is a weighted sum: $z_{final} = (1-\alpha) \cdot z_{img} + \alpha \cdot z'_{gene}$ . When the imaging model is confident (low entropy), $\alpha$ suppresses the genomic signal; when uncertain, $\alpha$ increases the genomic contribution.

Key Contributions

Adaptation of FiLM for Imaging-Genetics: The authors adapt feature-wise linear modulation to align imaging-genomic features through GCFM, using genomic embeddings to generate channel-wise affine parameters that modulate volumetric imaging features at multiple scales.
Uncertainty-Aware Fusion Strategy: The introduction of U-GRF, a lightweight module that uses imaging-derived predictive uncertainty to adaptively gate the contribution of genotypic information, mitigating the risk of performance degradation from noisy genomic signals.
Integration and Validation: The components are integrated into a unified network and validated on Alzheimer's disease detection tasks (NC vs. MCI and NC vs. AD) using the ADNI dataset in both single-locus (APOE) and multi-locus settings.

Experimental Results

The framework was evaluated on the ADNI dataset ( $N=182$ ) using stratified 5-fold cross-validation.

Performance: In the APOE-centered setting, GeneFuse achieved AUROCs of 0.77 (NC vs. MCI) and 0.83 (NC vs. AD).
Comparisons: These results outperformed existing imaging-genetics fusion methods (e.g., MADDi, MMDL, Stage-wise DNN) and standard fusion strategies (Late Concat, FiLM Fusion, Cross-Attention).
Improvement over Baselines: GeneFuse improved upon the image-only baseline by $\Delta 0.08$ and $\Delta 0.12$ AUROC for the respective tasks. These improvements were statistically significant ( $p < 0.05$ ) based on paired DeLong tests.
Ablation Studies:
- Modules: Both GCFM and U-GRF demonstrated complementary roles; GCFM improved risk ranking by conditioning features, while U-GRF improved decision reliability.
- Insertion Scale: Using all three encoder scales (1/2, 1/4, 1/8) for GCFM yielded the highest performance, indicating benefits from both fine anatomical detail and deeper semantic features.
- GLM Selection: NT-v2 outperformed other GLMs (DNAGPT, GROVER, GENA-LM).
- Multi-locus: Integrating multiple AD-related loci (e.g., PVRL2, ABCA7, CD33) alongside APOE further improved performance (AUROC 0.84 for NC vs. AD), suggesting complementary diagnostic information.
Qualitative Analysis: Visualizations showed that GCFM modulation produced spatially structured responses around periventricular white matter and medial temporal regions, areas relevant to AD pathology.

Significance and Claims

The paper claims that GLM-derived genomic embeddings provide additional, actionable information to imaging that is lost when using traditional scalar SNP encodings. By aligning genomic representations with volumetric neuroimaging features, GeneFuse demonstrates that:

Context-sensitive genomic representations (via GLMs) are superior to discrete tabular encodings for imaging-genetics fusion.
Adaptive fusion strategies that account for the uncertainty of the imaging modality (U-GRF) are necessary to prevent the degradation of performance when genetic signals are weak or loosely coupled.
The framework effectively bridges the semantic gap between genomics and neuroimaging, offering a more robust approach for early cognitive decline identification and dementia screening.

The authors note limitations, specifically that the experiments were conducted on a single cohort (ADNI), which may limit generalizability. Future work is proposed to extend the framework to other disease states with more diffuse genomic-imaging associations and to investigate GLM fine-tuning on larger datasets.