🤖 AI

NeuroAlign: Hierarchical Multimodal Fusion of Dynamic and Structural Neuroimaging for MCI Analysis

The paper proposes NeuroAlign, a hierarchical multimodal fusion framework that integrates fMRI and DTI data through dual-modal alignment and dual-domain interaction mechanisms to improve the detection of Mild Cognitive Impairment while providing interpretable feature attribution.

Original authors: Xiongri Shen, Zhenxi Song, Jiaqi wang, Yi Zhong, Leilei Zhao, Chenqi Xu, Linling Li, Yichen Wei, Lingyan Liang, Demao Deng, Luping Song, Ping Luan, Ahmed M. Anter, Shuqiang Wang, Baiying Lei, Zhiguo Z

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiongri Shen, Zhenxi Song, Jiaqi wang, Yi Zhong, Leilei Zhao, Chenqi Xu, Linling Li, Yichen Wei, Lingyan Liang, Demao Deng, Luping Song, Ping Luan, Ahmed M. Anter, Shuqiang Wang, Baiying Lei, Zhiguo Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Putting Together a Puzzle with Different Pieces

Imagine you are trying to solve a mystery about why a brain is having trouble thinking clearly (a condition called Mild Cognitive Impairment, or MCI). You have two types of clues:

The "Activity" Clue (fMRI): This shows how different parts of the brain are talking to each other right now. It's like listening to a conversation in a busy room.
The "Structure" Clue (DTI): This shows the physical roads (white matter) that connect those parts. It's like looking at a map of the city's highways.

The problem is, these two clues speak different languages. The "activity" changes every second, while the "roads" are mostly static. Also, the data comes from different hospitals with different machines, making the clues look slightly different depending on where you get them.

NeuroAlign is a new computer program designed to be the ultimate translator and detective. It takes these messy, different clues and forces them to work together perfectly to spot the signs of cognitive trouble.

How It Works: The Three Superpowers

The paper describes NeuroAlign as having three main "superpowers" to solve the puzzle:

1. The Time-Traveling Translator (DMHA)

The Problem: Brain activity happens at different speeds. Some things happen in a flash (like a sudden spark), while others are slow waves. Traditional tools often look at the brain through a "fixed window," like watching a movie with the frame rate stuck on one setting. You might miss the fast sparks or the slow waves.
The Solution: NeuroAlign uses a Pyramid of Time. Imagine looking at a landscape through five different zoom lenses at once:

One lens looks at 20-second chunks.
Another looks at 40-second chunks.
And so on, up to 100 seconds.
It takes all these different views and blends them into one perfect picture. It also makes sure the "Activity" clues and the "Road Map" clues are speaking the same language, so they don't get confused.

2. The Team Huddle (DDHI)

The Problem: Once the computer has all the clues, it needs to decide which ones matter. Old methods just dumped all the clues into a pile and hoped for the best. This is like throwing all the ingredients for a cake into a bowl without mixing them properly.
The Solution: NeuroAlign holds a Team Huddle.

Fine-Grained Chat: It lets specific brain regions (like the "front office") talk directly to the connections between them (the "delivery trucks").
Global Meeting: Then, it holds a big meeting where everything talks to everything else. This ensures the computer understands not just isolated facts, but how the whole brain is working together.

3. The "Why" Detector (SAM)

The Problem: Deep learning models are often "black boxes." They give an answer ("Yes, this patient has MCI"), but they don't explain why. Doctors need to know which part of the brain triggered the alarm.
The Solution: The authors created a tool called Synergistic Activation Mapping (SAM).

Think of this as a highlighter pen. After the computer makes a decision, SAM goes back and highlights exactly which "roads" (structural) and which "conversations" (functional) were most important for that decision.
It creates a heat map showing, for example, "The computer noticed the connection between the memory center and the thinking center was weak."
Important Note: The paper stresses that these are "model-derived" highlights. They show what the computer thought was important, not necessarily a proven medical fact about the disease.

What Did They Find? (The Results)

The researchers tested NeuroAlign on three different groups of people from different hospitals (datasets named GUTCM, ADNI, and OASIS).

Better Accuracy: When they compared NeuroAlign to other top-tier computer programs, NeuroAlign did a better job at spotting MCI. It was particularly good at catching the "troublemakers" (high recall), meaning it didn't miss as many people who actually had the condition.
The Power of Mixing: They proved that mixing the "Activity" and "Structure" clues together works much better than using just one type.
The "Alignment" Matters: When they removed the special "Time-Traveling Translator" and "Team Huddle" features, the program got much worse. This proves that simply throwing data together isn't enough; you have to align it carefully.
Cross-Hospital Testing: They tried training the computer on one hospital's data and testing it on another. It worked okay, but not perfectly. This shows that while the tool is smart, the differences between hospital machines (different scanners, settings) still create a barrier. It's like trying to recognize a friend's voice over a phone call when the connection quality changes.

The Bottom Line

NeuroAlign is a sophisticated tool that teaches a computer how to listen to the brain's fast conversations and read its physical maps at the same time. By aligning these different types of data and explaining which parts of the brain it is looking at, it offers a more accurate way to detect early signs of cognitive decline.

However, the authors are careful to say this is a research tool. While it performs well in tests, it still faces challenges when moving between different hospitals, and more testing with larger groups of people is needed before it can be used in real-world clinics.

Technical Summary: NeuroAlign

1. Problem Statement

The integration of functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) offers complementary insights into cognitive impairment, specifically Mild Cognitive Impairment (MCI) and Subjective Cognitive Decline (SCD). However, effective multimodal fusion remains challenging due to three primary issues identified in current literature:

Heterogeneous Feature Spaces: fMRI-derived features (e.g., Dynamic Functional Connectivity [DFC], Static Functional Connectivity [SFC], Amplitude of Low-Frequency Fluctuations [ALFF]) and DTI-derived features (e.g., Fractional Anisotropy [FA]) exist in misaligned semantic and distributional spaces. Simple concatenation fails to model their covariant relationships.
Disjoint Modeling: Existing methods often treat static and dynamic connectivity separately, overlooking the complementary nature of stable network organization versus time-varying interactions. Furthermore, Dynamic Functional Connectivity (DFC) analysis frequently relies on fixed-duration sliding windows, failing to capture temporal patterns across multiple scales.
Lack of Structured Interaction and Attribution: Current deep learning frameworks often process modalities as independent channels without explicit cross-modal alignment. Additionally, they provide limited feature-level attribution, making it difficult to inspect which specific connectivity or regional patterns drive the model's decisions.

2. Methodology: NeuroAlign

The authors propose NeuroAlign, a hierarchical framework designed for structured multimodal fusion. The architecture consists of three synergistic components:

2.1. Dual-Modal Hierarchical Alignment (DMHA)

DMHA harmonizes representations across time scales, modalities, and domains through three sub-modules:

TSA (Multi-Scale Temporal Alignment): Addresses the limitation of fixed sliding windows by constructing a pyramid of DFCs using sliding windows ranging from 20s to 100s. These scale-specific tensors are projected into a shared latent space via lightweight adapters and fused using a depth-wise refinement module to create a unified connectivity embedding ( $l_{CD}$ ).
DSA (Dynamic-Static Contrastive Alignment): Uses contrastive learning to align embeddings of dynamic (DFC) and static (SFC) connectivity for the same subject. This enforces subject-level consistency, ensuring that dynamic and static representations for a single patient are semantically closer than those of different patients.
FSA (Functional-Structural Fusion via Affine-Invariant Learning): Treats the fusion of functional (fMRI) and structural (DTI) features as an invariance learning problem. By creating complementary probes (element-wise product and summation) and aligning them via contrastive loss, the model learns representations that are robust to specific arithmetic fusion rules and heterogeneous acquisition settings.

2.2. Dual-Domain Hierarchical Interaction (DDHI)

DDHI facilitates structured interaction between connectivity-level features ( $l_{CD}, l_{CS}$ ) and regional-level features ( $l_{RF}, l_{RS}$ ):

FI (Fine-grained Modulated Attention): Implements a top-down modulation mechanism where one domain's embedding (e.g., static connectivity) conditions the attention mechanism of another (e.g., functional regional activity), allowing for feature-conditioned routing.
GI (Global Cross-domain Attention): Treats all four latent representations as tokens in a sequence and applies multi-head self-attention to enable all-to-all cross-domain communication, allowing the model to capture task-relevant relationships without a fixed interaction order.

2.3. Synergistic Activation Mapping (SAM)

To support post-hoc interpretability, the authors introduce SAM, a gradient-free, marker-oriented attribution method. Unlike standard Grad-CAM, SAM is adapted for heterogeneous inputs (connectivity matrices and 3D regional maps). It generates modality-specific attribution maps for DFC, SFC, ALFF, and FA by perturbing input channels and observing class-specific responses. This allows for the qualitative inspection of model-relevant brain patterns without claiming clinical causality.

3. Experimental Setup and Results

The framework was evaluated on three independent datasets: GUTCM (local cohort), ADNI, and OASIS. Experiments utilized a five-fold stratified cross-validation protocol.

Modality-Controlled Baselines: To ensure fair comparison, the authors re-implemented several State-of-the-Art (SOTA) methods (e.g., Fang et al., Gao et al., He et al.) by adapting their input interfaces to accept the unified four-feature set (DFC+SFC+ALFF+FA) while preserving their original core architectures.
Performance on GUTCM: NeuroAlign achieved superior performance across multiple diagnostic tasks (SCD/HC, MCI/SCD, MCI/HC, and three-class MCI/SCD/HC) compared to the adapted baselines. Notably, it showed significant improvements in F1-scores and accuracy for the more challenging MCI/SCD and three-class tasks, suggesting the hierarchical design effectively distinguishes adjacent cognitive states.
Performance on ADNI and OASIS: NeuroAlign achieved competitive results, with an accuracy of 0.825 on ADNI and 0.735 on OASIS. While the accuracy gain on ADNI over the strongest reported baseline (Cao et al.) was numerically small, NeuroAlign demonstrated improved recall (0.900 vs. 0.840), indicating better sensitivity to MCI subjects.
Cross-Dataset Transferability: Preliminary transfer experiments showed partial transferability. Models trained on GUTCM or ADNI transferred reasonably well to OASIS, but models trained solely on the smaller OASIS subset performed worse when transferred to larger datasets. This highlights remaining domain gaps due to scanner and protocol differences.
Ablation Studies:
- Removing DMHA and DDHI components (leaving an unaligned four-feature fusion) resulted in a significant drop in performance (Accuracy: 0.565 vs. 0.825), characterized by high precision but very low recall. This confirms that structured alignment is necessary to prevent feature conflict and imbalance.
- Replacing the proposed contrastive alignment objectives (DSA/FSA) with Cosine Similarity or MMD loss resulted in lower performance, validating the necessity of subject-level discriminative alignment.

4. Attribution Analysis

Using SAM, the authors visualized model-relevant patterns:

Connectivity Domain: DFC and SFC highlighted dynamic and static connections involving the Default Mode Network (DMN), Frontoparietal Network (FPN), and regions such as the precuneus, posterior cingulate, and angular gyrus.
Region Domain: ALFF and FA maps highlighted structural and functional activity in the supplementary motor area, superior frontal gyrus, putamen, and thalamus.
Consistency: The analysis revealed partially consistent patterns across modalities (e.g., precuneus and parietal regions), supporting the hypothesis that functional and structural degeneration are covariant.

5. Significance and Claims

The authors position NeuroAlign not as a definitive solution for clinical deployment, but as a structured approach to address specific limitations in multimodal neuroimaging fusion. Key claims include:

Structured Fusion: The framework successfully models alignment and interaction across modalities (fMRI/DTI), temporal scales (multi-scale DFC), and domains (regional/connectivity), leading to more discriminative representations.
Balanced Performance: The method improves the balance between sensitivity (recall) and specificity, particularly in distinguishing subtle cognitive transitions (e.g., MCI vs. SCD).
Interpretability: The SAM module provides a novel way to inspect feature-level attribution in multimodal settings, offering model-derived evidence for brain patterns associated with cognitive impairment.
Modesty on Generalizability: The authors explicitly state that while the method shows competitive performance and partial transferability, significant domain gaps remain due to heterogeneous acquisition protocols. They caution that the current scale (even with >500 participants) is insufficient for population-level generalizability and that the results should be viewed as preliminary evidence for multimodal representation learning rather than validated clinical utility.

The paper concludes that while NeuroAlign offers a promising direction for structured multimodal fusion, future work requires larger multi-center cohorts, harmonization strategies, and prospective validation before clinical translation.