Fusion Learning from Dynamic Functional Connectivity: Combining the Amplitude and Phase of fMRI Signals to Identify Brain Disorders

This paper proposes MSFL, a multi-scale fusion learning framework that integrates both amplitude and phase information from dynamic functional connectivity to significantly improve the classification of autism spectrum disorder and major depressive disorder compared to existing models.

The Gist

Imagine your brain isn't a static statue, but a bustling, ever-changing city. For a long time, scientists studying this city (using fMRI scans) only took a single, blurry photograph of the traffic patterns. They looked at how busy different neighborhoods were and how much they seemed to talk to each other on average. This is called Static Functional Connectivity.

But the brain is dynamic! The traffic lights change, rush hour hits, and quiet nights settle in. To understand the city better, scientists started taking a video instead of a photo. This is Dynamic Functional Connectivity (dFC).

However, there was a problem with the video cameras they were using. They were only good at measuring how loud the traffic was (Amplitude). They could tell you if two neighborhoods were both "noisy" at the same time, but they missed the rhythm of the traffic. Did the cars in Neighborhood A and Neighborhood B move in perfect sync? Did they wave at each other? That's the Phase.

This paper introduces a new, super-smart detective named MSFL (Multi-Scale Fusion Learning) that solves this problem by listening to both the volume and the rhythm of the brain's signals.

Here is the breakdown of how it works, using simple analogies:

1. The Two Types of Clues (SWC and PS)

The researchers realized that to catch "brain disorders" (like Autism or Depression), you need two different types of clues:

• The Volume Clue (SWC - Sliding Window Correlation): Imagine watching two people talking. SWC measures: "Are they both shouting at the same time?" or "Are they both whispering?" It looks at the strength of the connection. If two brain regions are both very active together, SWC says, "They are connected!"
• The Rhythm Clue (PS - Phase Synchronization): Now, imagine those same two people. Even if they are whispering (low volume), are they speaking in perfect unison? Are their words landing at the exact same millisecond? PS measures the timing and synchrony. It catches connections that SWC might miss, like two people whispering a secret in perfect rhythm.

The Problem: Previous studies mostly relied on the "Volume Clue" (SWC). They missed the subtle "Rhythm Clue" (PS).

2. The Solution: The "Fusion Kitchen" (MSFL)

The authors built a new model called MSFL. Think of it as a high-tech kitchen where two chefs bring in different ingredients:

• Chef A brings the "Volume" data.
• Chef B brings the "Rhythm" data.

Instead of just mixing them into a big soup (which often loses flavor), MSFL uses a special tool called Cross-Difference Attention (CDA).

• The Analogy: Imagine a translator who doesn't just translate words, but specifically looks for where the two languages disagree or where they have unique nuances. The CDA module looks at the Volume and Rhythm data and asks: "Where are they different? Where do they complement each other?"
• It highlights the unique parts of the Rhythm that the Volume missed, and vice versa.

3. The "Zoom Lens" (Multi-Scale Convolution)

Once the clues are fused, the model needs to understand the story over time.

• The Analogy: Imagine watching a movie. Sometimes you need to see the whole scene (long-term trends), and sometimes you need to zoom in on a specific facial expression (short-term spikes).
• MSFL uses a Multi-Scale Convolutional Network which acts like a camera with a zoom lens that can instantly switch between wide-angle and telephoto. It looks at brain connections over short bursts of time and long periods simultaneously, ensuring no detail is missed.

4. The Verdict (The Results)

The researchers tested this new detective (MSFL) on two big datasets:

1. ABIDE I: A collection of brain scans from people with Autism Spectrum Disorder.
2. REST-meta-MDD: A massive collection of scans from people with Major Depressive Disorder.

The Outcome:

• MSFL was significantly better at identifying these disorders than any previous method.
• It proved that combining the "Volume" (Amplitude) and "Rhythm" (Phase) gives a much clearer picture of what's wrong in the brain.
• When they used a tool called SHAP (which acts like a magnifying glass to see which clues mattered most), they found that both types of clues were essential. Removing either the Volume or the Rhythm made the detective worse at solving the case.

Why This Matters

Think of it like diagnosing a car engine.

• Old Method: You only listened to how loud the engine was. If it was loud, you thought it was running well.
• New Method (MSFL): You listen to the loudness and the rhythm of the pistons. You realize that even if the engine is quiet, if the pistons are out of sync, the car is broken.

By combining the Amplitude (loudness) and Phase (rhythm) of brain signals, this paper gives doctors and scientists a much sharper tool to detect and understand brain disorders, potentially leading to earlier and more accurate diagnoses for conditions like Autism and Depression.

Technical Summary

1. Problem Statement

Dynamic Functional Connectivity (dFC) derived from resting-state fMRI is a critical tool for understanding brain disorders. However, current state-of-the-art methods rely almost exclusively on Sliding Window Correlation (SWC) to extract dFC features.

• Limitation of SWC: SWC captures amplitude correlations between brain regions but has limitations, such as sensitivity to parameter choices (window size/step) and a poor estimation of true correlations during abrupt state transitions. It primarily focuses on signal amplitude.
• The Gap: SWC overlooks phase synchronization (PS), which measures the temporal coupling and phase coherence between neural signals. PS can detect functional synergies that SWC misses (e.g., when signals move in opposite directions but maintain phase coherence).
• Objective: The authors propose that relying solely on amplitude (SWC) is insufficient. A comprehensive characterization of brain dynamics requires integrating both amplitude (SWC) and phase (PS) information to improve the detection and classification of brain disorders like Autism Spectrum Disorder (ASD) and Major Depressive Disorder (MDD).

2. Methodology: The MSFL Framework

The authors propose MSFL (Multi-Scale Fusion Learning), a deep learning framework designed to integrate SWC and PS features. The architecture consists of three main components:

A. Feature Extraction

Two distinct feature branches are derived from preprocessed BOLD signals:

1. SWC Branch: Uses a fixed-size sliding window to calculate Pearson correlation coefficients between pairs of brain regions, capturing amplitude-based temporal correlations.
2. PS Branch: Uses the Hilbert transform to extract instantaneous phase information from BOLD signals. The Phase Locking Value (PLV) is calculated to quantify phase coherence between regions.
   • Preprocessing Note: The paper identifies that narrow-band filtering (0.01–0.13 Hz) is superior to Empirical Mode Decomposition (EMD) for PS preprocessing in deep learning contexts.

B. Feature Fusion Module (Cross-Difference Attention)

This is the core innovation of the paper. Instead of simple concatenation, MSFL uses a sophisticated attention mechanism to fuse the two heterogeneous feature sets:

• Cross-Difference Attention (CDA): This submodule explicitly learns the differences between SWC and PS features. It calculates attention weights and then subtracts the output from 1 ($1 - \text{Attention}$). This forces the model to focus on the dissimilarities and unique temporal variations between the two feature types, rather than just their similarities.
• Cross-Attention (CA): Two subsequent CA modules iteratively fuse the CDA output with the original PS and SWC features, allowing the model to learn complementary correlations between the two modalities.

C. Multi-Scale Convolutional Module

After fusion, the integrated features pass through a multi-scale convolutional network:

• It employs 1D convolutions with progressively increasing kernel sizes (3, 5, 7, 9).
• This design allows the model to capture dynamic connectivity patterns across multiple temporal scales, from short-term fluctuations to long-term dependencies.
• A channel projection and split mechanism preserves global information while extracting local patterns.

D. Classification Head

The refined features are flattened and passed through fully connected layers to output a binary classification (Patient vs. Healthy Control).

3. Key Contributions

1. Novel Feature Integration: This is the first study to combine amplitude (SWC) and phase (PS) information from dFC for brain disorder identification, demonstrating that these two modalities are highly complementary.
2. Cross-Difference Attention (CDA): The introduction of a CDA mechanism that specifically targets the differences between feature types, enhancing the model's ability to leverage unique information from PS that SWC misses.
3. Multi-Scale Temporal Modeling: The design effectively captures dFC dynamics across various time scales, addressing the limitation of single-scale analysis.
4. Comprehensive Validation: The framework is validated on two large-scale public datasets (ABIDE I for ASD and REST-meta-MDD for MDD) against both single-channel and dual-channel baselines.

4. Experimental Results

The model was evaluated on ABIDE I (Autism) and REST-meta-MDD (Depression) datasets using 5-fold cross-validation.

• Performance on ABIDE I:
  • MSFL achieved 70.98% Accuracy, 74.61% Sensitivity, and 76.24% AUC.
  • It outperformed the best baseline (BrainNetFormer with SWC) by ~2.1% in Accuracy and significantly improved upon single-channel models.
• Performance on REST-meta-MDD:
  • MSFL achieved 68.84% Accuracy, 72.19% Sensitivity, and 74.53% AUC.
  • It surpassed the best baseline (FE-STGNN) by ~2.7% in Accuracy.
• Ablation Studies:
  • Preprocessing: Narrow-band filtering (0.01–0.13 Hz) was proven superior to EMD for PS feature extraction.
  • Fusion: The proposed CDA + CA fusion method outperformed simple concatenation, standard Cross-Attention, and other fusion baselines (like LAFF, HAT, MMViT), confirming the necessity of explicitly modeling feature differences.
  • Dual-Channel vs. Single-Channel: Adding PS features to existing single-channel models (e.g., CNNLSTM, STAGIN) consistently improved their performance, validating the utility of phase information.

5. Significance and Interpretability

• Model Explanation (SHAP): Using SHAP (Shapley Additive Explanations), the authors analyzed feature importance.
  • Complementarity: Masking experiments showed that removing either SWC or PS features degraded performance, but removing both caused the most significant drop. This confirms that the two feature types provide non-redundant, complementary information.
  • Key Biomarkers: The analysis identified specific brain regions (e.g., AAL regions 67, 51, 21, 22) and functional connections (e.g., pairs 39-59 and 86-87) that were critical in both SWC and PS features. These recurring connections likely represent robust pathological markers for the studied disorders.
• Clinical Impact: By combining amplitude and phase, MSFL offers a more robust and accurate tool for diagnosing brain disorders, potentially leading to earlier detection and a deeper understanding of the underlying neural mechanisms (e.g., distinguishing between stable connectivity and stimulus-driven synchronization).

Conclusion

The paper successfully demonstrates that Dynamic Functional Connectivity is best characterized by a multi-modal approach. The MSFL framework, through its novel Cross-Difference Attention and multi-scale architecture, effectively fuses amplitude and phase information, setting a new state-of-the-art for classifying Autism and Depression using fMRI data.