🤖 machine learning

Towards a general-purpose foundation model for fMRI analysis

The paper introduces NeuroSTORM, a general-purpose foundation model pre-trained on 28.65 million fMRI frames from over 50,000 subjects that outperforms existing methods across diverse downstream tasks, offering a standardized solution for reproducible and transferable fMRI analysis.

Original authors: Cheng Wang, Yu Jiang, Zhihao Peng, Chenxin Li, Changbae Bang, Lin Zhao, Wanyi Fu, Jinglei Lv, Jorge Sepulcre, Carl Yang, Lifang He, Tianming Liu, Xue-Jun Kong, Quanzheng Li, Daniel S. Barron, Anqi Qiu

Published 2026-03-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cheng Wang, Yu Jiang, Zhihao Peng, Chenxin Li, Changbae Bang, Lin Zhao, Wanyi Fu, Jinglei Lv, Jorge Sepulcre, Carl Yang, Lifang He, Tianming Liu, Xue-Jun Kong, Quanzheng Li, Daniel S. Barron, Anqi Qiu, Randy Hirschtick, Byung-Hoon Kim, Hongbin Han, Xiang Li, Yixuan Yuan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your brain is a massive, bustling city. Every second, billions of neurons are firing like streetlights, traffic signals, and power grids, creating a complex, 4D movie of activity (3D space + time). For decades, scientists have tried to watch this movie to understand how the brain works, diagnose diseases, or even predict if someone is happy or stressed.

But here's the problem: The cameras are all different, and the editors are all using different rules.

Some scientists chop the city into neighborhoods (regions of interest) and only look at the traffic between them. Others try to watch the whole city at once but get overwhelmed by the sheer amount of data. Because everyone uses different tools and methods, it's hard to compare results. If one lab says "this brain pattern means depression," another lab might not be able to repeat the experiment because their "camera settings" were different.

Enter NeuroSTORM: The Universal Brain Translator.

This paper introduces a new AI model called NeuroSTORM. Think of it as a "Foundation Model" for the brain, similar to how ChatGPT is a foundation model for language. Instead of learning from text, it learned from 28.65 million frames of brain movies from over 50,000 people.

Here is how it works, using some simple analogies:

1. The Problem with Old Methods

The "Map" Problem: Old methods tried to simplify the brain by projecting it onto a pre-made map (like a subway map). This is like trying to understand a city by only looking at the subway lines. You miss the parks, the side streets, and the unique architecture. You lose information.
The "Redundancy" Problem: Brain movies are full of repetitive stuff. If you know what's happening in one room, you can guess what's happening in the room next door. Old AI models got lazy; they just memorized these easy patterns instead of learning the deep, complex secrets of the brain.

2. How NeuroSTORM Solves It

NeuroSTORM is built with three special superpowers:

The "Smart Window" (Shifted-Window Mamba):
Imagine trying to watch a 4-hour movie on a tiny phone screen. It's impossible. NeuroSTORM uses a "Smart Window" technique. Instead of trying to see the whole city at once, it looks at a few blocks at a time, but it constantly shifts the window so it catches the connections between different neighborhoods. It's efficient enough to run on standard computers without needing a supercomputer.
The "Redundancy Dropout" (STRD):
To stop the AI from getting lazy, the researchers played a game of "Hide and Seek" during training. They randomly covered up (masked) parts of the brain movie. But here's the trick: they covered up the boring, repetitive parts first. This forced the AI to pay attention to the interesting, unique signals that actually matter, rather than just copying its neighbors.
The "Universal Adapter" (Task-Specific Prompt Tuning):
Once the AI learned the general language of the brain, how do you use it for specific jobs? Instead of rebuilding the whole brain from scratch for every new task, NeuroSTORM uses a tiny "adapter" (like a USB dongle). You plug in a small, specific instruction (a "prompt") for the job you need—like "Diagnose ADHD" or "Predict Age"—and the AI instantly switches gears. It's like having a Swiss Army knife where you just swap the blade for the job at hand.

3. What Can It Do?

The researchers tested NeuroSTORM on five different "challenges" and it crushed them all:

Guessing Demographics: It can tell your age and gender just by looking at your brain scan, better than any previous method.
Predicting Personality: It can guess if you are good at math, how emotional you are, or how stressed you feel.
Diagnosing Disease: It can spot signs of schizophrenia, ADHD, or motor neuron disease with high accuracy, even when there is very little data to learn from.
Identifying People: It can tell if two brain scans belong to the same person (like a fingerprint), which is crucial for security and research.
Reading Thoughts (State Classification): It can tell if a person is currently thinking about "fear," "math," or "walking," just by watching their brain activity.

4. Why This Matters

The biggest breakthrough is efficiency and reliability.

Data-Starved: Usually, AI needs thousands of labeled examples to learn a new task. NeuroSTORM can learn a new task with very few examples because it already knows the "language" of the brain.
Standardization: Because it uses the same model for everything, scientists can finally compare results across different hospitals and countries without worrying about "camera settings."

In a nutshell:
Before, every scientist had to build their own custom brain-reading machine from scratch, leading to a mess of incompatible tools. NeuroSTORM is the first "universal brain engine." It learns the general rules of how the brain works from a massive library of data, and then scientists can easily plug in specific tasks to diagnose diseases or study the mind, all with higher accuracy and less computing power.

It's like moving from a world where every driver had to build their own car engine to a world where we all drive the same reliable, high-performance vehicle, just changing the destination based on where we need to go.

1. Problem Statement

Functional MRI (fMRI) analysis faces significant challenges in reproducibility and transferability due to:

Fragmented Pipelines: Existing methods rely on diverse preprocessing steps and task-specific model designs, leading to inconsistent results.
Dimensionality and Redundancy: Raw 4D fMRI data contains up to $10^6$ voxels per scan. Traditional approaches often project data onto pre-defined brain atlases (Region-of-Interest or ROI-based), causing irreversible information loss and structural bias.
Ineffective Self-Supervision: Standard Masked Autoencoders (MAEs) struggle with fMRI data because the high spatiotemporal redundancy allows masked voxels to be trivially reconstructed from neighbors, failing to learn meaningful representations.
Data Scarcity: Clinical datasets often have limited labeled data, making it difficult to train robust models without extensive fine-tuning resources.

2. Methodology: NeuroSTORM

The authors propose NeuroSTORM (Neuroimaging Foundation Model with Spatial-Temporal Optimized and Representation Modeling), a general-purpose foundation model designed to learn directly from raw 4D fMRI volumes.

A. Data Corpus

Scale: Pre-trained on 28.65 million fMRI frames from over 50,000 subjects.
Sources: UK Biobank (40,842), ABCD (9,449), and HCP datasets (HCP-YA, HCP-A, HCP-D).
Diversity: Spans ages 5–100, multiple centers, and diverse acquisition protocols.
Preprocessing: All data is aligned to MNI152 space (2mm resolution) and standardized to $96 \times 96 \times 96 \times T$ volumes. No labels are used during pre-training.

B. Architecture Innovations

Shifted-Window Mamba (SWM) Backbone:
- Replaces the standard Transformer backbone to handle high-dimensional 4D inputs efficiently.
- Combines Linear-time State-Space Models (SSM/Mamba) with Shifted-Window mechanisms.
- Benefit: Reduces computational complexity from quadratic (in Transformers) to linear, significantly lowering GPU memory usage while capturing long-range spatiotemporal dependencies.
Spatiotemporal Redundancy Dropout (STRD):
- A novel masking strategy designed to address the inherent redundancy in fMRI signals.
- Instead of random masking, it calculates spatial and temporal matching probabilities. It selectively drops voxels that can be trivially reconstructed from neighbors.
- Goal: Forces the model to learn complex, long-range spatiotemporal relationships rather than relying on local correlations.
Task-specific Prompt Tuning (TPT):
- For downstream adaptation, the backbone parameters remain frozen.
- Only a small set of learnable prompt parameters (inserted at the input layer) are trained.
- Benefit: Enables rapid, parameter-efficient transfer to new tasks with minimal data requirements (<5% of parameters updated).

3. Key Contributions

First General-Purpose fMRI Foundation Model: NeuroSTORM is the first model to perform large-scale pre-training directly on raw 4D volumes without atlas-based dimensionality reduction.
Architectural Efficiency: The integration of Mamba and Shifted Windows solves the computational bottleneck of processing whole-brain 4D data, making it feasible to train on massive datasets.
Robust Pre-training Strategy: The STRD module effectively mitigates the "trivial reconstruction" problem in fMRI, leading to more robust feature representations.
Comprehensive Benchmark: The authors established a unified benchmark covering five distinct downstream tasks, demonstrating the model's versatility.
Open Source: The code, pre-trained weights, and benchmarking suite are publicly released to standardize fMRI analysis.

4. Experimental Results

NeuroSTORM was evaluated against state-of-the-art (SOTA) ROI-based (e.g., BrainGNN, Brain-JEPA) and volume-based (e.g., SwiFT, TFF) methods across five tasks:

Task	Performance Highlights
Demographic Prediction	Gender: 93.28% accuracy on HCP-YA (outperforming SwiFT at 92.66%). Age: MSE of 7.0 on ABCD (vs. 7.72 for SwiFT).
Phenotype Prediction	Achieved superior Pearson Correlation Coefficients (PCC) across mental, social, and cognitive domains. For Emotion Task Accuracy (P04), achieved 0.419 PCC vs. 0.373 for SwiFT.
Disease Diagnosis	Outperformed all baselines on 5 datasets (ADHD200, COBRE, HCP-EP, UCLA, MND). Notable: 86.50% accuracy on MND (vs. 84.76% for SwiFT) and 71.46% on ADHD200.
fMRI Re-identification	Achieved 93.1% Rank-1 Accuracy and 92.4% mAP on HCP-YA, surpassing SwiFT by +1.5% and +3.0% respectively, proving strong subject-specific signature retention.
State Classification	92.40% overall accuracy across 7 cognitive states (HCP-YA) and 90.0% accuracy in medication state classification (DMT-HAR-MED).

Label Efficiency:

NeuroSTORM demonstrated exceptional robustness in data-scarce scenarios.
When trained with only 30% of the labels, it retained ~83-88% of its full-data performance, whereas baselines (SwiFT, BrainLM) dropped by 5–6% or more.
On the COBRE dataset, using 30% of labels, NeuroSTORM (62.4% accuracy) nearly matched SwiFT trained on 100% of data (64.5%).

Computational Efficiency:

During fine-tuning, NeuroSTORM required 7.46 GB of GPU memory per device, compared to 19.01 GB for SwiFT, despite a modest 14.6% reduction in training speed. This allows for processing longer temporal sequences.

5. Significance and Impact

Standardization: NeuroSTORM provides a unified framework that eliminates the need for task-specific architectural redesigns, promoting reproducibility across neuroimaging studies.
Clinical Applicability: Its ability to generalize across diverse clinical cohorts (schizophrenia, ALS, ADHD) and maintain performance with limited labeled data makes it highly suitable for real-world clinical deployment.
Scalability: By processing raw 4D volumes, the model preserves fine-grained anatomical and functional information that is lost in ROI-based approaches, potentially revealing new biomarkers.
Future Directions: The framework serves as a foundation for multimodal integration (e.g., combining with DTI or EEG) and offers a pathway toward personalized brain modeling.

In conclusion, NeuroSTORM represents a paradigm shift in fMRI analysis, moving from fragmented, task-specific models to a scalable, general-purpose foundation model that is both computationally efficient and highly transferable.