Fusion hidden Markov modeling reveals a dominant backbone state and transient alternatives in simultaneous resting-state EEG-fMRI

This study introduces a robust fusion hidden Markov modeling framework for simultaneous resting-state EEG-fMRI data, revealing a dominant, persistent brain state alongside two transient alternative states that exhibit distinct hemodynamic and electrophysiological characteristics.

The Gist

Imagine your brain at rest isn't actually "resting" in the sense of being a still pond. Instead, it's more like a bustling city that never truly sleeps. Even when you are just sitting with your eyes open, daydreaming, the traffic patterns, the lights, and the activity in different neighborhoods are constantly shifting, rising, and falling.

For a long time, scientists have tried to take a snapshot of this city using two different cameras:

1. fMRI (The Slow Camera): This takes pictures of blood flow. It's great at showing where things are happening in the whole city, but it's slow. It's like watching a time-lapse video of traffic; you see the general flow, but you miss the individual cars.
2. EEG (The Fast Camera): This listens to the electrical sparks of neurons. It's incredibly fast, capturing the split-second decisions of the brain, but it's hard to tell exactly where in the city those sparks are coming from.

The Problem:
Trying to watch these two cameras at the same time is like trying to sync a slow-motion drone shot with a high-speed dashcam. The timing is off, the data is messy, and it's hard to build a single story that explains what's happening in both views simultaneously.

The Solution (The Fusion Model):
Georgina Cruz, the author of this paper, built a new "translation tool" (a fusion framework) to sync these two cameras perfectly. She didn't just look at the average of the data; she used a clever mathematical trick called a Hidden Markov Model (HMM).

Think of the HMM as a weather forecaster for the brain. Instead of saying "it's cloudy today," it says, "The brain is currently in a 'Sunny' state, but it might switch to a 'Rainy' state in a few seconds." The goal was to find out: How many distinct "weather patterns" does the resting brain actually have?

The Discovery: Three Brain "Weather Patterns"

After analyzing data from 12 healthy people, the model revealed that the brain doesn't wander randomly. Instead, it cycles through just three main states:

1. The "Backbone" State (The Sunny Day)

• What it is: This is the brain's "default mode." It happens the most often (about 80-90% of the time).
• The Analogy: Imagine a well-oiled machine or a city during rush hour where everything is flowing smoothly. The traffic lights are synchronized, the power grid is stable, and the major highways (brain networks) are all working together efficiently.
• The Science: In this state, the brain shows the clearest, most organized patterns of activity. It's the "home base" the brain returns to constantly.

2. The "Dimmed" State (The Overcast Day)

• What it is: A temporary state where the brain's activity is generally quieter.
• The Analogy: Think of this as the city lights being turned down a bit. The traffic is still moving, but it's slower, and the connections between neighborhoods feel a bit weaker. It's not a different city; it's just the same city running on "low power" or "economy mode."
• The Science: The brain networks are still there, but the signals are "attenuated" (weakened). It looks like a faded version of the Sunny Day.

3. The "Reorganized" State (The Stormy/Choppy Day)

• What it is: A temporary state where the brain changes its priorities.
• The Analogy: Imagine a sudden shift in the city. The main highway is closed, so traffic reroutes to side streets. The "emotional" district (limbic system) gets a lot of attention, while the "planning" district (default mode) takes a break. It's not just quieter; it's different.
• The Science: This state shows a selective reshuffling. The brain strengthens some connections (like emotional processing) while weakening others (like high-level planning).

The Big Surprise: The "Sync" Mystery

The most fascinating part of the study is how the two cameras (EEG and fMRI) behaved in these states.

• In the "Sunny" (Backbone) State: The slow blood flow (fMRI) was very organized, but the fast electrical sparks (EEG) seemed a bit "muted" or hard to pin down in relation to the blood flow. It's like the city is running smoothly, but the radio broadcast (EEG) is staticky or hard to hear clearly against the background hum.
• In the "Dimmed" and "Reorganized" States: Suddenly, the radio and the traffic cameras synced up perfectly! The electrical sparks and the blood flow moved in a very tight, coordinated dance.

The Takeaway:
This suggests that the brain isn't just one thing. It has a stable backbone that keeps us grounded, but it occasionally dips into transient states where the electrical and chemical signals align in very specific, interesting ways.

Why This Matters

Before this study, trying to combine these two cameras was like trying to mix oil and water. This paper provides a recipe for how to mix them properly. It shows that even when we are "doing nothing," our brains are actually cycling through a simple, predictable, and beautiful choreography of three main states.

It's like realizing that a jazz musician isn't just playing random notes; they are improvising within a strict, repeating structure of three main themes. Understanding this structure helps us see how the brain works when it's healthy, and it gives us a better baseline to spot what goes wrong in conditions like Alzheimer's or epilepsy.

Technical Summary

1. Problem Statement

Simultaneous EEG-fMRI offers a unique opportunity to study brain dynamics by combining the high temporal resolution of EEG with the whole-brain spatial coverage of fMRI. However, integrating these two modalities into a unified whole-brain model faces significant challenges:

• Temporal Misalignment: EEG and fMRI operate on vastly different time scales, and EEG preprocessing often alters the effective time axis, making direct alignment difficult.
• Feature Space Construction: Creating a stable, shared feature space that accurately represents both modalities without introducing artifacts or losing dimensionality is technically demanding.
• Model Selection: Determining the optimal number of latent brain states is prone to overfitting or instability, especially when relying on single-modality predictors rather than joint observations.
• Interpretability: Many existing approaches treat one modality as a predictor of the other rather than modeling them as joint observations of a shared latent brain state, limiting biological interpretability.

The study aims to develop a robust fusion modeling framework to identify recurring, low-order whole-brain states in resting-state data that are consistent across both EEG and fMRI modalities.

2. Methodology

Dataset and Preprocessing

• Data Source: 15 eyes-open resting-state runs from 12 healthy adults using an open-access simultaneous EEG-fMRI dataset (Telesford et al.).
• EEG Processing:
  • Used preprocessed data with gradient and ballistocardiogram artifacts removed.
  • Applied conservative Independent Component Analysis (ICA) rejection (removing only high-confidence non-neural components).
  • Performed manual exclusion marking for MR artifacts and bad intervals.
  • Source Localization: Used Brainstorm with a 3-layer OpenMEEG boundary element model to project sensor data to a 3mm isotropic grid in MNI152 space.
  • Parcellation: Mapped source activity to the Schaefer 2018 atlas (200 parcels, 7 networks). Extracted the first principal component (PC1) of each parcel's covariance as the time series.
• fMRI Processing:
  • Used fMRIPrep derivatives in MNI space.
  • Applied nuisance regression (motion, WM, CSF, aCompCor, cosine drifts) and spike regression for high-motion volumes.
  • Extracted BOLD time series using the same Schaefer atlas, summarizing voxel residuals via PC1.
• Multimodal Alignment:
  • Implemented a timestamp-based alignment workflow to reconcile preprocessed EEG timelines with raw TRs.
  • TR Retention Rules: A TR was retained only if it contained $\ge$70% usable EEG coverage (or $\ge$50% in a single contiguous block) and met a minimum sample count (50 samples).
  • Feature Construction: Concatenated 200 BOLD PC1 features and 200 EEG power features (mean squared gain-normalized PC1) per TR, creating a 400-dimensional observation vector.
  • Filtering: Only contiguous segments of $\ge$15 TRs were retained, resulting in 3,550 TRs (124.25 min) of usable data.

Modeling and Selection

• Model Type: Fusion Hidden Markov Model (HMM) using the osl-dynamics library.
• Cross-Validation: Used Leave-One-Subject-Out (LOSO) cross-validation to select the number of states ($K$).
  • Tested $K = 2$ to $12$.
  • Selection Criteria: Minimized mean test free energy, applied the 1-SE rule (selecting the simplest model within one standard error of the optimum), and evaluated stability via state-signature matching (BOLD correlation structure) across folds.
• Final Fit: A final $K=3$ model was fitted to the full dataset.
  • Input features were reduced via modality-specific PCA (40 BOLD + 40 EEG components) to an 80-dimensional space.
  • Used full state covariance matrices and multi-seed fitting to avoid local optima.

3. Key Results

Model Selection

• Optimal State Count: The LOSO-CV sweep showed a sharp improvement in free energy from $K=2$ to $K=3$, followed by a plateau. While $K=12$ had the lowest numerical free energy, the improvement over $K=3$ was negligible relative to uncertainty.
• Stability: The $K=3$ solution demonstrated superior reproducibility (mean matched state-signature correlation of 0.855) compared to $K=5$ (0.736).
• Rejection of Complex Models: The $K=5$ model suffered from "state collapse," where one state dominated occupancy (93%) across folds, rendering the other states ghost states. $K=3$ showed balanced, subject-variable occupancy.

State Characterization ($K=3$)

The final model revealed three distinct states:

1. State 2 (Dominant Backbone):
   • Temporal: Highest fractional occupancy and longest dwell time. It acts as a persistent "attractor" state; transitions from S1 and S3 preferentially return to S2.
   • BOLD Structure: Exhibits the clearest canonical large-scale network organization (Visual, Somatomotor, Salience, Dorsal Attention, Default Mode).
   • Cross-Modal: Shows muted or near-neutral BOLD-EEG covariance structure.
2. State 1 (Transient Attenuated):
   • Temporal: Low occupancy, short dwell time.
   • BOLD Structure: A broadly attenuated version of the backbone state, showing reduced within-network connectivity across multiple systems.
   • Cross-Modal: Shows a broadly strengthened positive cross-modal structure (stronger alignment between EEG power and BOLD networks, particularly in Visual and Salience systems) relative to S2.
3. State 3 (Transient Selectively Reweighted):
   • Temporal: Low occupancy, short dwell time.
   • BOLD Structure: A selectively reweighted alternative. It shows relatively stronger Limbic organization but weaker Default Mode and Control network structure compared to S2.
   • Cross-Modal: Shows a selective redistribution of cross-modal alignment (positive in Visual/Somatomotor, negative/weak in Dorsal Attention).

4. Key Contributions

• Methodological Framework: Developed a rigorous, reproducible pipeline for simultaneous EEG-fMRI fusion that addresses critical alignment issues (timestamp reconciliation) and feature space stability (atlas-aligned parcellation).
• Robust Model Selection: Demonstrated the necessity of using LOSO-CV combined with stability analysis (state signature matching) rather than relying solely on free energy metrics, effectively preventing overfitting and state collapse.
• Biological Insight: Identified a hierarchical dynamic architecture in resting-state brain activity:
  • A stable, high-occupancy backbone state (S2) representing canonical network organization.
  • Two transient alternatives (S1, S3) that represent either a global attenuation of this backbone or a specific reweighting of network contributions.
• Cross-Modal Dynamics: Revealed that the relationship between electrophysiology (EEG) and hemodynamics (fMRI) is state-dependent. The dominant state (S2) shows muted same-TR cross-modal structure, while transient states (S1, S3) exhibit stronger, more structured BOLD-EEG alignment, suggesting that multimodal observability fluctuates over time.

5. Significance

This study provides a practical workflow for whole-brain EEG-fMRI fusion that moves beyond simple correlation analysis. By establishing that resting-state dynamics are organized into a small set of recurring, interpretable states, it challenges the view of the resting brain as a single stationary pattern.

The findings suggest that the "backbone" of resting brain activity is a stable, canonical network configuration (S2), while transient states represent moments of altered network cohesion or specific reorganization. Crucially, the study highlights that multimodal alignment is not constant; the coupling between EEG and fMRI signals varies depending on the latent brain state, offering new hypotheses about how electrophysiological and hemodynamic processes interact dynamically. This approach sets a precedent for future studies in clinical populations and task-based paradigms, emphasizing the need for reproducibility-aware model selection in multimodal neuroimaging.