Technical Summary: Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning

Problem Statement

Biosignals acquired from different body locations often capture temporally ordered views of the same underlying physiological process. A canonical example is the relationship between Electrocardiography (ECG), which records the electrical activation initiating a heartbeat, and Photoplethysmography (PPG), which records the resulting peripheral pulse delayed by vascular dynamics. This phenomenon, termed asymmetric temporal observability, induces structured, directional relationships between modalities.

However, most existing self-supervised learning (SSL) methods for biosignals treat different modalities as interchangeable views. They typically enforce agreement through contrastive alignment or symmetric joint reconstruction, overlooking the directional temporal dynamics that link them. This assumption obscures the time-ordered nature of physiological sensing, where ECG precedes PPG, and the temporal delays are critical for understanding vascular dynamics and cardiovascular health.

Methodology: xMAE

To address this, the authors introduce xMAE, a biosignal pretraining framework grounded in the principle that when modalities are temporally ordered, representation learning must respect directional information flow.

Core Architecture and Objective

xMAE reframes masked autoencoding as a structured inference problem under asymmetric temporal observability:

1. Asymmetric Masking: The model receives fully observed PPG signals and partially observed (masked) ECG signals. Specifically, continuous temporal blocks covering a high percentage (80–90%) of the ECG signal are masked.
2. Directional Cross-Modal Reconstruction: The objective is to reconstruct the masked ECG segments using the fully visible PPG signal and the remaining visible ECG context.
3. Directional Cross-Attention: The architecture employs a cross-attention mechanism where the masked ECG tokens act as queries, and the visible PPG tokens act as keys and values. This forces the model to reason about the physiological dependency where PPG provides a delayed view of the electrical activity captured by ECG.
4. Curriculum Masking Strategy: Training begins with a masking ratio of 80% and progressively increases to 90% in steps as reconstruction loss improves. This curriculum prevents the model from relying on trivial local interpolation of the ECG signal in early stages, forcing it to learn cross-modal temporal transitions as the task difficulty increases.

Data and Training

• Pretraining: The model is pretrained on approximately 9.4k hours of paired ECG-PPG recordings from the MIMIC-III database (intensive care setting).
• Evaluation: The learned PPG encoder is transferred to 19 downstream tasks across 6 datasets (including laboratory and free-living settings) involving cardiovascular conditions, sleep staging, abnormal lab test detection, and demographic inference. Evaluation uses linear probing with subject-level cross-validation.

Key Results

The paper reports that xMAE yields representations that outperform both unimodal and multimodal baselines on 15 of 19 downstream tasks.

• Classification Performance: xMAE achieved statistically significant improvements in tasks such as hypertension detection (AUROC 68.8 vs. 56.8 for the next best), ectopic beat detection (87.8 vs. 86.2), and abnormal blood lab detection (A1C: 65.1 vs. 52.5).
• Regression Performance: The model demonstrated lower Mean Absolute Error (MAE) in blood pressure estimation and age prediction compared to unimodal PPG-only models and other multimodal baselines.
• Data Efficiency: Despite using significantly less pretraining data (9.4k hours) and fewer parameters (6.5M) compared to large-scale open-source foundation models (e.g., Pulse-PPG with 28.5M parameters and 55k hours of data), xMAE achieved competitive or superior performance.
• Physiological Grounding: Analysis of the learned representations revealed that xMAE better preserves the ECG-PPG timing structure. The median error in estimating the physiological delay (R-peak to PPG onset) was 21.5ms for xMAE, a 53.3% reduction compared to a symmetric multimodal MAE baseline. Furthermore, HRV features derived from ECG reconstructed by xMAE showed lower error than those derived from raw PPG, indicating the model encodes meaningful beat-to-beat timing dynamics.

Significance and Claims

The authors claim that xMAE demonstrates the effectiveness of incorporating temporal structure and directional inductive biases into multimodal pretraining.

• Beyond Waveform Fidelity: Unlike prior work that treats ECG generation from PPG as a signal synthesis task optimized for waveform realism, xMAE uses ECG reconstruction as a training-time scaffold. The goal is not to generate perfect ECG waveforms but to learn transferable temporal abstractions that reflect the underlying physiological process.
• Generalization: The framework shows that modeling cross-modal temporal transitions can be as important as increasing model scale or expanding pretraining data volume. The learned representations generalize across different devices, body locations, and acquisition settings.
• Broader Applicability: The authors posit that this approach is applicable to other paired biosignals that observe different stages of a shared underlying process, such as ECG-ballistocardiography or EMG-motion, offering a principled alternative to the "exchangeable view" assumptions common in current multimodal pretraining.

The paper concludes that by respecting the directional relationship between electrical and mechanical cardiovascular signals, xMAE supports more interpretable and physiologically grounded representation learning, which is critical for developing trustworthy foundation models for health applications.