FHRFormer: A Self-Supervised Masked Transformer Framework for Fetal Heart Rate Time-Series Inpainting and Forecasting

Technical Summary: FHRFormer

Problem Statement
Fetal Heart Rate (FHR) monitoring is critical for assessing fetal well-being and detecting distress during labor. While wearable FHR monitors (e.g., the Moyo device) have improved accessibility in low-resource settings by enabling continuous monitoring without restricting maternal mobility, they are prone to signal dropouts. These gaps arise from sensor displacement during movement or changes in fetal/maternal position. Such missing data compromises clinical interpretation and hinders the reliability of Artificial Intelligence (AI) systems designed to predict neonatal outcomes. Traditional imputation methods, such as linear or spline interpolation, fail to preserve the complex spectral characteristics and temporal coherence of FHR signals, particularly during prolonged dropouts. Furthermore, existing deep learning architectures like RNNs and CNNs often struggle to capture the long-range temporal dependencies and non-stationary nature of FHR dynamics required for accurate reconstruction and forecasting.

Methodology
The authors propose FHRFormer, a self-supervised masked Transformer-based autoencoder framework designed for FHR signal inpainting and forecasting.

• Architecture: The model utilizes an encoder-decoder structure based on Transformers. The input FHR signal (normalized to a fixed length of 7,200 time steps, representing one hour) is divided into non-overlapping patches.
  • Encoder: Processes visible (unmasked) patches through five Transformer blocks using multi-head self-attention (MHSA) to capture global temporal and spectral dependencies.
  • Decoder: Reconstructs the full signal by processing both the encoded visible patches and learnable mask tokens representing missing segments. It employs bidirectional attention to reason over the entire temporal context.
• Training Strategy: The model is trained using a self-supervised masked autoencoding approach. A random binary mask is applied to a subset of patches (target masking ratio $\gamma$), and the model is tasked with reconstructing the original signal values for these masked regions.
• Loss Function: To ensure both temporal accuracy and physiological fidelity, the training objective combines two losses:
  1. Reconstruction Loss ($L_{recon}$): Mean Squared Error (MSE) on the masked patches.
  2. Focal Frequency Loss ($L_{freq}$): A frequency-domain loss that penalizes discrepancies in the Discrete Fourier Transform (DFT) magnitude spectra between the original and reconstructed signals. This ensures the preservation of critical spectral features like baseline variability and deceleration patterns.
     The total loss is a weighted sum: $L_{total} = 0.95 \cdot L_{recon} + 0.05 \cdot L_{freq}$.
• Data: The model was trained on 4,486 FHR episodes from the "Safer Births" project in Tanzania, collected via the Laerdal Moyo monitor. The data was preprocessed to remove Doppler noise and normalized.

Key Contributions

1. Transformer-Based Reconstruction: Introduction of a masked Transformer autoencoder specifically tailored for FHR signal reconstruction, capable of modeling multi-scale temporal dependencies better than RNNs or CNNs.
2. Self-Supervised Inpainting Strategy: A method to recover missing FHR segments without requiring paired labeled data, leveraging synthetic masking on unlabeled physiological data.
3. Forecasting Pipeline: An extension of the trained model to perform progressive forecasting of future FHR values, enhancing its utility for real-time risk assessment.
4. Frequency-Aware Learning: The integration of focal frequency loss to maintain spectral fidelity, addressing a key limitation of time-domain-only reconstruction methods.

Results
The model was evaluated on validation and test sets using multiple metrics, including Reconstruction Loss (RL), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Fréchet Inception Distance (FID), MSE, RMSE, and Correlation Coefficient (CC).

• Patch Size Sensitivity: Experiments with patch sizes ranging from 30 to 480 samples revealed that smaller patches yielded superior performance. The configuration with a patch size of 30 (Hybrid-30) achieved the best results, with a test PSNR of 41.38 dB, SSIM of 0.9984, and CC of 0.9993. Larger patches resulted in a loss of local morphological resolution.
• Masking Ratio: Performance peaked at a masking ratio of 15%. Beyond this threshold, the density of missing data degraded the structural context available for reconstruction.
• Inpainting: Visual comparisons demonstrated that FHRFormer effectively reconstructed missing segments, preserving transient decelerations and baseline stability better than linear interpolation.
• Forecasting: In a progressive forecasting task (predicting 15 seconds ahead based on 30 minutes of history), FHRFormer outperformed the TimeGPT baseline, successfully tracking complex physiological trajectories without succumbing to error accumulation or steady-state assumptions.

Significance and Claims
The paper positions FHRFormer as a transformative tool for next-generation AI-enabled FHR monitoring, particularly for wearable devices in resource-limited settings. By effectively handling signal dropouts, the framework ensures the continuity and quality of data required for robust risk stratification algorithms. The authors claim that the method can be applied retrospectively to research datasets to improve AI algorithm development. Furthermore, they suggest that integrating this approach into wearable devices could enable earlier and more robust detection of fetal distress by maintaining data continuity even during transient signal loss. The work bridges the gap between traditional signal processing limitations and modern deep learning capabilities, specifically addressing the non-stationary and spectral complexities of FHR signals.