Technical Summary: Extending Pretrained 10-Second ECG Foundation Models to Longer Horizons

Problem Statement

Electrocardiogram (ECG) foundation models have demonstrated strong transferability across clinical applications when pretrained on standard diagnostic 10-second segments. However, a critical limitation exists: these models are trained as fixed-length "snapshot" processors. Real-world clinical scenarios, such as continuous monitoring, Holter recordings, and intensive care, often involve variable-length recordings that extend far beyond 10 seconds.

Directly applying 10-second pretrained models to longer horizons presents two fundamental challenges:

1. Structural Incompatibility: The positional encodings, tokenization schemes, and input assumptions of snapshot models are optimized for short segments. Naive interpolation of positional embeddings (common in vision transformers) is unsuitable for biosignals, as it conflates resolution with sampling rate and physiological information.
2. Semantic Limitations: Existing approaches for long recordings typically rely on sliding-window inference followed by simple aggregation (e.g., mean pooling) or lightweight sequential layers (e.g., LSTM) over window representations. These methods fail to explicitly model long-range temporal dependencies, often missing sparse or transient events distributed unevenly over time.

Methodology

The authors propose a parameter-efficient framework that extends frozen 10-second ECG foundation models to longer and variable-length inputs without retraining the backbone. The approach leverages the pretrained model's knowledge while introducing a lightweight plug-in module guided by three core insights: (1) preservation of general ECG interpretation knowledge, (2) locality-aware representation learning, and (3) dynamics-aware long-term learning.

The framework consists of two complementary extensions:

1. Structural Extension

To enable compatibility with long-horizon recordings, the authors introduce a two-level positional encoding scheme rather than interpolating existing embeddings:

• Local Component: Reuses the original pretrained positional embeddings ($E^{[10s]}$) for the fine-grained structure within each 10-second window. These weights are repeated for longer sequences to preserve the temporal scale of local waveform morphology (e.g., P-QRS-T complexes).
• Global Component: Introduces a learnable embedding matrix ($E^{global}$) to model relationships between consecutive 10-second segments. The final positional embedding is constructed by summing the local and global components.
• Soft Prompt Tuning: Additional learnable tokens are prepended to the input sequence and inserted at intermediate layers to facilitate the learning of global temporal relationships with minimal parameter updates.

2. Semantic Extension

To ensure meaningful temporal aggregation, the framework employs a teacher-student framework where the frozen 10-second model acts as a teacher to guide the adaptation of the extended model:

• Locality-Aware Representation Learning: A soft contrastive loss enforces semantic alignment between the student (extended model) and the teacher (frozen backbone) at the segment level. This preserves the teacher-induced similarity structure of local 10-second segments without imposing explicit temporal pairing assumptions.
• Dynamics-Aware Consistency Learning: Recognizing that physiological signals are non-stationary, the method uses the teacher's affinity matrix to identify high-affinity segments (those most consistent with the global recording). A consistency loss encourages these high-affinity segments to remain aligned with the global representation of the entire long sequence, ensuring the model captures global dynamics even when dynamic segments are excluded.

The total training objective combines the standard task loss with these two auxiliary losses:
$$L = L_{task} + \lambda_1 L_{local} + \lambda_2 L_{dyn}$$

Key Contributions

• Parameter-Efficient Extension: A framework that extends snapshot foundation models to variable-length inputs (up to 180s in experiments) without retraining the backbone, utilizing only a lightweight plug-in module.
• Novel Positional Encoding: A hierarchical positional encoding strategy that avoids interpolation artifacts by combining repeated local embeddings with learnable global segment embeddings, specifically tailored for biosignals.
• Semantic Guidance Mechanism: A teacher-student approach that leverages the pretrained model's robust local representations to guide long-horizon learning, addressing the semantic gap in temporal aggregation.
• Robustness Across Modalities: Demonstration that the framework generalizes beyond ECG to other physiological signals (e.g., PPG) and joint modalities.

Experimental Results

The method was evaluated on four downstream tasks (VT alarm reduction, ED disposition, triage acuity, and paroxysmal AF detection) across four foundation model backbones (CSFM-Tiny/Base, MERL, ECG-JEPA).

• Performance: The proposed method consistently outperformed snapshot aggregation baselines (Token/Logit Pooling) and lightweight adaptation methods (Linear Probing, Adapter, VPT, LSTM Tuning). It achieved state-of-the-art or competitive results across all backbones, particularly excelling in longer input durations (180s).
• Generalization: Models trained on 3-minute recordings generalized effectively to shorter inference durations (10s to 2 minutes) without retraining.
• Ablation Studies:
  • The proposed global positional encoding strategy significantly outperformed both repeated and interpolated positional encodings.
  • Combining locality-aware and dynamics-aware objectives yielded the best performance, confirming the necessity of both local semantic preservation and global dynamic modeling.
  • Guidance Source Analysis: When the MERL backbone (which showed weaker self-guidance) was paired with external guidance from stronger backbones (e.g., CSFM-Base), performance improved significantly, suggesting the quality of the teacher representation is crucial for semantic extension.
  • Affinity-Guided Dropping: Using affinity-based segment selection for consistency learning outperformed random dropping, validating the importance of similarity-based structure in the dynamics-aware objective.

Significance and Claims

The paper claims to provide a practical and scalable path for extending 10-second ECG foundation models to the longer horizons required in continuous monitoring and clinical decision-making. By avoiding computationally expensive full retraining, the method enables robust long-horizon adaptation while preserving the rich domain knowledge embedded in pretrained snapshot models.

The authors modestly note that while their method handles variable lengths up to 180 seconds effectively, extending snapshot models to arbitrary or substantially longer horizons under strict memory and computational constraints remains an open challenge for future work. The proposed framework represents a significant step toward bridging the gap between fixed-length pretraining and real-world variable-length clinical deployment.