Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning

Problem Statement

The cyclic nature of physiological processes, particularly the cardiac cycle, offers a natural prior for self-supervised representation learning. However, standard self-supervised objectives for time-series data, such as autoregressive or reconstruction losses, often fail to exploit this structure effectively. These methods are compelled to reproduce raw sensor observations, including noise (motion, electrode contact, baseline wander), and do not inherently organize the latent space according to the periodicity of the signal. While Joint-Embedding Predictive Architectures (JEPAs) offer an alternative by predicting latent targets rather than raw samples, standard latent-prediction objectives leave the organization of cardiac phase unspecified. Consequently, models may encode phase without ensuring that latent trajectories are consistent across beats, failing to capture the specific symmetry of the heart's recurring depolarization and repolarization sequence.

Methodology: WINDER

The authors introduce WINDER, a phase-equivariant extension of the LeJEPA architecture designed to explicitly encode cardiac-phase symmetry.

1. Phase Reparameterization
Instead of treating the heartbeat as a function of linear time $t \in \mathbb{R}$, WINDER reparameterizes it as a function of phase $\phi \in S^1$ (a closed loop). Given detected R-peaks $\{R_i\}$, the phase map $\Phi: \mathbb{R} \to S^1$ is defined as:
$$\phi(t) := 2\pi \frac{t - R_i}{R_{i+1} - R_i}, \quad t \in [R_i, R_{i+1})$$
This maps the interval between consecutive R-peaks to the unit circle, preserving within-cycle ordering while normalizing beat duration.

2. Phase-Equivariant Latent Space
The core theoretical contribution is the enforcement of phase equivariance. The latent space $\mathcal{Z}$ is constrained to obey the same symmetry as the cardiac cycle. If a signal is translated in phase by $\Delta$ before encoding, the latent representation must undergo a corresponding transformation $R_\Delta$ after encoding:
$$f_\theta(T_\Delta x) = R_\Delta f_\theta(x)$$
The transport operator $R_\Delta$ is fixed and closed-form, derived from the geometry of the $U(1)$ group (isomorphic to $SO(2)$) rather than learned. The latent space is decomposed into:

• Invariant Capacity ($K_0$): Coordinates that remain fixed under phase rotation.
• Harmonic Capacity: Subspaces corresponding to integer harmonics $n$, where pairs of dimensions rotate by $n\Delta$.
  The transport operator is block-diagonal, with each harmonic block performing a rotation $\rho(n\Delta)$. This structure adds no learnable parameters to the transport mechanism.

3. Learning Objective
WINDER optimizes a joint objective combining three terms:
$$\mathcal{L}_{Winder} = \mathcal{L}_{pred} + \lambda_{SIG}\mathcal{L}_{SIG} + \lambda_{trans}\mathcal{L}_{trans}$$

• $\mathcal{L}_{pred}$ (Latent Prediction): A causal predictor forecasts the next latent token, inheriting from LeJEPA.
• $\mathcal{L}_{SIG}$ (Spectral Regularization): Uses the Sketched Isotropic Gaussian Regulariser (SIGReg) to prevent representation collapse by enforcing an isotropic Gaussian distribution on the latent embeddings.
• $\mathcal{L}_{trans}$ (Phase-Equivariant Transport): A novel loss term that penalizes deviations from phase equivariance. It measures the squared directional error between the transported current latent and the actual next latent:
  $$\mathcal{L}_{trans} = \mathbb{E}_{(t,t')} \left[ 1 - \langle R_{\Delta_{tt'}} \hat{z}_t, \hat{z}_{t'} \rangle \right]$$
  This encourages the latent trajectory to follow the cardiac phase order consistently.

4. Architecture
WINDER is implemented as a compact causal architecture for 12-lead ECGs (PTB-XL dataset).

• Encoder: Processes 80ms patches of 12-lead signals (100 Hz) through a patch MLP and two residual causal convolutional blocks.
• Projector: Maps encoder outputs to 256-dimensional latent tokens.
• Predictor: A causal Transformer (discarded at inference) used only during training to predict future tokens.
• Parameter Footprint: The deployed model (encoder + projector) contains approximately 1.2 million parameters.

Key Results

Evaluated on the PTB-XL dataset under a frozen linear-probe protocol on a sealed test fold (Fold 10):

• Diagnostic Accuracy: WINDER achieves a Superclass macro-AUROC of 0.8702, significantly outperforming the transport-free control (0.7826). This performance is comparable to state-of-the-art self-supervised methods that often have orders of magnitude more parameters.
• Mechanism Validity: The model passes a "G1 gate" test, confirming that the learned representations utilize the declared cardiac clock. The transport gain fraction is 0.8786 for WINDER versus -0.0744 for the control, indicating that the phase transport constraint is actively shaping the latent space.
• Latent Geometry:
  • Fundamental-harmonic Amplitude: WINDER exhibits a roughly 5-fold stronger phase-locked fundamental component (1.89 vs 0.38) compared to the control.
  • Orbit-subspace Coherence: WINDER achieves a coherence of 0.9927, indicating that phase-bin means lie almost entirely within the expected rotational plane, whereas the control collapses to 0.3159.
  • Visualization: UMAP visualizations show WINDER's tokens forming a phase-ordered annulus, while the control's tokens form a disordered cluster.
• Robustness: WINDER demonstrates superior robustness to single-lead dropout, with a mean degradation of -0.0034 compared to -0.0107 for the control.
• Anomaly Detection: In synthetic perturbation tests, WINDER shows strong detection capabilities for ectopic beats (AUROC 0.8662 vs 0.6198) with a median localization latency of 80ms (one token).

Significance and Claims

The paper claims that explicitly encoding cardiac-phase symmetry via a fixed, closed-form transport operator yields a latent geometry that is:

1. Legible and Parameter-Efficient: It achieves high diagnostic performance with a small parameter footprint (~1M) by injecting physiological structure analytically rather than learning it through scale.
2. Physiologically Grounded: The latent space is directly tied to a measurable physiological quantity (cardiac phase), ensuring that the representation respects the cyclic nature of the heart.
3. Diagnostic Utility: The phase-equivariant organization preserves diagnostically useful information, making the representation more linearly accessible for downstream tasks.

The authors emphasize that this work fills a gap in ECG representation learning by imposing phase equivariance on a learned latent space, a capability previously unoccupied in healthcare-specific models. They conclude that this approach opens a broader class of phase-equivariant world models for cyclic biosignals where physiological structure is injected analytically. The findings are presented as a controlled within-study comparison, establishing the benefits of the phase-transport mechanism in isolation, without claiming immediate superiority over all existing models on external datasets or in deployed clinical systems.