⚡ electrical engineering

A comprehensive evaluation of pretraining strategies for channel-agnostic contrastive self-supervision of biosignals

This paper introduces Contrastive Random Lead Coding (CRLC), a channel-agnostic pretraining strategy that creates positive pairs using random channel subsets to effectively generalize across variable biosignal configurations, outperforming existing methods in EEG tasks and achieving comparable results in ECG tasks.

Original authors: Thea Brüsch, Mikkel N. Schmidt, Tommy S. Alstrøm

Published 2026-05-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Thea Brüsch, Mikkel N. Schmidt, Tommy S. Alstrøm

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize different types of music. In the world of computer vision (teaching robots to see), we usually teach them by showing the same picture twice but with slight changes—like cropping it, turning it sideways, or adding a little static. The robot learns that "this is still a cat," even if the picture looks a bit different.

But biosignals (like brain waves or heartbeats) are tricky. You can't just "rotate" a heartbeat or "crop" a brain wave without breaking the meaning. Also, medical devices are inconsistent: one hospital might use 6 sensors on a patient's head, another might use 20, and a third might use only 2. If you train a robot to recognize heartbeats using exactly 12 sensors, it might get confused when it sees a patient with only 6.

This paper is about teaching a robot to understand these messy, variable medical signals without getting confused by how many sensors are attached.

The Big Problem: The "Sensor Mismatch"

Think of a biosignal dataset like a choir.

The Challenge: Sometimes the choir has 4 singers (channels), sometimes 12, and sometimes 20.
The Old Way: Most AI models are like conductors who only know how to lead a choir of exactly 12 singers. If you bring a choir of 6, they have to pretend the missing 6 are there (silence). If you bring a choir of 24, they have to ignore the extra 12. This wastes information.
The Goal: The authors wanted to build a "universal conductor" (a channel-agnostic model) that can lead a choir of any size, learning the music itself rather than memorizing the number of singers.

The Solution: Three Ways to Make "Positive Pairs"

To teach the robot without human labels (self-supervised learning), the researchers use a game called Contrastive Learning. The robot is shown two versions of the same signal (a "positive pair") and told, "These are the same song, so they should look similar in my brain."

The paper tests three different ways to create these "same song" pairs:

The "Time-Split" Method (CSC):
- The Analogy: Imagine listening to a 10-second song. You cut it in half. You tell the robot, "The first 5 seconds and the second 5 seconds are the same song."
- The Assumption: The song doesn't change much over time.
The "Augmentation" Method (CAC):
- The Analogy: You take the song and add some static noise, change the volume, or shift it slightly. You tell the robot, "This noisy version is the same song as the clean version."
- The Assumption: The song is the same even if it sounds a bit rough.
The "Random Lead" Method (CRLC) - The Star of the Show:
- The Analogy: Imagine a choir with 20 singers. You split them into two groups: Group A (singers 1–10) and Group B (singers 11–20). You tell the robot, "Even though Group A and Group B are singing different parts, they are singing the same song at the same time."
- The Assumption: All the sensors are recording the same underlying event (like a heart beating), so any subset of them should look similar to any other subset.

The Experiment: Synthetic and Real Life

The authors first tested this on fake data (simulated signals) where they knew the rules.

Result: If the fake data was designed so that the "Time-Split" method made sense, that method worked best. If the data was designed so that "Random Leads" made sense, that method won.
Lesson: You have to pick the right teaching method based on the nature of the data.

Then, they tested on real medical data:

EEG (Brain Waves): These are very messy. Different hospitals use different numbers of sensors.
- Result: The Random Lead (CRLC) method was the clear winner. It beat the current "state-of-the-art" models (like BENDR).
- Why it worked: By forcing the robot to learn that "Sensors 1–5" and "Sensors 6–10" are singing the same song, the robot learned the core rhythm of the brain activity, ignoring the specific sensor arrangement. It could then easily handle a new patient with 2 sensors or 20 sensors.
ECG (Heartbeats): These are usually recorded with a standard 12 sensors.
- Result: A giant, complex model (the "State-of-the-Art" reference) still won overall, likely because it was huge and trained on a massive amount of data. However, among the smaller, flexible models the authors built, the Random Lead (CRLC) method was still the best.
- Nuance: The authors noted that for heartbeats, the "Time-Split" method (CSC) sometimes worked well too, but CRLC was generally more robust.

The Secret Sauce: The "Message Passing" Network

How did they make the robot handle different numbers of sensors?
They used a special component called a Message Passing Neural Network (MPNN).

The Analogy: Imagine a round table where everyone (each sensor) can talk to everyone else. Even if 2 people leave the table or 5 new people join, the conversation continues because everyone is just sharing what they know with their neighbors.
This allowed the model to learn from 6 sensors, and then immediately work on a dataset with 20 sensors without needing to be retrained or having "fake" sensors added.

The Bottom Line

The paper concludes that Contrastive Random Lead Coding (CRLC) is the best strategy for teaching AI to understand biosignals when the number of sensors varies.

For Brain Waves (EEG): It is a huge success, beating existing top models and allowing the AI to work with any number of sensors.
For Heartbeats (ECG): It is the best of the flexible models, though a massive, rigid model still holds the top spot overall.

The key takeaway is simple: To teach an AI to understand biological signals across different machines, don't just show it the same signal twice; show it different groups of sensors recording the same event at the same time. This teaches the AI to listen to the music, not the instruments.

Technical Summary: A Comprehensive Evaluation of Pretraining Strategies for Channel-Agnostic Contrastive Self-Supervision of Biosignals

Problem Statement
Self-supervised learning (SSL), particularly contrastive learning, has achieved significant success in computer vision and natural language processing by leveraging unlabeled data to learn generalizable representations. However, applying these methods to multivariate biomedical time series, such as electroencephalography (EEG) and electrocardiography (ECG), presents unique challenges. First, designing effective data augmentations for time series is non-trivial and often requires specific domain knowledge. Second, and more critically, the number of input channels varies significantly across different applications and datasets (e.g., sleep studies using 2 channels vs. clinical setups using 20+ channels). Existing self-supervised models often rely on fixed channel configurations, requiring zero-padding or discarding of channels during fine-tuning, which limits their transferability and generalization capabilities across diverse channel setups.

Methodology
The authors propose a channel-agnostic framework that combines a single-channel convolutional encoder with a Message Passing Neural Network (MPNN). This architecture allows the model to process variable numbers of input channels without structural changes, effectively extracting inter-channel information during both pretraining and fine-tuning.

The core investigation focuses on identifying optimal strategies for creating positive pairs in contrastive learning for multivariate time series. The paper evaluates three distinct strategies:

Contrastive Random Lead Coding (CRLC): Creates positive pairs by splitting the input channels into two random, non-overlapping subsets within the same time window. This assumes that different channels recorded simultaneously share common underlying content.
Contrastive Segment Coding (CSC): Creates positive pairs by splitting a single time window into two sequential segments. This assumes stationarity of the signal content across time within the window.
Contrastive Augmented Coding (CAC): Creates positive pairs by applying various data augmentations (e.g., scaling, masking, noise addition, time shifting) to the same input window.

The models are pretrained using two contrastive loss functions: NT-Xent and TS2Vec. The evaluation is conducted in two phases:

Synthetic Data: A custom simulator generates multivariate time series based on linear mixing of sources. Two settings are created: one where channels share information (favoring CRLC) and one where information is stationary over time but distinct across channels (favoring CSC).
Biomedical Data: The framework is tested on real-world EEG (Physionet Challenge 2018, SleepEDFx, MASS) and ECG (Physionet Challenge 2021) datasets. The models are pretrained on large unlabeled datasets and fine-tuned on downstream classification tasks (sleep staging for EEG, arrhythmia classification for ECG) with varying numbers of input channels.

Key Contributions

Introduction of CRLC: A novel strategy for generating positive pairs by randomly sampling subsets of input channels, specifically designed for multivariate time series where channels share common sources.
Synthetic Benchmark: The creation of a synthetic dataset designed to rigorously test the assumptions underlying different pretraining strategies (channel-sharing vs. temporal stationarity).
Channel-Agnostic Architecture: A demonstration of an MPNN-based architecture that enables seamless transfer between datasets with different channel configurations without zero-padding or discarding data.
Comprehensive Evaluation: A thorough comparison of pretraining strategies (CRLC, CSC, CAC) and loss functions (NT-Xent, TS2Vec) across two distinct biomedical domains.

Results

Synthetic Data: The study confirms that the alignment between the pretraining strategy and the data generation process is critical. Pretraining with a strategy mismatched to the data characteristics (e.g., using CSC on non-stationary, channel-sharing data) yields representations only marginally better than random.
EEG Performance: The CRLC strategy consistently outperformed CSC and CAC across all sample sizes and loss functions. Notably, the CRLC-pretrained models surpassed the state-of-the-art BENDR reference model on both the SleepEDFx (2 channels) and MASS (20 channels) datasets. This highlights the model's ability to generalize from a 6-channel pretraining setup to downstream tasks with significantly different channel counts.
ECG Performance: While the proposed MPNN-based models generally underperformed a larger, state-of-the-art reference model (W2V+CMSC+RLM) which utilizes a fixed 12-channel input and masking, the CRLC strategy remained the superior approach among the tested contrastive strategies when combined with the TS2Vec loss. The reference model's superiority is attributed to its larger parameter count and specific masking-based pretraining, though it lacks the channel-agnostic flexibility of the proposed method.
Ablation Studies: The number of message passing rounds ( $K$ ) had a minimal impact on EEG performance but showed slight benefits for ECG with larger sample sizes, suggesting that the choice of positive pair formation and loss function is more impactful than the aggregation depth.

Significance and Claims
The paper claims that for channel-agnostic self-supervised learning of biosignals, the method of forming positive pairs is a decisive factor in model performance. Specifically, Contrastive Random Lead Coding (CRLC) is identified as the most effective strategy for EEG data, enabling models to learn representations that generalize across variable channel setups where traditional fixed-channel models fail.

The authors emphasize that while large foundation models with fixed inputs perform well when channel counts are stable, the proposed channel-agnostic approach is essential for scenarios where channel configurations vary significantly between pretraining and deployment. The work suggests that using groups of channels as positive views helps the model disentangle and extract the underlying common signals useful for classification. While the current MPNN-based models do not yet surpass the largest fixed-input reference models in ECG tasks, the CRLC strategy provides a robust foundation for future channel-agnostic models, particularly in domains like EEG where channel variability is the norm.

The Big Problem: The "Sensor Mismatch"

The Solution: Three Ways to Make "Positive Pairs"

The Experiment: Synthetic and Real Life

The Secret Sauce: The "Message Passing" Network

The Bottom Line

More like this