Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals

Technical Summary: Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals

1. Problem Statement

Cardiovascular diseases (CVDs) are the leading cause of global mortality, necessitating accurate and affordable pre-screening methods. While cardiac auscultation is a primary diagnostic tool, it suffers from low accuracy due to the subtle nature of heart sounds. Deep learning offers a solution, yet state-of-the-art (SOTA) architectures, particularly transformer-based models, remain underutilized in this domain. This limitation stems from the scarcity of large, synchronized, and multichannel datasets required to train data-hungry models effectively. Existing datasets often suffer from class imbalance, low signal-to-noise ratios (SNR), and limited size, hindering the robust classification of abnormal heart sounds using raw audio signals.

2. Methodology

The authors propose a scalable framework that combines traditional signal processing, synthetic data generation via diffusion models, and the fine-tuning of large pre-trained models to overcome data scarcity. The core of the approach involves fine-tuning a Wav2Vec 2.0 (BASE variant) encoder, originally designed for speech, for heart sound classification.

Data Augmentation and Synthetic Generation:
To address data limitations, the study employs a dual strategy:

• Traditional Augmentation: Techniques include harmonic-percussive source separation (HPSS), white noise addition, time stretching, amplitude modulation, baseline wander, parametric equalization, and clinical noise injection. For multichannel data, time stretching is synchronized across channels to maintain physiological consistency.
• Synthetic Signal Generation: The authors utilize WaveGrad and DiffWave diffusion models to generate synthetic Phonocardiogram (PCG) signals.
  • For single-channel PCG, ECG signals from the Icentia dataset serve as conditioners (via mel-spectrograms and disease labels) to generate 3,200 synthetic patient records.
  • For multichannel PCG (mPCG), the diffusion models are adapted to condition on specific channels to generate synthetic data for other channels, utilizing data from the CinC 2016 training-a and training-b datasets.

Model Architecture and Training:
The study evaluates three model configurations:

1. Single-Input: Classifies PCG or ECG signals independently.
2. Multimodal: Concatenates features extracted from synchronized PCG and ECG signals.
3. Multichannel: Processes mPCG data from a wearable vest (up to 6 channels).

All models utilize the Wav2Vec 2.0 encoder to extract features. For the multichannel model, which faces significant data scarcity, the authors employ Low-Rank Adaptation (LoRA) for fine-tuning and replace the final Multi-Layer Perceptron (MLP) layers with a Support Vector Machine (SVM) to prevent overfitting.

Datasets:

• CinC 2016: A large single-channel PCG dataset (3,153 recordings).
• CinC 2016 Training-a: A subset containing synchronized PCG and ECG signals (405 recordings).
• Wearable Vest Dataset: A real-world mPCG dataset with 157 subjects (96 CAD, 61 normal) recorded with 7 sensors in a hospital setting with background noise and free breathing.

3. Key Contributions

The paper outlines four primary contributions:

• Scalable Architecture: A transformer-based architecture capable of handling any number of PCG channels and ECG inputs.
• Multichannel Diffusion Model: The development of a diffusion model specifically for generating synthetic multichannel PCG signals.
• Tailored Augmentation: Introduction of augmentation techniques specifically designed for multichannel PCG data, ensuring temporal synchronization across channels.
• Performance Benchmarking: Achievement of SOTA performance on the CinC 2016 dataset and near-SOTA results on the challenging multichannel vest dataset.

4. Results

The proposed method demonstrates significant performance improvements across all dataset types, particularly when augmented data is utilized.

• Single-Channel (CinC 2016): On the subject-level, the augmented model achieved 92.48% accuracy, 93.05% Unweighted Average Recall (UAR), 93.63% sensitivity, 92.48% specificity, and an MCC of 0.8283. This outperforms existing CNN and RNN-based SOTA methods, particularly in balancing sensitivity and specificity.
• Multimodal (CinC 2016 Training-a): Without augmentation, the transformer model collapsed (UAR ≈51%). With augmentation and resampling to 4.125kHz, the model achieved 93.14% accuracy, 92.21% UAR, 94.35% sensitivity, 90.10% specificity, and an MCC of 0.8380.
• Multichannel (Vest Dataset): Despite the noisy, real-world nature of the data, the augmented model achieved 77.13% accuracy, 74.25% UAR, 86.47% sensitivity, 62.04% specificity, and an MCC of 0.5082. This represents a substantial improvement over the non-augmented baseline (9.8% accuracy increase) and demonstrates robustness in free-breathing conditions where previous methods required breath-holding.

Interpretability:
Attention maps and GradCAM++ visualizations confirm that the models focus on clinically relevant features: abnormal subjects show emphasis on diastolic and systolic phases (murmur-prone), while normal subjects emphasize S1 and S2 heart sounds.

5. Significance and Claims

The paper claims that the integration of diffusion-based synthetic data and traditional augmentation effectively bridges the data gap required for training transformer-based models on biosignals. The authors assert that this approach enables the deployment of SOTA performance on smaller, noisier, and multimodal datasets where traditional deep learning approaches fail or require massive data volumes.

Specifically, the study highlights:

• The ability to scale from single-channel to multimodal and multichannel inputs without architectural overhaul.
• The effectiveness of Wav2Vec 2.0 as a feature extractor for cardiac signals when supported by robust augmentation strategies.
• The practical viability of the method for real-world deployment, as evidenced by its performance on the noisy, free-breathing vest dataset, which better reflects clinical conditions than controlled breath-holding datasets.

The authors conclude that while the method is effective, future work is needed to improve the multichannel diffusion model for simultaneous channel generation and to further optimize augmentation strategies for specific diseases.