Machine-learning for photoplethysmography analysis: Benchmarking feature, image, and signal-based approaches

This paper presents a comprehensive benchmarking study demonstrating that deep neural networks, particularly modern convolutional neural networks, operating on raw PPG waveforms outperform feature-based and image-based approaches for blood pressure estimation and atrial fibrillation detection.

The Gist

Imagine your smartwatch is a detective trying to solve two very different mysteries about your health: Is your heart beating irregularly (Atrial Fibrillation)? and What is your blood pressure right now?

To solve these mysteries, the detective (the computer) looks at a signal called PPG. Think of PPG as a tiny flashlight shining through your skin to watch your blood pulse like a wave. Every time your heart beats, a little wave of blood rushes through your finger or wrist, and the light changes.

This paper is a massive "cooking competition" where researchers tested three different ways to feed this heartbeat data to a computer brain (Machine Learning) to see which method makes the best predictions.

Here is the breakdown of the three "cooking styles" they tested:

1. The Raw Ingredient Approach (Raw Time Series)

The Analogy: Imagine you are a chef who is given a whole, uncut chicken, a bag of raw vegetables, and a raw egg. You don't chop them or mix them first; you just throw the whole raw mess into a high-tech blender that learns how to cook it perfectly on its own.

• How it works: The computer looks at the raw, unprocessed wave of light data, second by second. It doesn't try to understand why the wave looks a certain way; it just learns the pattern directly.
• The Result: This was the winner. The "Deep Neural Networks" (super-smart computers) that ate the raw data were the best chefs. They found patterns humans might miss. Specifically, a type of architecture called CNN (Convolutional Neural Network) was the star player. It's like a chef with a super-powerful knife that can slice through complex patterns instantly.

2. The Pre-Prepared Meal Approach (Feature-Based)

The Analogy: Imagine you are a chef who is given a pre-chopped onion, a pre-measured cup of salt, and a pre-cracked egg. You didn't have to do the hard work of measuring or chopping; you just mixed these specific ingredients together.

• How it works: Before feeding the data to the computer, human experts used math to extract specific "clues" (features). For blood pressure, they measured the height of the wave. For irregular heartbeats, they measured how chaotic the time between beats was.
• The Result: This approach was okay, but not the best. It's like cooking with pre-chopped veggies: it's easier and faster, but you lose some of the subtle flavors (nuances) that the raw data had. In fact, for blood pressure, this method sometimes performed worse than just guessing the average value!

3. The Picture Approach (Image-Based)

The Analogy: Imagine instead of giving the chef the raw chicken or the chopped veggies, you take a photo of the chicken, the veggies, and the egg, and then feed the photo to the chef. The chef then uses their "image recognition" skills to figure out how to cook it.

• How it works: The researchers turned the heartbeat wave into a picture (a spectrogram or a wavelet map). It looks like a colorful heat map or a topographic map of the heartbeat. They then used image-recognition software (the same kind that recognizes cats in photos) to analyze these pictures.
• The Result: This was a strong runner-up. It did surprisingly well, almost as good as the raw data approach. It's like taking a photo of a crime scene; sometimes seeing the whole picture helps you solve the mystery even if you didn't look at the raw evidence.

---

The Big Takeaways

1. The "Deep Learning" Chef Wins
When the researchers tested these methods on two big datasets (one from surgery patients for blood pressure, and one from people wearing watches for heart rhythm), the Raw Time Series approach using Deep Neural Networks consistently won.

• Why? Because the computer is smart enough to learn the recipe itself. It doesn't need humans to tell it what ingredients to measure. It can find the secret sauce in the raw data that humans might overlook.

2. Bigger isn't always better, but "Deep" helps
They found that very deep, complex computer brains (like XResNet) worked best when they could "memorize" specific patients (like in the surgery data). However, when trying to guess the health of a new person they had never seen before, slightly smaller, simpler models were actually quite competitive.

• Metaphor: A master chef who knows your family's specific taste (Calibration) is amazing. But a good, generalist chef (Generalization) is still better than a chef who only knows how to use pre-chopped veggies.

3. The "Black Box" Problem
The winning method (Raw Data + Deep Learning) is a "Black Box."

• The Catch: We know it works great, but we don't always know why. It's like a magic trick where the magician pulls a rabbit out of a hat, and we just have to trust the hat.
• The "Pre-Prepared Meal" method (Feature-Based) is transparent. We know exactly what ingredients (clues) were used. Doctors often prefer transparency, but in this race, the "Black Box" was the faster and more accurate runner.

The Bottom Line for You

If you are building a smart health app or a medical device:

• Don't waste time trying to manually measure every little detail of the heartbeat wave yourself.
• Feed the raw data directly into a modern Deep Learning model (specifically a CNN).
• It's the most reliable way to predict blood pressure and detect irregular heartbeats right now.

The study essentially says: "Stop trying to be the chef; let the computer learn to cook the raw ingredients itself."

Technical Summary

1. Problem Statement

Photoplethysmography (PPG) is a widely used, non-invasive sensing technique in both clinical and consumer wearable devices (e.g., smartwatches). While machine learning (ML) is increasingly applied to PPG data for tasks like Blood Pressure (BP) estimation and Atrial Fibrillation (AF) detection, the field lacks systematic, controlled comparisons.

Current research often suffers from:

• Incomparable Benchmarks: Studies typically compare models within a single input modality (e.g., only raw signals or only features) or against literature results with different experimental setups, making direct performance comparison impossible.
• Unclear Best Practices: It remains unclear whether deep learning on raw signals, hand-crafted clinical features, or image-based representations (time-frequency transforms) yields superior performance across different tasks (regression vs. classification) and generalization scenarios (seen vs. unseen patients).

2. Methodology

The authors conducted a comprehensive, "like-for-like" benchmarking study using identical data splits and evaluation protocols across two distinct clinical tasks and three input representation paradigms.

Datasets

• BP Estimation (Regression): Used the VitalDB subset of the PulseDB dataset (intraoperative PPG and invasive arterial blood pressure).
  • Scenario A (Calib): Training and test sets share subjects (allows subject-specific adaptation/memorization).
  • Scenario B (CalibFree): Training and test sets have disjoint subjects (tests generalization to unseen patients).
• AF Detection (Classification): Used the DeepBeat dataset (wrist-worn PPG). The authors created a new, balanced data split to correct for the original dataset's subject overlap and class imbalance issues.

Input Representations & Models

The study evaluated three categories of input representations:

1. Raw Time Series (T): Direct processing of 1D PPG waveforms.
   • Models: LeNet1d, Inception1d, XResNet1d (50/101), AlexNet1d, MiniRocket, TCN, PPNet (CNN-LSTM), iTransformer, TimesNet.
2. Feature-Based (F): Clinically interpretable features or mathematical transforms fed into traditional ML models.
   • Features: Pulse morphology (peaks, duration), derivative features (arterial stiffness indices), irregularity metrics (for AF), and Wavelet Packet Decomposition coefficients.
   • Models: Multi-Layer Perceptron (MLP), Gaussian Process Regression (GPR).
3. Image-Based (I): 1D signals converted to 2D time-frequency images.
   • Transforms: Continuous Wavelet Transform (CWT) scalograms and Short-Time Fourier Transform (STFT) spectrograms.
   • Models: ResNet-18 and ResNet-50 (initialized with ImageNet weights).

Evaluation Metrics

• Regression (BP): Mean Absolute Error (MAE), Mean Absolute Scaled Error (MASE), Bland-Altman analysis (Bias, Limits of Agreement), and IEEE 1708a-2019 grading (A-D).
• Classification (AF): AUC, F1-score, Sensitivity, Specificity, and Matthews Correlation Coefficient (MCC).
• Statistical Significance: Bootstrapping (1,000 iterations) to determine if performance differences were statistically significant.

3. Key Contributions

• First Controlled Benchmark: Provides the first direct comparison of raw signal, feature-based, and image-based approaches for PPG analysis across both regression and classification tasks using identical data splits.
• Task-Dependent Insights: Demonstrates that the "best" approach is highly dependent on the specific task (BP vs. AF) and the generalization scenario (Calib vs. CalibFree).
• Model Architecture Analysis: Evaluates a wide range of architectures, including modern CNNs, Transformers, and hybrid models, against simple baselines.
• Open Source: The complete code and preprocessing scripts are made publicly available to facilitate reproducibility.

4. Key Results

Blood Pressure Estimation (Regression)

• Best Performers: Deep Convolutional Neural Networks (CNNs) operating on raw time series achieved the best results. Specifically, XResNet1d50+GNLL (using Gaussian Negative Log Likelihood loss and dropout ensembling) performed best.
• Calib vs. CalibFree:
  • In the Calib scenario (seen patients), large, complex models (e.g., XResNet1d101) excelled, likely due to their ability to memorize subject-specific patterns.
  • In the CalibFree scenario (unseen patients), smaller models (e.g., LeNet1d, MiniRocket) performed comparably to or better than complex models, suggesting that over-parameterization leads to overfitting when generalizing to new subjects.
• Feature/Image Performance: Feature-based models generally underperformed raw signal models, particularly in the Calib scenario. Image-based models (CWT/STFT) were competitive but generally lagged behind raw signal CNNs.
• Performance Limits: Even the best models only achieved IEEE Grade A (error ≤5 mmHg) for ~48% of SBP and ~64% of DBP estimates, with significant variability (large Limits of Agreement) at the sample level.

Atrial Fibrillation Detection (Classification)

• Best Performers: Modern CNNs (Inception1d and XResNet1d50) operating on raw time series achieved the highest AUC (0.85) and F1-scores (0.69).
• Image vs. Raw: Image-based models (Spectrogram-ResNet) performed competitively with top raw signal CNNs, suggesting time-frequency representations are viable for AF detection.
• Feature Limitations: Feature-based models (especially those using clinically interpretable features like irregularity metrics) performed significantly worse than deep learning models. Wavelet-based features performed better than clinical features but still lagged behind raw signal CNNs.
• Transformer Performance: Transformer-based models (iTransformer, TimesNet) and hybrid models (PPNet) did not outperform standard CNNs, with some (iTransformer) performing poorly.

5. Significance and Conclusions

• Recommendation for Practitioners: For robust, generalizable PPG applications, modern CNN architectures (specifically ResNet or Inception variants) operating on raw time series are the safest and most effective choice. While image-based methods are competitive, they add preprocessing complexity without guaranteed gains.
• Complexity vs. Generalization: Model complexity should be matched to the task. Large models are beneficial when subject-specific adaptation is possible (Calib), but smaller, shallower models are often superior for generalization to unseen populations (CalibFree).
• Interpretability Trade-off: While feature-based models offer inherent clinical interpretability, they sacrifice predictive performance. The study highlights a fundamental trade-off: deep learning on raw data achieves superior accuracy but requires post-hoc explainability (XAI) techniques to bridge the gap with clinical understanding.
• Future Directions: The authors note that current models still struggle with sample-level reliability (high variance). Future work should focus on uncertainty estimation, out-of-distribution detection, and self-supervised pretraining (foundation models) to improve performance, particularly in real-world wearable scenarios with motion artifacts.

In summary, this paper establishes that while deep learning on raw signals is currently the state-of-the-art for PPG analysis, the choice of model architecture must be carefully tuned to the specific generalization requirements of the clinical application.