💻 computer science

Video-based Heart Rate Estimation with Angle-guided ROI Optimization and Graph Signal Denoising

This paper proposes two plug-and-play modules, an Angle-guided ROI Adaptive Optimization module and a Multi-region Joint Graph Signal Denoising module, to significantly improve the accuracy of remote photoplethysmography (rPPG) heart rate estimation by effectively mitigating motion artifacts caused by facial movements such as speaking and head shaking.

Original authors: Gan Pei, Junhao Ning, Boqiu Shen, Yan Zhu, Menghan Hu

Published 2026-04-15

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Gan Pei, Junhao Ning, Boqiu Shen, Yan Zhu, Menghan Hu

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to listen to a friend's heartbeat by watching a video of their face. This is what Remote Photoplethysmography (rPPG) does. It uses a standard camera to spot tiny, almost invisible color changes in the skin caused by blood pumping through your veins.

However, there's a big problem: Motion.
If your friend turns their head, talks, or smiles, the camera gets confused. The "heartbeat signal" gets drowned out by "noise," like trying to hear a whisper in a hurricane. Most existing methods try to listen to the entire face as one big, messy signal, which often fails when the person moves.

This paper proposes a clever two-step solution to fix this, acting like a smart sound engineer for your video.

Step 1: The "Angle-Guided" Spotter (The Smart Camera Operator)

The Problem: Imagine you are filming a singer. If they turn their face away from the camera, the lighting changes, and the audio gets muddy. Similarly, if a face turns away, the skin color data becomes unreliable.

The Solution: The authors created a module that acts like a super-observant camera operator.

How it works: Instead of just staring at the whole face, the system breaks the face down into 60 tiny "windows" (like a grid of small tiles).
The Angle Check: It constantly calculates the angle of each tiny window relative to the camera. If a window on the cheek turns too far away (like a singer turning their back), the system knows, "Hey, this tile is blurry and unreliable right now!"
The Fix: Instead of using that bad data, it instantly swaps it with data from a neighboring tile that is still facing the camera clearly. It also detects if the whole head is spinning too fast (global motion) and marks those moments to be handled carefully.

Analogy: Think of it like a relay race. If one runner (a specific part of the face) trips or runs out of the lane, the system immediately passes the baton to the teammate right next to them who is still running smoothly, so the race (the heartbeat measurement) never stops.

Step 2: The "Graph Signal" Detective (The Noise-Canceling Team)

The Problem: Even after fixing the angles, the data is still a bit noisy. Some parts of the face might be twitching because the person is talking, while others are calm.

The Solution: The authors use a second module that acts like a team of detectives working together.

The Network: They treat those 60 tiny face windows as nodes in a social network.
The Rules: The system knows that the left cheek and right cheek are "best friends" (they usually move together), while the chin and forehead might be "distant cousins" (they move differently).
The Cleanup: The system looks at all 60 signals at once. If the forehead signal is screaming "I'm noisy!" but the cheeks are whispering "We're calm," the system trusts the calm ones and filters out the noise from the forehead. It uses a mathematical technique called Graph Signal Processing to smooth out the bumps, effectively canceling out the "static" caused by movement.

Analogy: Imagine you are in a crowded room trying to hear one person speak. If you listen to everyone at once, it's chaos. But if you have 60 friends standing around the room, and you ask them to compare what they hear, they can figure out who is actually speaking and ignore the background chatter. This module does exactly that for heartbeats.

The Result

When the researchers tested this "Angle-Guided Spotter" and "Graph Signal Detective" together:

They tested it on three different public datasets (like different groups of people moving around).
The Outcome: The system became 20% more accurate at guessing the heart rate compared to standard methods.
It works like a "plug-and-play" upgrade, meaning it can be added to existing heart-rate cameras to make them much smarter without needing a complete redesign.

In a nutshell: This paper teaches a computer how to ignore the bad angles and listen to the team rather than the individual, allowing it to measure your heart rate accurately even if you are talking, smiling, or moving your head.

1. Problem Statement

Remote photoplethysmography (rPPG) allows for non-contact heart rate (HR) estimation from facial videos by detecting subtle skin color changes caused by blood volume pulses. However, the performance of existing rPPG methods degrades significantly in real-world scenarios due to motion artifacts.

Limitations of Current Methods: Most existing approaches treat the face as a single holistic signal source or rely on deep learning models that require large datasets. They often fail to account for:
- Physiological Heterogeneity: Different facial regions (e.g., forehead vs. chin) exhibit varying signal qualities due to localized motion (speaking, expressions) and global motion (head rotation).
- Angle Sensitivity: The quality of the optical signal drops as the angle between the facial region and the camera deviates, leading to illumination changes and signal loss.
- Lack of Correlation Modeling: Existing methods often overlook the intrinsic correlations between signals from multiple facial regions, failing to leverage signal redundancy to suppress noise.

2. Methodology

The authors propose a framework integrating two plug-and-play modules compatible with reflection model-based rPPG methods (e.g., PBV, POS, OMIT). The pipeline consists of:

A. Angle-Guided ROI Adaptive Optimization Module

This module addresses global motion and signal degradation caused by head orientation.

ROI Selection: Using MediaPipe, 60 facial landmarks are selected (forehead, cheeks, chin) to construct 60 small rectangular ROIs (20x20 pixels).
Angle Quantification:
- The system estimates the 3D normal vector of each ROI plane using 3D landmarks and 2D corner points.
- It calculates the angular feature ( $C$ ) as the angle between the ROI normal and the camera vector.
Global Motion Detection:
- Angular signals from all 60 ROIs are fused via Principal Component Analysis (PCA) to generate a global angular signal.
- The first-order derivative of this signal yields the motion signal ( $S_{motion}$ ).
- Rapid head rotations (angular velocity > $30^\circ/s$ for > 0.5s) are labeled as global motion sequences.
Signal Refinement:
- During global motion, ROIs with orientation angles $> 60^\circ$ are deemed low-quality.
- Low-quality RGB features are replaced by those of the nearest high-quality neighbor (based on center-point proximity) to maintain signal continuity.

B. Multi-Region Joint Graph Signal Denoising Module

This module suppresses motion artifacts by jointly modeling the relationships between the 60 ROIs using Graph Signal Processing (GSP).

Graph Construction: An adjacency matrix ( $A$ $A$ ) is constructed for the 60 ROI nodes based on a weighted sum of three metrics:
1. Node Signal Quality ( $q_{ij}$ ): Based on the power spectral ratio in the HR frequency band and inter-frame stability. High-quality nodes within the same region receive higher weights.
2. Inter-ROI Regional Correlation ( $r_{ij}$ ): A static weight based on anatomical proximity (e.g., Cheek-Cheek = 1.0, Cheek-Forehead = 0.6, Forehead-Chin = 0.2).
3. Pairwise Signal Consistency ( $d_{ij}$ ): Dynamic synchronization measured via sliding-window Pearson correlation, down-weighted if nodes are low-quality.
Denoising via MAP Estimation:
- The problem is formulated as a Maximum A Posteriori (MAP) estimation to find a clean signal ( $S''_{bvp}$ ) from noisy observations ( $S'_{bvp}$ ).
- Objective Function: $\min \|S'_{bvp} - S''_{bvp}\|^2_2 + \lambda S''_{bvp}{}^\top L S''_{bvp}$ , where $L$ is the Graph Laplacian and $\lambda$ is a regularization parameter.
- Solution: The closed-form solution is $\hat{S}''_{bvp} = (I + \lambda L)^{-1}S'_{bvp}$ .
- Weight Optimization: The weighting coefficients ( $w_1, w_2, w_3$ ) for the adjacency matrix are learned adaptively using gradient descent to maximize the MAP objective.
Final Fusion: The denoised signals are weighted fused based on periodicity and quality, and segments affected by global motion are removed before final PCA fusion to derive the HR.

3. Key Contributions

Angle-Guided ROI Optimization: A novel method to quantify ROI-camera angles to detect global head motion and adaptively replace degraded signals with high-quality neighbors, preserving signal redundancy.
Multi-Region Joint Graph Denoising: A graph-based approach that explicitly models both intra-regional and inter-regional correlations to suppress motion artifacts, moving beyond single-signal aggregation.
Plug-and-Play Compatibility: The modules are designed to be integrated with existing reflection-model-based rPPG algorithms without requiring retraining of deep learning models.

4. Experimental Results

The method was validated on three public datasets: COHFACE, PURE, and MMPD, using Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) as metrics. The proposed modules were tested against three baseline rPPG methods: PBV, POS, and OMIT.

Performance Improvement: Jointly applying both modules resulted in a significant reduction in MAE compared to baselines.
- Average MAE Reduction: 20.38% decrease over the baseline across all datasets and methods.
- Specific Example (POS method on COHFACE): MAE dropped from 5.18 (Baseline) to 4.06 (Full Method).
- Specific Example (OMIT method on PURE): MAE dropped from 5.32 (Baseline) to 3.78 (Full Method).
Ablation Studies:
- Adding only the Angle-Guided module improved results over the baseline.
- Adding the Graph Denoising module on top of the Angle-Guided module provided further significant gains, confirming the complementary nature of the two modules.

5. Significance

This work demonstrates that explicitly modeling geometric relationships (angles) and signal correlations (graph topology) is crucial for robust rPPG in motion scenarios.

Robustness: It offers a solution for non-contact physiological monitoring in real-world settings where subjects move, speak, or turn their heads.
Generalizability: By being compatible with standard reflection models, it enhances the performance of existing rPPG pipelines without the computational overhead of training massive deep neural networks.
Clinical Potential: The improved accuracy in dynamic scenarios makes video-based HR estimation more viable for telemedicine, continuous health monitoring, and driver state assessment.