CardiacMamba: Fair and Robust RGB-RF Fusion for Remote Heart Rate Estimation via State Space Modeling
CardiacMamba is a fair and robust remote heart rate estimation framework that leverages state space modeling to fuse RGB and RF signals, achieving state-of-the-art performance on the EquiPleth dataset while significantly reducing skin-tone bias and maintaining resilience against illumination changes and sensor degradation.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Measuring a heartbeat without touching the skin has long been a dream of medical technology, promising a way to monitor health that is invisible and unobtrusive. For years, scientists have relied on cameras to capture this vital sign. By recording the subtle, rhythmic changes in skin color caused by blood pumping through the face, these camera-based systems can estimate heart rate. This method, known as remote photoplethysmography, works well in ideal conditions but struggles when the light changes, when a person moves, or when the camera views darker skin tones, where the optical signals are harder to detect. To solve this, researchers have begun looking at a different kind of sensor: radio frequency radar. This technology detects the tiny physical vibrations of the chest wall caused by the heart beating, a mechanical signal that does not care about lighting or skin color. However, radar alone lacks the fine detail of a camera and can be confused by body movement. The challenge has been to combine these two very different ways of seeing the heart into a single, reliable system that works for everyone, regardless of their appearance or the environment.
A team of researchers has now built a new system called CardiacMamba that successfully merges these two streams of information. Instead of simply stacking the data from a camera and a radar side by side, the team designed a framework that lets the two signals talk to each other in a sophisticated way. They created a structure that first isolates the most important parts of each signal: the camera focuses on the color shifts in the face, while the radar zeroes in on the minute movements of the chest. The system then uses a specialized modeling technique to align these two different types of data, ensuring that the timing of the color changes matches the timing of the physical vibrations. Finally, it refines the combined signal by filtering out noise in the frequency domain, a process that sharpens the final heart rate reading. This approach allows the system to use the strengths of the camera to fill in the gaps of the radar, and vice versa, creating a robust measurement that is far more stable than either method could achieve alone.
When tested on a dataset designed to evaluate fairness across different skin tones, the new system proved highly effective. It achieved an average error of just 0.96 beats per minute, a level of precision that surpassed all previous methods using cameras, radar, or a combination of both. More importantly, the system addressed a long-standing issue in this field: the tendency for camera-based heart rate monitors to be less accurate for people with darker skin. By relying on the radar's ability to detect mechanical motion, which is unaffected by skin pigmentation, the new framework reduced the performance gap between light and dark skin tones to a mere 0.26 beats per minute. This suggests that the system can provide equitable health monitoring for a diverse population, a critical step toward making non-contact vital sign monitoring truly universal.
The researchers also tested how well the system held up when one of the sensors was compromised or missing. In scenarios where the camera feed was degraded by noise or poor lighting, the system maintained its accuracy by leaning more heavily on the radar data. Conversely, when the radar signal was unavailable, the system could still function using only the camera, though with slightly reduced precision. This flexibility means the technology is resilient; it does not fail completely if one part of the setup is disturbed. The study confirms that by integrating optical and mechanical cues through this specific fusion method, it is possible to create a heart rate monitor that is both highly accurate and fair, overcoming the limitations that have previously hindered non-contact health tracking.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.