Google creates heart rate monitor using smartphone selfie cameras

Google Research has developed a system that can monitor your heart rate just by looking at your face through your phone's front camera. The technology works in the background during normal smartphone use, capturing brief video clips after you unlock your phone with face recognition.

The system, called PHRM (Passive Heart Rate Monitoring), can track both your current heart rate and your resting heart rate throughout the day. Most importantly, it's the first remote heart monitoring technology to work accurately for people of all skin tones - a major breakthrough in a field where previous systems performed poorly on darker skin.

Published in Nature journal, the research represents seven years of work and the largest study of its kind. Google tested the system with nearly 700 participants in both lab and real-world conditions, making it available to qualified researchers along with the dataset.

How does it work?

PHRM uses photoplethysmography (PPG), the same principle behind pulse oximeters and fitness trackers. It detects tiny changes in how light reflects off your skin as blood pulses through it with each heartbeat.

The system captures 8-second video clips of your face immediately after face unlock events. Deep learning algorithms then analyze these clips to estimate your heart rate along with a confidence score. Key technical features include:

• Temporal shift convolutional neural networks that work efficiently on phones
• Confidence scoring to filter out unreliable measurements
• Kalman filtering to combine multiple readings into daily resting heart rate estimates
• On-device processing that doesn't require internet connectivity

The system achieved a mean absolute percentage error of less than 10% compared to medical-grade electrocardiogram measurements. For daily resting heart rate, it matched fitness tracker accuracy with less than 5 beats per minute average error.

Why does it matter?

Heart rate monitoring could become accessible to billions more people without requiring additional hardware. Around five billion people already own smartphones, compared to much smaller numbers who use dedicated fitness trackers or medical devices.

This matters particularly for health equity. Previous remote heart monitoring systems worked poorly on darker skin tones due to melanin making the optical signals harder to detect. PHRM is the first to meet medical accuracy standards across all skin tones - a critical breakthrough given that cardiovascular disease disproportionately affects underserved communities.

The health implications are significant:

• Resting heart rate is a key indicator of cardiovascular health and mortality risk
• Higher resting heart rates and increases over time predict major heart problems
• Early detection could help prevent cardiovascular events
• Passive monitoring provides more comprehensive data than occasional manual checks

Google's validation study confirmed the system captures real health signals - participants with higher measured resting heart rates were more likely to have high BMI and poor cardiovascular fitness.

The context

This research addresses a major problem in digital health technology: most systems work poorly or not at all for people with darker skin. Pulse oximeters, for example, have been shown to give less accurate readings for Black patients, leading to potential medical errors.

Google specifically designed PHRM to avoid this bias. They ensured their dataset included at least 25% of participants with light skin, 25% with medium skin, and 33% with dark skin using the Monk Skin Tone scale. They also required that accuracy differences between skin tone groups stay within 5 percentage points.

The company tested PHRM against 15 other leading heart monitoring algorithms and found it was the only one to achieve medical-grade accuracy across all skin tones. This sets a new standard for the field and demonstrates how inclusive design from the start can create better technology for everyone.

The research builds on Google's previous work using smartphone cameras for on-demand heart rate measurement when users place their finger over the camera lens. The new system works passively without any special user action, making it much more practical for continuous health monitoring.

While the technology shows promise, challenges remain. Success rates were lower for people with darker skin, likely due to the inherent difficulty of detecting optical signals through melanin. Future versions could address this through better camera exposure optimization or taking multiple measurement attempts.