(Un)fair devices: Moving beyond AI accuracy in personal sensing

This literature review highlights the hidden biases in AI-driven personal sensing devices across various demographics and advocates for shifting from purely performance-based evaluations to human-centered approaches, providing guidelines to ensure the development of fair and unbiased health technologies.

The Gist

Imagine your smartwatch, fitness ring, or health app as a personal detective. This detective is always watching, listening, and measuring your every move to tell you how healthy you are, how well you slept, or if your heart is skipping a beat.

For years, we've asked this detective one question: "Are you accurate?" If the detective says you took 10,000 steps, we want to know if it was actually 10,000.

But this paper argues that asking only about accuracy is like asking a chef, "Does this soup taste good?" without asking, "Does it taste good to everyone?"

The authors, a team of researchers from universities and tech giants, are saying: "No, our personal devices are not fair to everyone." They are often biased, meaning they work great for some people but fail miserably for others.

Here is the breakdown of the problem and the solution, using some everyday analogies.

1. The Problem: The "One-Size-Fits-None" Detective

The core issue is that the "training data" used to teach these devices is skewed. Think of it like a cooking school where 90% of the students are tall, white, young men from wealthy countries.

• The Result: The school teaches the students how to cook for tall, white, young men.
• The Reality: When a short woman, an elderly person, or someone with a different skin tone tries to eat the food, it tastes wrong.

The paper highlights several real-world examples of this "bad cooking":

• The Skin Tone Blindness: Pulse oximeters (the clip that goes on your finger to check oxygen) often give wrong readings for people with darker skin. It's like a camera with a flash that is calibrated for pale faces; it overexposes dark skin, making it look like there's no oxygen when there is. This can be dangerous during medical emergencies.
• The Body Type Blindness: Heart rate sensors on smartwatches often struggle with people who have larger arms or more muscle mass. It's like trying to measure a river's depth with a ruler meant for a bathtub; the tool just doesn't fit the context.
• The Age Blindness: Voice assistants and hearing aids often fail to understand children or the elderly because they were trained mostly on the voices of middle-aged adults. It's like a translator who only speaks "Gen Z slang" and can't understand a grandparent's dialect.

2. The Hidden Danger: The "Invisible Glitch"

In traditional software, if a program crashes, you see a big red error message. But in personal devices, the bias is invisible.

Imagine a GPS navigation app. If the GPS is biased, it doesn't just say "Error." Instead, it quietly sends you down a bumpy, dangerous road because it thinks you prefer bumpy roads, simply because it has only ever driven people who live on bumpy roads.

The paper warns that these devices are now making high-stakes decisions:

• Diagnosing heart conditions (Atrial Fibrillation).
• Predicting fertility.
• Detecting sleep apnea.

If the "detective" is biased, it might tell a Black patient they are fine when they are actually in danger, or tell a woman her fertility chances are lower than they really are. The paper found that in the last few years, less than 10% of research papers on these devices even bothered to check if their tools were fair to different groups of people.

3. Why Is This So Hard to Fix?

The authors explain that personal device data is tricky.

• It's a moving target: Unlike a static photo, a heart rate signal changes every second. It's like trying to judge a movie by looking at a single, blurry frame.
• It's messy: A high heart rate could mean you are running, or it could mean you are stressed, or you just drank coffee. The device has to guess the cause, and if it guesses wrong for certain groups, that's bias.
• The "WEIRD" Trap: Most research is done on Western, Educated, Industrialized, Rich, and Democratic populations. It's like testing a new car only on smooth highways in California, then selling it to people who drive on muddy roads in the rain.

4. The Solution: "Fairness by Design"

The authors propose a new rulebook called "Fairness by Design." Instead of building the device first and checking for bias later (like checking for cracks in a bridge after it's built), we must build fairness into the foundation.

They offer 14 guidelines for researchers and companies, which can be summarized as:

• Recruit a Diverse Crew: Don't just test on your university friends. Test on people of all ages, sizes, skin tones, and backgrounds.
• Listen to the Critics: If a device works poorly for a specific group, don't just say "that's an outlier." Fix the model.
• Be Transparent: Tell users, "This device works best for people with X characteristics, and might be less accurate for Y."
• Keep Watching: Devices get old, sensors get dirty, and people change. A model that is fair today might be biased tomorrow. You have to keep checking it.

The Bottom Line

The paper concludes that accuracy is not enough. A device that is 99% accurate for 50% of the population is a failure if it is 50% accurate for the other 50%.

We need to move from asking "Is this device smart?" to asking "Is this device smart for me?"

By treating fairness as a core feature—just like battery life or water resistance—we can ensure that these digital companions actually help everyone, not just a lucky few.

Technical Summary

1. Problem Statement

The paper addresses the critical lack of fairness assessments in personal sensing devices (wearables, smart rings, smartphones) that rely on Machine Learning (ML) to interpret physiological and behavioral data. While these devices are increasingly used for high-stakes health applications (e.g., detecting Atrial Fibrillation, diagnosing sleep apnea, predicting fertility), their underlying algorithms often harbor hidden biases.

Key Issues Identified:

• Algorithmic Unfairness: ML models deployed on personal devices often exhibit performance disparities across demographic groups (race, gender, age, body mass index, etc.).
• Data Characteristics: Unlike standard tabular datasets used in general AI ethics research, personal device data is temporal, sequential, mutable, and longitudinal. This makes bias harder to detect because sensor signals (e.g., heart rate, accelerometer data) are indirect observations influenced by external factors (stress, alcohol) and physiological variations, rather than direct semantic labels.
• The "WEIRD" Bias: Research in mobile and ubiquitous computing heavily relies on Western, Educated, Industrialized, Rich, and Democratic (WEIRD) populations, leading to models that underperform for non-Western, older, or diverse body-type users.
• Evaluation Gap: A systematic review reveals that only a tiny fraction of research papers (approx. 9% in top venues) report fairness assessments or bias mitigation strategies, prioritizing raw accuracy over equitable performance.

2. Methodology

The authors conducted a systematic literature review following the Kitchenham and Charters protocol to synthesize the state of fairness in personal devices.

• Data Source: The primary search focused on the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), published between 2018 and 2025.
• Search Query: A comprehensive query combined terms for "mobile/wearable/ubiquitous computing," "machine learning," and "fairness/bias/non-discrimination."
• Screening Process:
  • Initial Retrieval: 957 papers were identified.
  • Eligibility Criteria: Papers were included only if they provided quantitative assessments of empirical contributions in mobile/wearable tech AND included quantitative assessments of bias or performance discrepancies regarding sensitive attributes (age, gender, race, disability, physiology, etc.).
  • Exclusion: Papers focusing solely on NLP, computer vision without ubiquitous components, or theoretical bias (e.g., bias-variance trade-off) were excluded.
• Final Corpus: 89 papers (approx. 9% of retrieved, 5% of total IMWUT publications) were included in the synthesis.
• Validation: The authors cross-referenced findings with other major venues (MobiCom, MobiSys, SenSys, IEEE Pervasive Computing) to ensure generalizability, finding similar low inclusion rates for fairness-focused work.

3. Key Contributions

The paper makes four primary contributions to the field of Ubiquitous Computing and AI Ethics:

1. Empirical Evidence of Bias: It documents specific instances of bias in personal devices, including:

   • Acoustic Sensing: Gender bias in respiratory monitoring (female voices have higher frequencies affecting enhancement); race bias in speech enhancement (hair texture affecting signal quality).
   • Radio-Frequency (RF) Sensing: Performance degradation in sleep apnea detection for severe cases vs. mild cases; gait recognition failures in children due to variable walking patterns.
   • Computer Vision & Sensors: Racial bias in pulse oximetry and PPG sensors; accuracy drops in dehydration detection for heavier individuals due to vein visibility.
   • Inertial Sensors: Age bias in gait detection (elderly users) and sex bias in neurological screening.

2. Analysis of the "Fairness Gap": The study highlights that while 67% of papers attempt to improve robustness via ablation studies or personalization, they rarely frame these efforts through the lens of group fairness (equitable outcomes across demographics). Most papers fail to report sensitive attributes (e.g., race is reported in only ~10% of papers; education in <20%).

3. Taxonomy of Mitigation Strategies: The authors categorize existing (though sparse) mitigation efforts into three stages:

   • Pre-processing: Data balancing, fair client selection in federated learning, and recruiting diverse cohorts.
   • In-processing: Multi-task learning (e.g., learning activity and gender simultaneously), disentangled representations to isolate noise, and causal modeling to account for confounders.
   • Post-processing: Bias-aware calibration (e.g., correcting oxygen saturation based on skin tone) and per-person model adaptation.

4. "Fairness by Design" Guidelines: The paper proposes 14 specific recommendations for researchers and developers to integrate fairness throughout the ML pipeline, moving beyond post-hoc fixes.

4. Key Results & Findings

• Prevalence of Bias: Bias is pervasive but often "blended" into the background of sensor data. For example, pulse oximeters trained on light-skinned populations significantly underestimate hypoxia in Black and Hispanic patients.
• Demographic Reporting Deficit:
  • Gender: Reported in most papers, but female representation is often low (mean 31% in reported samples).
  • Race: Reported in only ~10% of papers; when reported, samples are overwhelmingly White (89%).
  • Age: Reported in <40% of papers; the mean participant age is 34, despite the technology being used by an aging global population.
• Lab vs. Wild: Controlled "in-the-lab" studies (54% of papers) are less likely to identify biases compared to "in-the-wild" studies (29%), suggesting that real-world variability exposes fairness issues that lab environments conceal.
• Mitigation Rarity: Despite the presence of bias, 75% of the included papers did not report any fairness enhancement mechanism, prioritizing proof-of-concept and accuracy over equity.

5. Significance and Recommendations

The paper argues that the mobile and wearable computing community must shift from a performance-oriented evaluation paradigm to a human-centered approach.

The "Fairness by Design" Framework (14 Recommendations):
The authors provide a structured guide for integrating fairness:

• Data Collection: Recruit diverse samples (beyond legally protected attributes to include physiology like BMI/height); train annotators on implicit bias; report sensitive attributes transparently.
• Data Exploration: Validate data across groups to detect anomalies; investigate hardware-induced measurement errors (e.g., device fit on smaller wrists).
• Model Development: Utilize fairness toolkits (e.g., FairLearn, AIF360) adapted for time-series data; consider indirect fairness (e.g., excluding clients with poor connectivity in federated learning); use generative models to synthesize data for underrepresented groups.
• Deployment & Use: Monitor real-time performance for drift; reassess models post-deployment using unsupervised protocols; ensure transparency in model versioning to track how updates affect different user groups.

Conclusion:
The paper concludes that personal devices have the potential to revolutionize healthcare and lifestyle management, but only if they are built on principles of equity. Without proactive "Fairness by Design," these technologies risk exacerbating existing health disparities and marginalizing vulnerable populations. The authors call for a cultural shift in the research community to treat fairness as a core metric alongside accuracy.