🤖 machine learning

Accelerometry-Derived Digital Biomarkers for Cardiometabolic Risk: A Population-Representative Tabular Benchmark with Uncertainty Quantification

This paper introduces the NHANES Accelerometry Cardiometabolic Benchmark, a population-representative dataset derived from 2003-2006 NHANES data, to evaluate tabular learning models and uncertainty quantification methods for predicting cardiometabolic risk markers while highlighting challenges in achieving fair subgroup coverage.

Original authors: Federico Felizzi

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Federico Felizzi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict how healthy a person's heart and metabolism are, just by looking at a list of numbers: how much they moved, what they ate, their age, and their weight. This is the challenge of "tabular data" in medicine.

However, most computer programs trained to do this have been tested on "toy" datasets that don't look like real life. They miss important details like how people were selected for a study, whether certain groups were over-represented, or if the predictions are fair for everyone.

This paper introduces a new, realistic "test drive" for these computer programs, using real data from a massive US health survey called NHANES. Here is a breakdown of what they did and found, using simple analogies.

1. The New "Driving Test" (The Benchmark)

Think of existing health data benchmarks as a driving test done in an empty parking lot with perfect weather. It's too easy and doesn't prepare drivers for the real world.

The authors built a new test track called the NHANES Accelerometry Cardiometabolic Benchmark.

The Car: They used data from 1,381 adults who wore hip-mounted pedometers (accelerometers) for a week.
The Route: The data includes not just steps, but also blood tests (like sugar and inflammation levels), diet logs, and body measurements.
The Rules: Crucially, this test track includes the messy realities of real life: complex sampling methods (ensuring the group represents the whole country) and strict rules about fairness across different races and genders.

2. The Three Drivers (The Models)

The researchers put three different types of "drivers" (computer algorithms) through this test to see who could best predict three specific health markers:

HbA1c: A measure of long-term blood sugar (diabetes risk).
Triglycerides: A type of fat in the blood.
CRP: A marker for inflammation in the body.

The three drivers were:

Ridge Regression: The "Old Reliable." A simple, straight-line logic that is easy to understand but might miss complex patterns.
XGBoost: The "Seasoned Veteran." A powerful, complex tree-based algorithm that is currently the standard for medical predictions.
TabPFN v2: The "Super-Intelligent Apprentice." This is a new type of "foundation model." Imagine a student who has read millions of fictional textbooks about how data works before ever seeing a real patient. Instead of learning from scratch, it uses its vast pre-existing knowledge to guess the answer immediately.

3. The Race Results

Who won?
The Super-Intelligent Apprentice (TabPFN v2) won the race overall. It made the most accurate predictions for blood sugar and inflammation, beating both the Old Reliable and the Seasoned Veteran.

Why? Because the dataset was relatively small (about 800 people for training), the Apprentice's pre-existing knowledge acted like a strong safety net, helping it avoid overfitting (memorizing the noise instead of learning the signal).

The "Unpredictable" Outcome:
When it came to Triglycerides (blood fat), all three drivers failed miserably. Their predictions were basically random guesses.

The Metaphor: Trying to predict triglycerides based only on a week of movement and one day of diet is like trying to guess someone's lottery numbers based on the weather. The paper explains that triglycerides are mostly driven by genetics and what you ate in the last few hours, neither of which was captured well by the data used here.

4. The Safety Net (Uncertainty Quantification)

In medicine, it's not enough to just make a guess; you need to know how confident you are. The researchers added a "Safety Net" called Conformal Prediction.

How it works: Instead of just saying "Your risk is 5%," the model says, "Your risk is between 4% and 6%, and we are 90% sure the truth is in that range."
The Goal: They wanted this safety net to catch the truth 90% of the time.

Did the Safety Net work?

On average: Yes! For blood sugar and inflammation, the safety net caught the truth almost exactly 90% of the time.
The Catch (Fairness): When they looked closer at specific groups, the net had holes.
- For Mexican American participants, the safety net failed to catch the truth for blood sugar predictions (it only caught it about 72% of the time).
- For Triglycerides, the net failed for everyone because the predictions were so bad to begin with.

The Lesson: Just because a safety net works for the average person doesn't mean it works for everyone. A system can be "fair" on paper but still leave specific groups exposed.

5. The Bottom Line

This paper didn't just build a new dataset; it showed that:

New AI models (TabPFN v2) are already powerful enough to beat traditional methods on small, real-world health datasets without needing extra training.
Not everything is predictable. You can't predict blood fat levels just by knowing how much someone moved or ate for a day; genetics play a huge role.
Fairness is tricky. Even when a model looks good overall, it might fail specific groups. We need better tools to ensure the "safety net" works for everyone, not just the average.

The authors made all their code and data public, essentially handing the keys to this new "driving test" to other researchers so they can build better, fairer health tools in the future.

Technical Summary: Accelerometry-Derived Digital Biomarkers for Cardiometabolic Risk

Problem Statement
Structured tabular data is the primary format for clinical decision-making, yet existing benchmarks for tabular learning fail to capture critical properties of real-world health data. Current evaluations often rely on generic datasets that lack complex survey sampling designs, demographic oversampling of minority groups, clinically validated outcomes, and requirements for equitable performance across subgroups. Furthermore, translating raw wearable sensor data into clinically actionable predictions requires models capable of handling heterogeneous features, missing values, and providing reliable uncertainty estimates for clinical deployment.

Methodology
The authors introduce the NHANES Accelerometry Cardiometabolic Benchmark, derived from the National Health and Nutrition Examination Survey (NHANES) 2003–2006.

Dataset: The analytic sample consists of 1,381 adults (ages 20–85) with valid hip-worn ActiGraph accelerometry data (≥4 wear days), fasting laboratory biomarkers, dietary intake (24-hour recall), and anthropometric measurements.
Features: Inputs include six activity phenotypes (e.g., total activity counts, sedentary time, moderate-to-vigorous physical activity) and twelve demographic/clinical covariates (e.g., age, sex, race/ethnicity, BMI, smoking, energy intake).
Outcomes: Three log-transformed cardiometabolic markers: glycated haemoglobin (HbA1c), fasting triglycerides, and C-reactive protein (CRP).
Models Evaluated: Three methods spanning the specialized-to-general spectrum of tabular learning (per Jiang et al., 2025):
1. Ridge Regression: A linear baseline.
2. XGBoost: A gradient-boosted tree ensemble representing the current state of practice.
3. TabPFN v2: A tabular foundation model that utilizes in-context learning on a pre-trained prior (derived from millions of synthetic datasets) to perform approximate Bayesian inference without task-specific fine-tuning.
Experimental Setup: The data was split into training (60%), calibration (20%), and test (20%) sets using stratified random splitting.
Uncertainty Quantification: Split conformal prediction was applied to all models to generate distribution-free 90% prediction intervals. The study specifically evaluated whether these intervals maintained equitable coverage across sex and race/ethnicity subgroups.

Key Contributions

Benchmark Introduction: The creation of a population-representative health tabular benchmark incorporating survey design weights, demographic oversampling, clinically validated fasting outcomes, and a two-wave temporal structure.
Rigorous Model Comparison: A systematic evaluation of specialized classical models versus general foundation models on three distinct cardiometabolic outcomes.
Uncertainty Quantification: The application of split conformal prediction to generate finite-sample coverage guarantees for all models.
Fairness Analysis: An assessment of whether conformal coverage holds equitably across demographic subgroups, directly addressing gaps in current fairness evaluations.

Results

Predictive Performance: TabPFN v2 achieved the best overall performance across all three outcomes without task-specific training.
- CRP: TabPFN v2 achieved an $R^2$ of 0.383, outperforming Ridge (0.339) and XGBoost (0.313). The authors attribute this to the model's ability to capture non-linear interactions via its in-context Bayesian prior.
- HbA1c: TabPFN v2 achieved an $R^2$ of 0.156, slightly outperforming Ridge (0.133) and XGBoost (0.139).
- Triglycerides: All models performed poorly ( $R^2 < 0.05$ ), with some negative values, reflecting the known dominance of genetic factors and acute dietary intake not captured by the feature set.
Conformal Coverage:
- Marginal coverage was close to the 90% target for CRP and HbA1c (e.g., Ridge achieved 0.910 for HbA1c).
- Coverage was lower for triglycerides (e.g., 0.819 for TabPFN v2), consistent with the outcome's low predictability.
Subgroup Analysis: While coverage was generally consistent for CRP, localized undercoverage was observed for specific groups. Notably, Mexican American participants showed a coverage of only 0.719 for HbA1c, despite other groups meeting or exceeding the 90% target. This highlights a gap between marginal guarantees and the conditional coverage required for clinical fairness.

Significance and Claims
The paper claims that TabPFN v2 serves as a strong baseline for small, clinically structured health datasets ( $N < 10,000$ ), outperforming task-specific models by leveraging a broad prior learned from synthetic data. This suggests that foundation models may be particularly well-suited for health applications where data collection is expensive and sample sizes are limited.

However, the authors maintain a modest tone regarding their findings:

The performance margins, while consistent with prior work on generic benchmarks, are small and based on a single data split; they should be viewed as indicative rather than statistically definitive rankings.
The study demonstrates that marginal conformal prediction is insufficient to certify demographic fairness, as it can mask undercoverage in specific subgroups.
The benchmark is limited by the specific NHANES 2003–2006 protocol (hip-worn, uniaxial sensors), the lack of genetic data, and the single 24-hour dietary recall, meaning results may not generalize to modern wrist-worn triaxial sensors or populations with different genetic/dietary profiles.

The work provides a reproducible foundation for future comparisons of tabular learning methods on clinically structured health data, emphasizing the need for group-conditional calibration to address fairness gaps.