Evaluating Open-Source Wrist-Worn Accelerometer Models for Sedentary Time Detection Against Thigh-Worn Accelerometer Data

Technical Summary: Evaluating Open-Source Wrist-Worn Accelerometer Models for Sedentary Time Detection

Problem Statement
Wrist-worn accelerometers are widely adopted in large-scale epidemiological studies and consumer settings due to their scalability and cost-effectiveness. However, unlike thigh-worn devices, which can directly infer posture (sitting/lying vs. standing) and serve as a "silver standard" for sedentary behavior, wrist-worn devices typically infer sedentary time based on low movement intensity. This limitation raises uncertainty regarding the accuracy of wrist-derived sedentary estimates in free-living environments. While previous studies have suggested broad equivalence in mean daily sedentary time between wrist and thigh monitors, these findings often rely on small samples. Furthermore, the construct validity of wrist-based models regarding detailed sedentary characteristics—such as the frequency of prolonged sedentary bouts and the proportion of time spent in them—remains poorly characterized. Additionally, the performance of various open-source activity recognition models across different demographic subgroups and the specific issue of misclassifying sleep as sedentary time require further investigation.

Methodology
This study utilized baseline data from the SMART Work & Life (SWAL) study, a cluster randomized controlled trial involving 662 working-age adults (mean age 45 years; 72% female) in England. Participants wore two accelerometers simultaneously for up to eight days: a thigh-worn activPAL3 micro (reference standard) and a wrist-worn Axivity AX3.

• Reference Standard: Thigh data were processed using the proprietary PALbatch software (CREA algorithm v1.3) to generate 1-second classifications. For this analysis, sedentary behavior was defined as epochs classified as secondary lying, sedentary (non-vehicle sitting), or vehicle sitting.
• Wrist Models Evaluated: Six open-source activity recognition models were applied to the wrist data:
  • Machine Learning Models: OxWearables 'Accelerometer' (balanced random forest + HMM), OxWearables 'ActiNet' (HARNet deep learning model + HMM), and Actimetric 'Trost' (random forest).
  • Cutpoint-Based Models: Three variations implemented via the 'GGIR' R package: Default (ENMO threshold 40mg), Bakrania (ENMO threshold 32.8mg), and Fraysse (ENMOabs threshold 62.5mg).
• Data Harmonization: All model outputs were harmonized to 30-second epochs.
• Evaluation Metrics:
  • Epoch-Level: Accuracy, F1 score, Cohen's kappa, sensitivity, and specificity against the thigh reference.
  • Participant-Level: Agreement in mean daily sedentary time, mean daily number of prolonged sedentary bouts (≥30 minutes with ≤90s interruptions), and the proportion of sedentary time in prolonged bouts. Equivalence was determined if the mean difference and 90% confidence interval fell within ±10% of the thigh-derived mean.
• Analyses: Performance was assessed across subgroups (age, sex, BMI, season, recruitment center) and in a sensitivity analysis restricted to waking hours (08:00–22:00) to mitigate sleep misclassification.

Key Results

• Model Performance: The ActiNet machine learning model demonstrated the highest overall performance. It achieved a mean per-participant accuracy of 0.87 (±0.05), an F1 score of 0.85 (±0.06), and a Cohen's kappa of 0.73 (±0.09). In contrast, cutpoint-based approaches showed lower accuracy (approx. 0.80), and the Actimetric Trost model performed the worst (accuracy 0.73).
• Agreement on Derived Metrics:
  • Daily Sedentary Time: All wrist models overestimated daily sedentary time relative to the thigh reference (8.8 hours/day). Only the ActiNet model fell within the ±10% equivalence region for this metric.
  • Prolonged Bouts: All six models produced estimates for the daily count of prolonged bouts within the equivalence region, though ActiNet showed the lowest error (RMSE/MAE) and highest correlation.
  • Proportion in Prolonged Bouts: Only ActiNet produced estimates and confidence intervals that fell entirely within the equivalence bounds for the proportion of sedentary time in prolonged bouts.
• Subgroup and Sensitivity Analysis: The ranking of models remained consistent across all demographic subgroups (age, sex, BMI, etc.). In the sensitivity analysis restricted to waking hours (08:00–22:00), the ActiNet model's F1 score improved slightly to 0.88. Restricting to waking hours reduced the overestimation of sedentary time for most models, as it removed the misclassification of sleep as sedentary behavior.

Significance and Claims
The authors claim this study represents the largest single evaluation of wrist-worn accelerometers for sedentary behavior to date and the first to assess equivalency in derived metrics beyond total daily time. The primary significance of the work is providing evidence that wrist-worn accelerometers can yield accurate measurements of sedentary behavior in free-living settings, provided that appropriate machine learning models are used. Specifically, the ActiNet model is identified as a viable alternative to thigh-worn devices for assessing sedentary patterns in similar working-age populations.

The study highlights that while total sedentary time can be estimated with reasonable accuracy by several models, more complex metrics (such as the proportion of time in prolonged bouts) are more sensitive to model limitations. The findings suggest that model selection is a critical determinant of measurement accuracy, with deep learning models incorporating temporal smoothing (like ActiNet) outperforming traditional cutpoint-based or simpler random forest approaches. This work aims to provide confidence for future epidemiological research to utilize wrist-worn data for examining sedentary behavior patterns and their associations with health outcomes.