A Deep Set-Based Aggregation Approach for Repeated Measurements: Insights from Variable-Length Wearable Device-Measured Physical Activity Data

Technical Summary: A Deep Set-Based Aggregation Approach for Repeated Measurements

Problem Statement
The rapid integration of wearable devices into digital health has enabled continuous, real-time monitoring of patient metrics, such as physical activity (PA). However, leveraging this data for predictive modeling presents significant challenges. Repeated measurements from wearables often vary in length across individuals due to missing data or differing compliance, creating "variable-length" inputs that standard machine learning algorithms cannot process directly, as they typically require fixed-length feature vectors.

Traditional solutions involve aggregating these repeated observations into summary statistics (e.g., means) to fix dimensionality. While effective for reducing dimensionality, simple aggregation often discards valuable temporal dynamics and interaction patterns between observations. Furthermore, existing deep learning architectures (e.g., CNNs, RNNs) often assume a one-to-one correspondence between feature vectors and outcomes, making them ill-suited for mapping multiple time-series observations to a single clinical outcome without restrictive assumptions about independence.

Methodology
The study utilized data from the 2011–2014 National Health and Nutrition Examination Survey (NHANES), focusing on adults with arthritis. The dataset included multi-day accelerometer-derived PA measurements (recorded as Monitor-Independent Movement Summary [MIMS] units) and one-time covariates (demographics, self-reported PA, medical conditions). The study evaluated two binary outcomes: the need for special equipment to walk and the presence of poor physical functioning.

Three aggregation strategies were compared to transform variable-length PA data into fixed-length predictors:

1. Simple Aggregation (Method 1): Calculated the arithmetic mean of daily PA measurements for each participant. This serves as a baseline, assuming that the average value sufficiently represents the individual's activity profile.
2. Variance-Adjusted Aggregation (Method 2): Employed a statistical approach to adjust the simple mean based on the reliability of the measurements. This method decomposes variance into between-individual and within-individual components. It applies weights to the population mean, the individual's observed mean, and one-time covariates. The weighting scheme increases reliance on repeated measurements when within-individual variance is low and the number of observations is high, while shifting weight toward covariates when measurements are sparse or highly variable.
3. Deep Set Approach (Method 3): Implemented a deep learning architecture designed specifically for set-based, variable-length inputs. This approach leverages two key properties: permutation invariance (the output is independent of the order of input elements) and permutation equivariance.
   • Architecture: The model treats each participant's PA history as a set. For day-level data, the model utilized a self-attention mechanism with Query, Key, and Value (QKV) vectors to capture temporal dependencies and interactions across days without explicit feature engineering. For minute-level data (high-frequency, complex inputs), the architecture incorporated Convolutional Neural Network (CNN) layers for feature extraction prior to the self-attention aggregation.
   • Training: The model was trained to jointly learn feature extraction and aggregation, allowing it to discover non-linear patterns across repeated observations that traditional methods might miss.

Key Results
The study evaluated model performance using AUC, accuracy, precision, recall, and F1 score across 100 iterations of 80/20 train-test splits.

• Day-Level Data: For day-level PA summaries, the Deep Set approach generally improved accuracy and F1 scores compared to simple and variance-adjusted aggregation. However, it exhibited lower AUC scores, particularly among participants with fewer observations (< 6 days). In this low-observation subgroup, the Deep Set model underperformed, likely due to the small sample size of this group and the lack of meaningful temporal interactions in short-term data. Conversely, for participants with 6–7 days of data, performance differences between methods narrowed.
• Minute-Level Data: When applied to minute-level PA data, the Deep Set approach demonstrated superior overall performance, achieving the highest AUC, accuracy, and F1 scores for both outcomes. This suggests that the model's ability to extract features and model complex, high-frequency temporal dynamics provides a significant advantage when data resolution is high.
• Variance Adjustment: The variance-adjusted method showed minimal practical improvement over simple aggregation in this specific dataset, largely because most participants had sufficient observations (6–7 days), resulting in low within-individual variance and thus small adjustment weights.

Significance and Contributions
The paper contributes a comparative framework for selecting aggregation strategies in digital health, emphasizing that the optimal method depends on data complexity, variability, and sample size.

• Context-Dependent Selection: The authors propose that simple aggregation remains sufficient for relatively stable, low-frequency repeated measurements (e.g., day-level PA with consistent patterns). In contrast, Deep Set-based approaches are more beneficial for complex, high-frequency data (e.g., minute-level streams) where capturing non-linear temporal dynamics is crucial.
• Handling Variable Length: The study validates the Deep Set architecture as a viable solution for integrating variable-length wearable data into clinical models without discarding participants with incomplete data or losing temporal nuance.
• Practical Guidelines: The findings offer a roadmap for researchers: if the dataset contains many participants with numerous, complex, and dynamic observations, Deep Sets are advantageous. However, for sparse data or simple metrics, traditional statistical aggregation may be more robust and less prone to overfitting.

The authors conclude that while Deep Sets offer exceptional flexibility for real-world digital medicine applications involving complex data streams, their performance is contingent upon having sufficient data volume and observation density to learn meaningful interactions.