📊 statistics

Random-Effects Algorithm for Random Objects in Metric Spaces

This paper proposes a nonlinear Fréchet-based random-effects algorithm for modeling arbitrary random objects in metric spaces, establishing its consistency through M-estimation theory and demonstrating its superior performance over existing Hilbert space-based methods on both synthetic and digital health datasets.

Original authors: Marcos Matabuena, Mateo Cámara

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marcos Matabuena, Mateo Cámara

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Predicting the Future of "Weird" Data

Imagine you are trying to predict how a person's health will look tomorrow. In the past, scientists mostly looked at simple numbers: a blood pressure reading of 120, a heart rate of 70, or a weight of 150 lbs. You can easily add, subtract, or average these numbers.

But modern technology (like smartwatches and continuous glucose monitors) gives us something much more complex. Instead of a single number, we get:

A whole curve showing glucose levels all day.
A cloud of points showing how a person moves.
A network map showing how different hours of the day are connected.

These are called "random objects" in a "metric space." Think of them as shapes or maps rather than simple numbers. The problem is: You can't just add or subtract shapes. You can't take a "glucose curve" and subtract a "heart rate curve" to find the average.

This paper introduces a new way to handle these complex shapes. It's like building a new kind of calculator that works for maps and clouds, not just numbers.

The Problem: Everyone is Different (The "Random Effect")

In statistics, we often try to find a "general rule." For example, "Older people tend to have higher blood pressure." This is the Fixed Effect.

But people are unique. Even if two people are the same age, their daily glucose patterns might look totally different because of their unique biology, diet, or habits. This unique, personal "fingerprint" is called the Random Effect.

Old Way: If you want to predict Person A's glucose, you look at the "average person" rule.
New Way: You look at the "average person" rule plus Person A's specific history.

The challenge is: How do you calculate a "personalized average" when your data isn't a number, but a complex shape?

The Solution: The "Anchor" Strategy

The authors propose a clever trick. Since you can't do math directly on these complex shapes, they turn the shapes into simple numbers using "Anchors."

Imagine you are trying to describe a strange new fruit to a friend, but you can't show them the fruit. Instead, you say:

"It is 2 inches away from an Apple."
"It is 5 inches away from a Banana."
"It is 1 inch away from a Pear."

By measuring the distance from the new fruit to known "Anchors" (Apples, Bananas, Pears), you can describe the new fruit using simple numbers (distances).

Here is how the algorithm works:

Pick Anchors: The computer picks a bunch of real, observed data points from the past (like yesterday's glucose curve) to serve as "Anchors."
Measure Distances: For every new observation, the computer measures how far it is from every Anchor. Now, instead of a complex shape, we have a list of simple numbers (distances).
Do the Math: The computer runs a standard statistical model on these distance numbers. It learns: "When a person has these specific traits (age, sex, etc.), their distance to the 'Apple Anchor' usually looks like this."
Add the Personal Touch: It also calculates a "personal bonus" for each individual based on their past history. This is the Random Effect. It learns, "Person A is usually 10% closer to the 'Apple Anchor' than the average person."
Reconstruct the Prediction: Finally, the computer looks at all the Anchors again. It asks: "Which of our original shapes is closest to the predicted distances?" It picks that shape as the final prediction.

What Did They Test?

The authors tested this "Anchor" method on four different types of real-world health data:

Daily Activity Curves (Walking): They looked at how people move throughout the day.
- Result: The personalized method (with Random Effects) was better at predicting the next day's activity than the general method.
Glucose Curves: They looked at blood sugar levels over 24 hours.
- Result: Again, the personalized method won. It captured the unique ups and downs of individual diabetics better than the average rule.
Glucose "Clouds" (Distributions): Instead of a line, they looked at the whole distribution of glucose (how often it was high, low, or normal) over 40 days.
- Result: This was the biggest win. The personalized method reduced errors by 57%. The "personal fingerprint" was crucial here.
Activity Maps (Graphs): They looked at how activity in the morning connects to activity in the evening (a network map).
- Result: Surprisingly, the personalized method didn't help much here. The authors suggest that for this specific type of map, the day-to-day changes were too chaotic to find a stable "personal pattern."

The Takeaway

This paper doesn't just say "we have a new math formula." It says: "We have a practical tool to predict complex health patterns for specific individuals."

Why it matters: It allows doctors or AI to say, "Based on your specific history, here is what your health data will likely look like tomorrow," rather than just giving a generic average.
The Catch: It requires a lot of computing power because it has to run many small calculations (one for each "Anchor"), but the authors show it can still be done quickly on modern computers.

In short, they built a bridge that lets us use simple statistical tools to understand and predict complex, non-numerical health data, while respecting the unique quirks of every single patient.

Technical Summary: Random-Effects Algorithm for Random Objects in Metric Spaces

Problem Statement
Modern scientific disciplines, particularly digital health, generate datasets where multiple observations are collected from the same experimental units (e.g., individuals). These observations often manifest as non-Euclidean random objects—such as probability distributions, networks, or functional curves—rather than scalar or vector outcomes. While mixed-effects models are well-established for scalar outcomes and, more recently, for functional data in Hilbert spaces, there is a lack of general random-effects frameworks for arbitrary random objects defined on metric spaces. Existing methods for metric-space data are often limited to linear models, geodesic spaces, or univariate distributions, failing to provide a flexible baseline for repeated responses in general metric spaces that require personalized prediction.

Methodology
The authors propose a nonlinear, Fréchet-based algorithm for random-effects modeling of arbitrary random objects in a metric space $(\Omega, d)$ . The core innovation is an anchor-based representation of latent subject-specific random effects, avoiding the need for direct addition or subtraction operations in the metric space.

Anchor Transformation:
- The method selects a set of observed responses (anchors) $A_\ell$ from the empirical response space.
- For each anchor $A_\ell$ , the metric-space response $Y_{ij}$ is transformed into a scalar variable: $Z^{(\ell)}_{ij} = \log(d^2(Y_{ij}, A_\ell) + \delta)$ , where $\delta > 0$ ensures numerical stability.
Scalar Mixed-Effects Fitting:
- For each anchor $\ell$ , a scalar mixed-effects model is fitted to the transformed distances:
  $Z^{(\ell)}_{ij} = f_\ell(X_{ij}) + b^{(\ell)}_i + \varepsilon^{(\ell)}_{ij}$
- Here, $f_\ell$ is a fixed-effect regression function (implemented via gradient-boosted trees), and $b^{(\ell)}_i$ is an anchor-specific random intercept representing the subject $i$ 's latent effect on the distance to anchor $\ell$ .
- The implementation utilizes GPBoost, combining gradient-boosted trees with Gaussian random-effect components.
Prediction via Discrete Optimization:
- To predict the response for a new covariate $x$ for subject $i$ , the algorithm estimates the subject-specific random effects $\hat{b}^{(\ell)}_i$ and fixed effects $\hat{f}_\ell(x)$ .
- The final prediction $\hat{m}_{n,i}(x)$ is obtained by solving a discrete optimization problem over the observed response space $\mathcal{Y}_n$ :
  $\hat{m}_{n,i}(x) \in \arg \min_{\omega \in \mathcal{Y}_n} \sum_{\ell \in I_n} \left( \log(d^2(\omega, A_\ell) + \delta) - (\hat{f}_\ell(x) + \hat{b}^{(\ell)}_i) \right)^2$
- Essentially, the method selects the observed object whose profile of transformed distances to the anchors best matches the predicted profile for the specific subject.

Theoretical Contributions
The paper establishes consistency results for the proposed estimator using M-estimation theory. Under specific assumptions regarding the uniqueness of the minimizer, separation of risk, and uniform convergence of the empirical criterion to the population criterion, the authors prove that the estimated prediction target converges in probability to the true oracle target, even when the random-effect structure is treated as a working model.

Empirical Results
The method was evaluated on four digital health datasets involving different types of metric-space objects:

$L^2$ Hilbert Space (Functional Data):
- NHANES (Physical Activity): Incorporating random effects reduced the held-out Mean Squared Error (MSE) by ~10% (16.26 to 14.64).
- CGMCR (Glucose Curves): MSE reduction of ~8.6% (4,368 to 3,994). The nonlinear approach outperformed existing linear functional mixed-effects models in both cases.
Wasserstein Metric (Multivariate Distributions):
- CGM 3D Distributions: Representing glucose dynamics as multivariate distributions (glucose, velocity, acceleration). Random effects yielded a substantial 57% reduction in squared Wasserstein error (986 to 421), indicating that subject-specific variation is the primary driver for personalized prediction in this setting.
Frobenius Metric (Graphs):
- NHANES Laplacian Graphs: Representing temporal correlation structures of activity. In this case, random effects provided negligible improvement (MSE 0.665 vs. 0.664), suggesting that the thresholded graph construction did not capture stable subject-specific signals. However, a small subgroup of highly sedentary individuals showed significant gains from the random-effects approach.

Significance and Claims
The paper claims to provide the first broadly applicable, nonlinear random-effects framework for arbitrary random objects in metric spaces. Its significance lies in:

Flexibility: It extends massive univariate fitting strategies to metric-space-valued data without requiring linear vector-space structures.
Performance: It demonstrates that anchor-based nonlinear modeling can outperform existing Hilbert-space-based linear methods and effectively capture individual heterogeneity in complex data structures like distributions and graphs.
Practicality: The algorithm is computationally simple, parallelizable, and suitable for high-frequency clinical data where signals cannot always be temporally aligned.

The authors acknowledge limitations, primarily the computational cost of fitting multiple univariate models, though they note that for typical sample sizes (e.g., $n \approx 4000$ ), the solution is computationally feasible on high-performance clusters. Future work is suggested for scalability via subsampling and uncertainty quantification.