📊 statistics

Evaluation of Surrogate Endpoints Based on Meta-Analysis with Surrogate Indices

This paper proposes a meta-analytic framework that utilizes the surrogate index as an optimal real-valued summary for evaluating complex surrogate endpoints, providing a formalized data-generating mechanism and demonstrating its validity through an application to COVID-19 vaccine efficacy trials.

Original authors: Florian Stijven, Peter B. Gilbert

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Florian Stijven, Peter B. Gilbert

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you can't wait for the final verdict. In the world of medicine, the "final verdict" is a clinical endpoint: does the patient actually feel better, function better, or survive longer? These are the gold standards, but they are like waiting for a slow-cooked stew to be ready; they take years to measure and cost a fortune. So, scientists often look for a "surrogate endpoint"—a shortcut clue, like a fever dropping or a blood marker changing, that should tell us the patient is on the mend. The problem is, sometimes these clues are misleading. A fever might drop because the patient is getting better, or because the medicine is just numbing the thermometer. To figure out if a clue is trustworthy, scientists use a method called "meta-analysis," which is like gathering all the detective reports from different cases to see if the clue consistently predicts the final outcome.

However, traditional detective work has a blind spot. It usually only looks at one single clue at a time, like just the fever or just the blood pressure. But in the real world, the body is complex. A patient's health is a symphony of many signals, not a solo instrument. If you only listen to the violin, you might miss the fact that the whole orchestra is out of tune. This is where the new paper steps in. It asks: What if we could combine all those messy, complex clues into one super-clue? The authors propose a method to build a "surrogate index," which is essentially a smart recipe that mixes together many different measurements (like antibody levels and patient history) to create a single, powerful number that might be a much better predictor of the final health outcome than any single clue could ever be on its own.

The paper, titled "Evaluation of Surrogate Endpoints Based on Meta-Analysis with Surrogate Indices," tackles the challenge of testing these complex, multi-part clues. The authors, Florian Stijven and Peter B. Gilbert, argue that the old way of doing things is too rigid for modern, complex data. They formalize a new framework that allows researchers to take a "surrogate index"—a real-valued summary created by combining various biomarkers and patient data—and test whether this new, complex summary is a good stand-in for the real clinical outcome.

Here is the core of their discovery: They show that under certain conditions, this "surrogate index" is actually the best possible summary you can make. It maximizes the connection between the treatment's effect on the clues and the treatment's effect on the real health outcome. Think of it like this: if you have a bag of different ingredients (clues), the old method tried to pick the single best ingredient to judge the cake. This new method mixes them all together into a batter (the index) that captures the true flavor of the cake much better.

The authors didn't just dream this up; they built a mathematical engine to prove it works. They formalized exactly how data is generated in these trials, making sure the rules of the game are clear. They showed that even when you have to estimate this index from the data (which is tricky because you have to learn the recipe while cooking), you can still make valid statistical conclusions about how good that recipe is. They proved that if you use standard software and follow their steps, you can get reliable answers, even if you use fancy machine learning to build the index.

To test their idea, they ran simulations (computer experiments) and applied their method to real-world data from COVID-19 vaccine trials. In these trials, scientists were trying to see if antibody levels could predict if a vaccine would stop people from getting sick. The results were promising. When they used their new "surrogate index" method, the connection between the antibody markers and the actual protection against infection was stronger and more precise than when they looked at the antibodies alone. Specifically, when they adjusted the antibody measurements to account for the specific virus strains circulating at the time, the "surrogate index" showed a very strong correlation (around 0.87 to 0.89) with the actual vaccine effectiveness. This suggests that their method successfully turned a messy collection of biological signals into a clear, reliable predictor.

However, the authors are careful not to declare total victory. They point out that their method still relies on having a decent number of trials to work with. In their COVID-19 example, they only had about six independent trials to work with, which is a small sample size. While the results looked great, the confidence intervals (the range of uncertainty) were still quite wide. They simulated what would happen if they had more trials or more precise data, and the results improved significantly, suggesting that the method is robust but needs enough data to shine. They also note that this approach requires individual participant data from all the trials, which isn't always available, unlike the older methods that can sometimes work with just summary statistics.

In short, this paper offers a new, more flexible toolkit for medical detectives. It suggests that by combining multiple clues into a smart "surrogate index," we can better evaluate whether a shortcut measurement is truly reliable. It doesn't solve every problem—especially when data is scarce or messy—but it provides a rigorous way to handle the complexity of modern biology, turning a jumbled pile of data into a clear signal that could help speed up the development of life-saving treatments.

Technical Summary: Evaluation of Surrogate Endpoints Based on Meta-Analysis with Surrogate Indices

Problem Statement
The meta-analytic (MA) framework is widely regarded as the gold standard for evaluating surrogate endpoints in clinical trials. However, the canonical MA framework is generally restricted to univariate surrogates. While methods exist for multivariate surrogates, they are typically limited to low-dimensional settings (e.g., bi- or trivariate) or require imposing specific structural constraints that may not reflect clinical reality. In many modern contexts, promising surrogates are complex and high-dimensional (e.g., multiple biomarkers or longitudinal measurements). Existing methods struggle to handle such complexity without overfitting or losing interpretability. Furthermore, the data-generating mechanisms and identifying assumptions underlying the MA framework are often left implicit in the literature, complicating valid inference, particularly when the surrogate is estimated from data.

Methodology
The authors propose extending the MA framework to evaluate complex surrogates by utilizing a surrogate index—a real-valued summary of complex surrogates and baseline covariates—as the candidate surrogate endpoint. The methodology proceeds through the following steps:

Formalization of the Data-Generating Mechanism: The paper formalizes the MA framework by defining a hierarchical data-generating mechanism. Trials are treated as random elements sampled from a population of trials. Within each trial, individual participant data $(X, Z, S, Y)$ are observed, where $X$ are baseline covariates, $Z$ is treatment, $S$ is the (potentially complex) surrogate, and $Y$ is the clinical endpoint. The framework distinguishes between full-data parameters (unobserved true trial-level treatment effects) and statistical parameters (observed estimates).
The Surrogate Index ( $g_0$ ): Drawing on the work of Athey et al. (2025) and Gilbert et al. (2025), the authors define a surrogate index $g_0(x, s)$ as a pooled regression function of the clinical endpoint $Y$ on the surrogate $S$ and covariates $X$ , estimated by pooling individual-participant data across all trials. Under strong identifying assumptions (surrogacy and comparability), $g_0$ maximizes the trial-level correlation between the treatment effect on the transformed surrogate and the clinical endpoint.
Data-Adaptive Target Parameters: Since $g_0$ is unknown, it must be estimated from the data, resulting in an estimated surrogate index $g_N$ . Consequently, the target parameter (the trial-level surrogacy) becomes data-adaptive, indexed by $g_N$ . The paper addresses the challenge of performing valid statistical inference for such data-adaptive parameters.
Estimation and Inference:
- Plug-in Estimator: The authors propose a plug-in estimator for the trial-level correlation and other parameters based on the estimated surrogate index $g_N$ .
- Asymptotic Theory: To ensure valid inference, the paper establishes conditions under which the plug-in estimator for the data-adaptive parameter is asymptotically linear and normally distributed. These conditions rely on the estimated surrogate index $g_N$ converging to a fixed function in the supremum norm and satisfying specific complexity constraints (Donsker conditions).
- Flexibility: The framework allows for the use of flexible machine-learning techniques (e.g., SuperLearner) to estimate $g_N$ , provided the complexity of the estimator is controlled to prevent overfitting. The authors specifically recommend using leave-one-trial-out cross-validation to mitigate trial-level overfitting.
- Variance Estimation: Inference is conducted using sandwich variance estimators or bootstrap methods (specifically the Bayesian bootstrap) to account for the uncertainty in both the trial-level treatment effects and the estimation of the surrogate index.

Key Contributions

Extension to Complex Surrogates: The paper provides a rigorous framework for evaluating trial-level surrogacy for complex, high-dimensional surrogates by reducing them to a univariate surrogate index.
Formalization of Assumptions: It explicitly formalizes the data-generating mechanism and the untestable identifying assumptions (unbiased estimation of trial-level treatment effects and within-trial sampling variability) required for valid inference in the MA framework.
Inference for Data-Adaptive Parameters: The authors derive conditions under which valid inference is possible for the trial-level surrogacy of an estimated surrogate index. This allows for the use of flexible, data-driven estimators (machine learning) while maintaining statistical validity, a significant departure from traditional parametric MA approaches.
Practical Implementation: The proposed approach is designed to be implementable using standard software and is illustrated through a detailed application to COVID-19 vaccine efficacy trials.

Results

Simulations: The authors conducted simulations in two scenarios: a proof-of-concept scenario and a realistic vaccine-trial scenario. Results indicated that the estimated surrogate indices yielded trial-level correlations closer to 1 than the original untransformed surrogates. The proposed plug-in estimator and bootstrap confidence intervals demonstrated appropriate coverage properties as the number of trials increased, though coverage was less stable in very small samples (e.g., $N=6$ ).
Application to COVID-19 Trials: The method was applied to data from five Phase 3 COVID-19 vaccine trials (Moderna, AstraZeneca, Janssen, Novavax, and Sanofi).
- Surrogates Evaluated: Log10 IgG binding antibody concentration (IgG Spike) and log10 neutralizing antibody titer (nAb ID50), both unadjusted and adjusted for circulating viral strains.
- Findings: The estimated surrogate indices (derived via Cox proportional hazards models and SuperLearner) showed higher trial-level correlations with vaccine efficacy than the raw antibody markers. Specifically, the adjusted nAb ID50 (accounting for circulating strains) demonstrated the strongest surrogacy.
- Limitations in Application: Due to the small number of independent trials ( $N=6$ ), confidence intervals for the trial-level correlations remained wide, leading to inconclusive definitive statements about surrogacy. However, the method successfully generated predictions for vaccine efficacy in non-naive populations (Sanofi trials) that were consistent with observed data, illustrating the predictive utility of the surrogate index.

Significance and Claims
The paper claims that its approach extends the canonical MA framework to settings where complex surrogates cannot be handled by existing methods. By formalizing the data-generating mechanism and providing conditions for valid inference on data-adaptive parameters, the authors enable the evaluation of surrogates that are defined via flexible regression functions.

The authors are modest regarding the limitations:

The approach still relies on the canonical MA framework's limitations, such as the ill-defined nature of the "population of trials" and low statistical power when the number of trials is small.
Unlike the canonical framework which can sometimes operate with summary-level statistics, this approach requires individual-participant data with harmonized covariates across all trials.
The validity of the inference depends on the plausibility of the identifying assumptions (surrogacy and comparability) and the convergence properties of the surrogate index estimator.
The method does not inherently solve the problem of intercurrent events (e.g., death or dropout) precluding surrogate measurement, though the application restricted analysis to a per-protocol cohort to address this.

Ultimately, the paper positions the surrogate index not just as a summary measure, but as a rigorous statistical tool that bridges the gap between complex biological markers and clinical endpoints within a meta-analytic context, offering a pathway for more efficient clinical trial design when strong surrogates are suspected but complex.

More like this