📊 statistics

Group Sequential Methods for the Win Ratio

This paper establishes that the independent increments assumption required for classical group sequential methods holds for the win ratio by deriving its asymptotic covariance structure, thereby demonstrating that traditional $\alpha$ -spending functions can be validly applied to randomized trials using this endpoint.

Original authors: Tracy Bergemann, Tim Hanson

Published 2026-02-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tracy Bergemann, Tim Hanson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a judge presiding over a series of head-to-head boxing matches between two teams: Team A (the new treatment) and Team B (the standard care). In a traditional medical trial, you might just count how many people in each team got sick or died. But in a Win Ratio trial, you look at every single possible pairing between a member of Team A and a member of Team B.

For each pair, you ask: "Who did better?"

If Team A's patient had a better outcome (e.g., kept their limb, didn't need surgery), Team A gets a "Win."
If Team B's patient did better, Team A gets a "Loss."
If they did the same, it's a "Tie."

The Win Ratio is simply the total number of Wins divided by the total number of Losses. If the ratio is high, Team A is winning the majority of these matchups.

The Problem: The "Moving Target"

Now, imagine you want to stop the trial early if Team A is clearly winning, rather than waiting until the very end. This is called a Group Sequential Design. You check the scoreboard at 50% of the time, then at 75%, and so on.

Here is the tricky part: In many statistical methods, the "score" you calculate at the 50% mark must be mathematically independent of the score you calculate at the 75% mark. Think of it like rolling dice. If you roll a 6 on your first try, it shouldn't change the odds of rolling a 6 on your second try.

However, in a Win Ratio trial, the math gets messy. When you add a new patient to the study at the 75% mark, you don't just compare them to the new Team B patients; you compare them to everyone in Team B, including the people you already looked at. This creates a "sticky" situation where the new score is mathematically tied to the old score.

For a long time, statisticians worried that because these scores were "sticky" (correlated), you couldn't use the standard, trusted rules (called Lan-DeMets alpha-spending) to decide when to stop the trial. They thought the "independent increments" rule was broken, meaning the standard rules might let too many false alarms through.

The Solution: The "Snapshot" Trick

The authors of this paper, Tracy Bergemann and Tim Hanson, asked a simple question: What if we only look at the data as a fixed snapshot?

They proposed a specific way of running the trial:

Fixed Time: Everyone is followed for a set amount of time (say, 12 months).
The Snapshot: When you do an interim check (e.g., at 6 months), you only count patients who have finished their full 12 months of data (or dropped out). You ignore anyone who is still "in progress."
No Moving Parts: Because you only count finished patients, the data you used for the 6-month check is exactly the same data you used for the 12-month check; you just added new finished patients to the mix.

The Big Discovery

The authors did some heavy mathematical lifting (using something called U-statistics) to prove that if you use this "snapshot" method, the "sticky" correlation disappears in the long run.

The Analogy:
Imagine you are counting the height of trees in a forest.

The Old Way (Correlated): You measure a sapling today, then measure it again tomorrow. The second measurement is heavily influenced by the first because it's the same tree growing.
The Authors' Way (Independent): You only measure trees that have reached their full adult height. When you check the forest at 6 months, you measure the adult trees. At 12 months, you measure the same adult trees plus new trees that have just finished growing. The "growth" of the new trees doesn't change the height of the old trees.

They proved that under these conditions, the "score" at the 6-month check and the "score" at the 12-month check act like independent dice rolls. This means you can use the standard, trusted rules (Lan-DeMets) to decide when to stop the trial early.

The Proof: Simulations and Real Data

To make sure their math wasn't just theory, they ran two tests:

Computer Simulations: They created 10,000 fake trials where the new treatment was actually no better than the old one. They checked if the standard rules kept the "false alarm" rate low. Result: Yes, the rules worked perfectly. The error rate stayed exactly where it was supposed to be.
Real Data Check: They looked at a real past trial called IN.PACT SFA (a study on leg artery disease). They re-analyzed the data as if they had been checking the scoreboard halfway through. Result: If they had used this method, the trial would have stopped early for success because the new treatment was winning so decisively.

The Bottom Line

This paper says: "Don't worry about the complex math of the Win Ratio. If you design your trial so you only count patients with complete data at each check-in, the math works out. You can use the standard, off-the-shelf software and rules that doctors already trust to stop trials early when a treatment is clearly winning."

What they did NOT claim:

They did not say this works for every type of trial (specifically those where patients have "partial" or incomplete data at the interim check, though they ran a simulation suggesting it might work there too, they said more research is needed).
They did not invent a new way to calculate the Win Ratio; they just proved the rules for stopping early apply to it.

Technical Summary: Group Sequential Methods for the Win Ratio

Problem Statement
The win ratio is increasingly utilized in randomized clinical trials as a primary endpoint due to its ability to incorporate composite outcomes with hierarchical clinical severity and combine disparate data types (e.g., time-to-event, binary, counts). However, a significant methodological gap exists regarding the implementation of group sequential designs (GSD) for win ratio endpoints. Specifically, it is unclear whether the independent increments assumption, a prerequisite for applying classical Lan-DeMets $\alpha$ -spending functions, holds for incremental win-loss test statistics.

Previous literature suggests that for exact bivariate distributions of incremental statistics, independence is violated because new patients enrolled at later interim looks are compared against pre-existing patients, creating correlation. Consequently, prior approaches have recommended complex modifications, such as block stratification (which requires infeasible spacing between looks) or empirical bootstrap estimation of covariance (which cannot be pre-specified). The central question addressed by this paper is whether the independent increment assumption is satisfied for the asymptotic distribution of U-statistics used to evaluate the win ratio, thereby allowing for the straightforward application of traditional GSD boundaries.

Methodology
The authors develop a mathematical framework to derive the covariance structure of incremental U-statistics under the asymptotic distribution.

Framework and Assumptions:
- The study assumes a fixed follow-up time $T$ for all participants, where the primary endpoint is evaluated at $T$ . This avoids dependence on variable follow-up times.
- For the primary theoretical derivation, the authors assume that only patients with completed follow-up are included in interim analyses. This ensures that the information contributed by a patient to an interim statistic $Z_k$ remains constant in subsequent statistics $Z_l$ ( $k < l$ ).
- The win ratio ( $\psi = \tau_1/\tau_2$ ) and win difference are defined via U-statistics based on pairwise comparisons between treatment ( $X$ ) and control ( $Y$ ) groups.
Mathematical Derivation:
- Win Difference: The authors prove that for incremental test statistics of the win difference ( $U_{1k} - U_{2k}$ ), the covariance between increments at times $k$ and $l$ equals the variance of the later increment: $\text{cov}(Z_k, Z_l) = \sqrt{I_k(\theta)/I_l(\theta)}$ . This confirms the independent increments assumption holds for the asymptotic distribution of the win difference.
- Win Ratio: The derivation is extended to the log-transformed win ratio. The authors demonstrate that $\text{cov}(\log(\hat{\psi}_k), \log(\hat{\psi}_l)) \approx \text{var}(\log(\hat{\psi}_l))$ , indicating that the independent increments assumption approximately holds for the win ratio under asymptotic conditions.
Simulation Studies:
- Scenario 1 (Complete Information): Simulations were conducted with sample sizes $N=200$ and $N=400$ to evaluate the overall Type I error rate using the Hwang-Shih-DeCani $\alpha$ -spending function. The analysis included only patients with complete follow-up at each interim look.
- Scenario 2 (Partial Information): Simulations were extended to include patients contributing partial follow-up information at interim analyses, calculating information fractions based on person-years.

Key Results

Theoretical Proof: The paper provides a mathematical proof that the independent increments assumption is met for the asymptotic distribution of U-statistics testing the win difference. It further shows that this property approximately holds for the log-transformed win ratio.
Type I Error Control: In the simulation study with $N=200$ , the empirical Type I error rate was 0.0517; for $N=400$ , it was 0.0487. These results demonstrate that traditional $\alpha$ -spending functions preserve the Type I error rate even with relatively small sample sizes.
Partial Information: The simulation results for Scenario 2 (partial information) upheld similar findings, suggesting the method is robust even when incomplete data is utilized, though the theoretical proof for this specific case is not fully established in the text.
Retrospective Application: A re-analysis of the IN.PACT SFA clinical trial data was performed. Under a hypothetical group sequential design with interim looks at 50% and 75% information, the trial would have stopped for success at the first interim look (50% information) with a win ratio of 4.29 ( $p < 0.001$ ), compared to the final result of 3.82.

Significance and Claims
The authors claim that this research demonstrates that straightforward application of Lan-DeMets $\alpha$ -spending functions is possible for randomized trials involving the win ratio, provided that sample sizes are sufficiently large to assume asymptotic distributions and specific conditions regarding patient follow-up are met.

Key implications highlighted by the authors include:

Elimination of Empirical Boundaries: Researchers no longer need to generate boundaries empirically via bootstrapping at the time of interim looks, which cannot be pre-specified.
Software Compatibility: Existing software packages capable of computing traditional group sequential boundaries (e.g., the gsDesign package in R) can be employed directly for win ratio endpoints.
Modest Scope: The authors explicitly note that their primary derivations rely on the assumption of using only patients with completed follow-up. While simulations suggest the method works with partial information, they state that "further research is needed to establish that result" for situations where partial patient information is used in earlier interim analyses akin to the log-rank test.

The paper concludes that the independent increment assumption is met for the win difference and approximately for the win ratio under asymptotic conditions, validating the use of standard group sequential methodologies for these endpoints.