📊 statistics

Tree models for covariate-dependent method agreement with repeated measurements in clinical research

This paper introduces Conditional Method Agreement Trees (COAT), a novel regression tree method that extends Bland-Altman analysis to handle repeated measurements by identifying covariate-dependent subgroups with heterogeneous agreement between clinical measurement techniques.

Original authors: Siranush Karapetyan, Achim Zeileis, Moritz Flick, Bernd Saugel, Alexander Hapfelmeier

Published 2026-04-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Siranush Karapetyan, Achim Zeileis, Moritz Flick, Bernd Saugel, Alexander Hapfelmeier

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to decide between two different ways to measure a patient's heart performance (Cardiac Output). One way is the "Gold Standard": a catheter inserted into the heart, which is accurate but invasive and risky. The other is a new, high-tech "finger cuff" that is non-invasive and easy to use.

Before you can trust the new finger cuff, you need to know: Does it give the same reading as the gold standard?

The Old Way: The "One-Size-Fits-All" Test

Traditionally, scientists use a method called Bland-Altman analysis. Think of this like taking a group of 50 people, measuring them with both devices, and calculating the average difference.

The Problem: This method assumes that everyone is the same. It assumes the finger cuff is equally accurate for a 20-year-old athlete, a 70-year-old with heart disease, and a heavy person just as it is for a light person.
The Reality: In the real world, devices often behave differently depending on the person. Maybe the finger cuff works great for young people but fails for older ones. Maybe it works well when the heart rate is low but gets confused when the heart is racing. The old method misses these hidden patterns because it just averages everything together.

The New Way: The "Tree" Approach (COAT)

The authors of this paper introduce a new tool called COAT (Conditional Method Agreement Trees).

The Analogy: The Detective's Tree
Imagine you are a detective trying to figure out why a new security camera sometimes misses suspects.

The Old Method: You look at all the footage, count the total misses, and say, "The camera misses 10% of the time." You stop there.
The COAT Method: You build a decision tree. You ask questions to split the suspects into groups:
- Question 1: "Is the suspect wearing a hat?" (No -> Good accuracy. Yes -> Bad accuracy).
- Question 2: "Is it raining?" (Yes -> Bad accuracy. No -> Good accuracy).
- Question 3: "Is the suspect moving fast?"

By the end, you don't just have one average error rate. You have a map that says: "The camera is perfect for people without hats on sunny days, but it fails for people wearing hats when it's raining."

How COAT Works:

It Handles Repeated Measurements: In clinical studies, doctors often measure a patient multiple times (e.g., 5 times in a row). COAT is smart enough to handle this "repeated" data without getting confused, separating the "noise" of a single measurement from the true pattern.
It Finds the "Subgroups": It automatically scans through patient data (age, weight, sex, health conditions) to find where the agreement between the two devices breaks down.
It's Statistically Safe: The authors ran thousands of computer simulations to prove that COAT doesn't just find "ghost" patterns (false alarms). It only splits the tree when there is a real, significant difference.

The Real-World Test: Heart Surgery

The authors tested this on real data from heart surgery patients. They compared the invasive catheter (Gold Standard) against the new finger cuff.

What they found:

The "Average" Lie: If you just looked at the average, the devices seemed okay.
The "Tree" Truth: COAT revealed that the finger cuff was much less accurate when the patient's heart output was very high (above a certain level). It also hinted that the patient's sex and weight might change how well the devices agreed.

Why This Matters

This is a game-changer for medical research because it moves us away from "One size fits all" to "Right tool for the right patient."

Instead of saying, "This new device is good," doctors can now say:

"This new device is excellent for patients under 70 with a BMI under 30, but we should be careful using it on older, heavier patients because the readings might drift."

Summary

The Problem: Old methods assume medical devices work the same for everyone, which isn't true.
The Solution: COAT is a "smart tree" that digs into the data to find specific groups of patients where a device works well or poorly.
The Benefit: It helps doctors know exactly when and for whom a new medical technology is safe and reliable, leading to better patient care.

The authors even made this tool available as a free software package (called coat) so other researchers can use it to build their own "agreement trees."

1. Problem Statement

In clinical research, the Bland-Altman (BA) analysis is the standard method for assessing the agreement between two measurement techniques (e.g., a new non-invasive device vs. an invasive gold standard). The BA analysis estimates two key parameters:

Bias: The mean difference between methods.
Limits of Agreement (LoA): A 95% prediction interval of the differences, derived from the standard deviation of the differences.

The Core Limitation: Traditional BA analysis relies on a strong, implicit assumption that method agreement is homogeneous across all subjects. It assumes that bias and LoA are constant regardless of patient characteristics (covariates like age, sex, BMI) or experimental settings.
The Challenge: In reality, agreement often varies based on covariates. Furthermore, clinical studies frequently involve repeated measurements per subject (either paired or unpaired), which introduces complex variance structures (between-subject and within-subject variance) that standard BA analysis handles via averaging but fails to model as a function of covariates.

2. Methodology

The authors propose Conditional Method Agreement Trees (COAT), a novel extension of Conditional Inference Trees (CTree) designed to model method agreement as a function of covariates in the presence of repeated measurements.

A. Theoretical Framework

COAT generalizes the multivariable modeling approach previously used for single measurements to handle repeated measurements. It decomposes the parameters of the BA analysis into subject-specific components:

Bias ( $E(Y|X)$ ): The conditional expectation of the difference $Y = A - B$ .
Variance ($Var(Y|X)$): The conditional variance of the difference, accounting for:
- Between-subject variance ( $\sigma^2_Y$ ).
- Within-subject variance ( $\sigma^2_A, \sigma^2_B$ ).

The method distinguishes between two measurement settings defined by Bland and Altman:

Unpaired Measurements: Repeated measurements of an invariant true value (e.g., measuring a static object multiple times).
Paired Measurements: Repeated measurements of a changing true value where measurements are paired by time/instance (e.g., cardiac output over time).

B. Algorithmic Implementation

COAT utilizes Conditional Inference Trees (CTree) with a specific transformation function $h(\cdot)$ to handle the bivariate outcome (bias and variance simultaneously).

Transformation Function: Instead of modeling raw differences, the algorithm transforms the data into subject-specific components of the BA parameters.
- For Unpaired data: $h(\cdot) = (y_i, \hat{\sigma}^2_{Yi})$ , where $y_i$ is the subject mean difference and $\hat{\sigma}^2_{Yi}$ is the subject-specific variance component.
- For Paired data: $h(\cdot) = (n m_i y_i / \sum m_i, \hat{\sigma}^2_{Yi})$ , incorporating weighting for the number of replicates.
Splitting Mechanism: The tree recursively partitions the data based on covariates ( $X$ ) that show a statistically significant association with the transformed outcome. It uses permutation tests to determine splits, controlling for multiple testing via Bonferroni correction.
Output: The resulting tree defines subgroups where the bias and LoA differ significantly, explicitly modeling heterogeneity.

C. Software

The method is implemented in the open-source R package coat.

3. Key Contributions

Extension to Repeated Measurements: The primary contribution is the generalization of conditional method agreement from single measurements to the complex, common scenario of repeated measurements (both paired and unpaired).
Simultaneous Modeling: Unlike standard regression trees that might model bias or variance separately, COAT models the joint distribution of bias and variance (the full BA parameters) using a bivariate transformation.
Subgroup Discovery: COAT explicitly identifies subgroups of patients where agreement differs, rather than assuming a global average.
Two-Sample Testing: The framework allows for a formal statistical test to compare agreement between two pre-defined groups (e.g., males vs. females) by applying COAT to a binary covariate.

4. Results

A. Simulation Study

A simulation study with 5,000 replications assessed Type-I error, power, and subgroup identification accuracy (Adjusted Rand Index - ARI).

Type-I Error Control: COAT successfully controlled the Type-I error rate at the nominal level (0.05) for both paired and unpaired measurements, even with varying sample sizes ( $N=50$ to $300$).
Power and ARI:
- Bias-only heterogeneity: Standard CTree performed slightly better than COAT in detecting bias-only changes because COAT tests a bivariate outcome (2 degrees of freedom) vs. CTree's univariate (1 degree of freedom), leading to slightly lower power in this specific narrow case.
- LoA (Variance) heterogeneity: COAT significantly outperformed standard CTree when the covariate influenced the Limits of Agreement (variance) but not the bias. Standard CTree failed to detect these associations entirely.
- Complex Scenarios: In "Tree" scenarios with nested covariates affecting both bias and LoA, COAT achieved high ARI values (up to 0.9), accurately reconstructing the true subgroups as sample size increased.

B. Application: Cardiac Output (CO) Monitoring

The method was applied to data from 51 cardiac surgery patients comparing a non-invasive finger-cuff method (NiCCI) against a pulmonary artery catheter (reference).

Unpaired Analysis: Agreement was significantly better (lower bias and narrower LoA) for patients with lower CO measurements ( $\leq 6.548$ L/min).
Paired Analysis: Similar results were found, with better agreement at lower CO values.
Exploratory Findings: When relaxing significance thresholds to explore covariates, the analysis suggested potential differences based on sex and weight (e.g., better agreement in males $\leq 79$ kg), though these specific splits lacked strict statistical significance in the primary model.
Two-Sample Test: A test for vascular disease showed no significant difference in agreement ( $p=0.705$ ).

5. Significance and Implications

Clinical Relevance: The study demonstrates that method agreement is not universal. For cardiac output monitoring, the accuracy of non-invasive devices depends on the patient's physiological state (measurement magnitude) and potentially their demographics. This has direct implications for patient safety and device selection in specific subpopulations.
Statistical Advancement: COAT provides a rigorous, statistically sound framework for "subgroup analysis" in method comparison studies, moving beyond the assumption of homogeneity that plagues many clinical evaluations.
Practical Utility: By being implemented in an open-source R package, COAT makes advanced, covariate-dependent method agreement analysis accessible to clinical researchers, allowing them to determine for whom a new measurement technology is valid.

Conclusion

The paper introduces COAT as a theoretically grounded, flexible tool for analyzing method agreement with repeated measurements. It successfully addresses the limitation of homogeneous agreement assumptions, offering a way to detect and define subgroups where measurement bias or precision varies, thereby improving the reliability of clinical method comparison studies.