📄 radiology and imaging

Agreement of an AI tool for joint space width measurement in radiographic knee osteoarthritis: data from the LOSEIT trial

This study presents a secondary analysis of the LOSEIT trial data designed to evaluate the agreement and equivalence between a commercially available AI tool and expert radiologist measurements of joint space width in weight-bearing, fixed-flexion knee radiographs of patients with osteoarthritis.

Original authors: Mayar, S., Henriksen, M., Christensen, R., Hansen, P., Bliddal, H., Nybing, J. U., Nielsen, C. T., Gudbergsen, H., Boesen, M. P., Brejnbol, M. W.

Published 2026-06-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mayar, S., Henriksen, M., Christensen, R., Hansen, P., Bliddal, H., Nybing, J. U., Nielsen, C. T., Gudbergsen, H., Boesen, M. P., Brejnbol, M. W.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The Big Picture: A "Double-Check" for Knee X-Rays

Imagine you have a very expensive, high-tech robot (the AI tool) that claims it can measure the tiny gap between the bones in your knee just as well as a highly trained human doctor (the radiologist).

This paper isn't about testing a new drug or a new treatment. Instead, it is a quality control check. The researchers want to know: If the robot measures the gap, will it get the same answer as the human expert?

They are using data from a previous study called LOSEIT, where people with knee pain had their knees X-rayed at the start and then again a year later.

The Problem: Measuring the Invisible Gap

Think of the space between your knee bones (the Joint Space) like the air gap between two shelves. In a healthy knee, there is a nice cushion of cartilage keeping them apart. In osteoarthritis, that cushion wears down, and the gap gets smaller.

The Challenge: Measuring this gap on an X-ray is tricky. It's like trying to measure the width of a shadow with a ruler. If you tilt your head slightly or move the light source, the shadow looks different.
The Human Way: Traditionally, a doctor looks at the X-ray and draws lines to measure this gap. But humans get tired, and two different doctors might measure the same picture slightly differently.
The Robot Way: The new AI tool uses a computer program to draw those lines automatically. It never gets tired and is supposed to be perfectly consistent.

The Goal: The "Tie-Breaker" Test

The researchers are setting up a contest between the AI Robot and the Human Doctors.

The Reference Standard (The Judges): To decide who is "right," they aren't just using one doctor. They are using two resident doctors to measure every X-ray independently.
- If the two doctors agree (their measurements are within 0.4mm of each other), they take the average.
- If they disagree too much, a senior expert doctor (the "Head Judge") steps in to make the final call.
- Analogy: Think of this like a sports referee. If two line judges disagree on whether a ball was "in" or "out," the head referee makes the final decision.
The Index Test (The Contestant): The AI tool measures the exact same X-rays.
The Comparison: They compare the change in the gap size. Did the gap get smaller over the year?
- If the AI says the gap shrank by 0.5mm, and the Human Judges say it shrank by 0.5mm, they agree.
- If the AI says 0.1mm and the Humans say 0.9mm, they disagree.

The Rules of the Game (The "Green Zone")

The researchers have set a specific rule for what counts as "good enough."

They have drawn an invisible Green Zone around the human measurements. This zone is 0.5mm wide (plus or minus).
The Goal: They want to prove that the AI's measurements fall inside this Green Zone.
The Test: They will use a special statistical method (called Bland-Altman) to draw a map of the differences.
- If the "cloud" of differences between the AI and the Humans stays safely inside the Green Zone, the AI passes the test.
- If the cloud spills outside the Green Zone, the AI is considered too unreliable for this specific job.

Why Do This?

The paper explains that while AI is fast and consistent, it can sometimes be "hallucinated" or biased if it was only trained on data from one specific hospital. This study is an external validation, meaning they are testing the AI on real-world data from a different setting to see if it actually works as advertised.

What They Will Report

At the end of the study, they will produce a report (Table 2 in the paper) that looks like a scoreboard:

The Average Difference: On average, how far off is the AI from the humans?
The Limits: What is the worst-case scenario? (e.g., "The AI might be off by up to 1mm in the worst case").
The Verdict: Did the AI stay inside the Green Zone?

Summary

In short, this paper is a stress test for a new AI tool. The researchers are asking: "Can this robot measure the shrinking gap in a knee X-ray as accurately as a team of expert human doctors?" If the robot passes the test (stays within the 0.5mm margin), it could eventually help doctors diagnose knee arthritis faster and more consistently. If it fails, we know we can't trust it yet.

Technical Summary: Agreement of an AI Tool for Joint Space Width Measurement in Radiographic Knee Osteoarthritis

Problem and Rationale
Knee osteoarthritis (KOA) is a leading cause of disability, with diagnosis and progression monitoring often relying on radiographic features. While the Kellgren-Lawrence grade is standard for eligibility, minimum joint space width (mJSW) is a critical structural biomarker used as an endpoint for disease progression and is currently one of the few imaging measures accepted in regulatory guidance for drug development. However, mJSW is an indirect measure sensitive to technical factors such as X-ray beam angulation and knee positioning. To mitigate this, clinical trials like the LOSEIT trial utilize fixed-flexion devices.

Historically, extracting these features has required manual annotation by radiologists, a process that is time-consuming and subject to interobserver variance. While artificial intelligence (AI) tools utilizing deep neural networks offer the potential for automated, consistent measurements, their performance in real-world, multi-center settings is not guaranteed. Previous studies suggest that AI models may perform worse on data from hospitals different from their training sites, highlighting the necessity for external validation before clinical or regulatory reliance.

Methodology
This study is a secondary analysis of data from the LOSEIT trial, a randomized, double-blind, placebo-controlled trial investigating the effects of liraglutide versus placebo on KOA following a weight-loss period. The analysis focuses on the agreement between a commercially available AI tool (RBknee-FDA) and a "best practice" radiologist reference standard for measuring joint space width (JSW) in tibiofemoral radiographs acquired at enrollment and at 52 weeks (60 weeks from enrollment).

Reference Standard: The reference JSW was established by two independent resident radiologists who annotated bony femoral and tibial surfaces using a digitizer. If their measurements differed by less than 0.40 mm, the average was used. If the difference exceeded 0.40 mm, the case was re-annotated; if the discrepancy persisted, a senior musculoskeletal radiologist arbitrated the final value.
Index Test: The AI tool (RBknee-FDA) analyzed the same radiographs locally as a containerized process, outputting medial and lateral minimal JSW (mmJSW, lmJSW) and fixed-location JSW (mfJSW, lfJSW).
Statistical Framework: The study employs an equivalence framework using the Two One-Sided Tests (TOST) approach. The primary analysis assesses the agreement of the change in JSW (follow-up minus enrollment) between the AI and the reference.
Equivalence Criteria: The equivalence margin ( $\delta$ ) is set at 0.5 mm. Agreement is considered established if the upper limit of the 95% confidence interval (CI) for the upper Limit of Agreement (LoA) and the lower limit of the 95% CI for the lower LoA both fall within the $\pm 0.5$ mm margins.
Supplementary Analyses: These include mixed-effects Bland-Altman analyses at individual time points, an analysis excluding "bone-on-bone" configurations (where mJSW = 0 mm), a "naïve" approach without mixed-effects components, and sensitivity analyses with varying equivalence margins (0.35 mm, 0.70 mm, and 0.85 mm).

Key Contributions and Planned Outputs
The paper outlines a rigorous Statistical Analysis Plan (SAP) designed to validate the performance of an off-the-shelf AI tool against a high-quality human reference standard in a controlled trial setting. Key contributions include:

External Validation: Providing an independent assessment of a commercially available AI tool on data from a specific clinical trial context, addressing the gap between regulatory approval and real-world performance.
Equivalence Testing: Moving beyond simple correlation to test for statistical equivalence in the change of structural biomarkers, which is the critical metric for monitoring disease progression.
Robust Reference Standard: Utilizing a multi-rater adjudication process to minimize human error in the reference standard.
Comprehensive Metrics: Evaluating not only minimal JSW but also fixed-location JSW, and specifically addressing the challenge of "bone-on-bone" cases where AI performance is historically weaker.

Results
As this document is a Statistical Analysis Plan (SAP) written prior to the execution of the analysis (dated June 2026), it does not contain empirical results. The paper details the planned methods for generating results, including the specific statistical models (Bland-Altman, mixed-effects), the criteria for establishing equivalence, and the structure for reporting data (e.g., Table 2 for mean bias and LoA). The anticipated output is a determination of whether the AI tool's measurements of JSW change are statistically equivalent to those of the expert radiologist panel within the predefined 0.5 mm margin.

Significance and Claims
The paper positions this analysis as a necessary step in the validation pipeline for AI in radiology. It claims that while AI tools offer efficiency and reproducibility, regulatory approval does not ensure uniform performance across different datasets. By establishing agreement (or lack thereof) in a rigorous trial setting, the study aims to determine if such tools can be reliably used to measure structural progression in KOA. The authors modestly frame the work as an evaluation of agreement rather than a definitive proof of clinical utility, noting that the study follows an equivalence framework to ensure that any differences between the AI and human readers are clinically negligible. The successful validation of such a tool could potentially free up radiologist time and provide more consistent measurements for future KOA trials and drug development.