💻 computer science

Learning Coarse-to-Fine Osteoarthritis Representations under Noisy Hierarchical Labels

This paper demonstrates that a simple dual-head deep learning model leveraging the natural hierarchy between coarse binary osteoarthritis presence and fine-grained Kellgren-Lawrence severity grades can effectively reshape disease representations, yielding improved grading accuracy, more ordered latent structures, and better anatomical alignment of saliency maps compared to single-task approaches.

Original authors: Tongxu Zhang

Published 2026-05-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tongxu Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to look at an X-ray of a knee and decide two things:

The Big Picture: Is there arthritis here or not? (Yes/No)
The Fine Details: If there is arthritis, how bad is it? (On a scale of 0 to 4, where 0 is healthy and 4 is severe).

In the medical world, these two questions are naturally connected. You can't really have a "severe" case without first having "arthritis." However, most computer programs (AI models) have been trained to treat these as two completely separate jobs. They either learn just to say "Yes/No" or they try to guess the specific number (0–4) directly.

The problem with guessing the number (0–4) directly is that it's a noisy, tricky task. Even human doctors sometimes disagree on whether a knee is a "2" or a "3." It's like trying to guess the exact temperature when the thermometer is slightly broken.

The "Two-Headed" Experiment

The author of this paper, Tongxu Zhang, asked a simple question: What if we let the computer learn both jobs at the same time, using the "Yes/No" answer to help guide the "How bad?" answer?

To test this, they didn't build a super-complex new machine. Instead, they used a very simple setup: a shared brain (the part that looks at the image) with two different heads (one for the Yes/No question, one for the severity scale).

Think of it like a student studying for two exams:

Exam A (The Coarse Label): A simple True/False quiz. "Is the knee sick?" This is easy and reliable.
Exam B (The Fine Label): A difficult essay question. "Rate the sickness from 1 to 5." This is hard and prone to mistakes.

The researchers tested three study strategies:

Study only Exam A: The student learns to spot sickness but ignores the severity.
Study only Exam B: The student tries to memorize the difficult severity scale without the safety net of the simple True/False check.
Study Both (Dual-Head): The student learns the easy True/False rule first, then uses that solid foundation to figure out the tricky severity scale.

What They Found

The results were interesting and depended on which "student" (computer architecture) was doing the learning.

1. The "Smart" Students (ResNet3D and M3T)
For some computer models, studying both exams together worked wonders.

Better Grades: They got better at predicting the severity scale (the hard part) than when they tried to learn it alone.
Better Organization: Inside the computer's "brain," the data started to organize itself in a logical way. It wasn't just a messy pile of numbers; the computer learned to separate "sick" from "healthy" first, and then neatly arranged the "sick" knees from "mild" to "severe" in a line.
Better Focus: When the researchers looked at where the computer was looking in the image, these models started focusing more on the cartilage (the cushion in the knee). This is the part of the knee doctors care about most. It's as if the "Two-Headed" approach taught the computer to look at the right spot, not just guess randomly.

2. The "Confused" Student (nnMamba)
For one specific type of computer model, the "Two-Headed" approach didn't help. In fact, it sometimes made things worse. This suggests that not every computer brain learns the same way; some need the extra help, while others get confused by it.

The Big Takeaway

The main point of this paper isn't that we have a new, perfect medical tool ready for hospitals tomorrow. Instead, it's a discovery about how computers learn.

The paper shows that when you have a "coarse" (easy) label and a "fine" (noisy/hard) label, teaching them together can act like a training wheel. The easy task stabilizes the learning, helping the computer build a better internal map of the disease. This map is not only more accurate but also makes more sense to human doctors because it focuses on the right parts of the body.

In short: By teaching the computer to answer the simple question ("Is it sick?") alongside the hard question ("How sick?"), we helped it understand the disease better, organize its thoughts more logically, and look at the right parts of the knee. But, this trick only works if the computer's "brain" is the right kind to begin with.

1. Problem Statement

Knee osteoarthritis (OA) assessment presents a unique challenge in medical image analysis due to the nature of its clinical labels:

Hierarchical Structure: OA assessment naturally follows a coarse-to-fine hierarchy. It begins with a coarse binary decision (OA present vs. absent) and proceeds to a fine-grained ordinal severity grade using the Kellgren–Lawrence (KL) system (grades 0–4).
Label Noise and Uncertainty: The KL grade is semi-quantitative, ordinal, and subject to significant inter- and intra-observer variability, making it a noisy target. Conversely, the binary OA label is more stable but lacks structural detail.
Current Limitations: Existing deep learning approaches typically treat these as separate, isolated classification tasks (either binary OA or multi-class KL). This separation fails to leverage the complementary supervisory signals: the stability of the coarse label and the structural detail of the fine label. Furthermore, optimizing solely on noisy KL labels may force models to learn from ambiguous boundaries, while binary-only training collapses subtle severity variations.

Core Question: Can the clinical label hierarchy itself serve as a representation-level supervisory prior to organize the latent space into a structured, coarse-to-fine disease representation, rather than just improving scalar prediction metrics?

2. Methodology

The authors propose a deliberately simple dual-head architecture to probe the effects of hierarchical supervision without introducing complex architectural changes.

Architecture:
- Shared Encoder: A backbone network (3D ResNet, M3T Transformer, or nnMamba) extracts features $z = f_\theta(x)$ .
- Dual Heads: Two parallel task-specific heads are attached to the shared encoder:
  1. OA Head: Predicts the binary OA label ( $y_{OA} \in \{0, 1\}$ ).
  2. KL Head: Predicts the ordinal KL grade ( $y_{KL} \in \{0, 1, 2, 3, 4\}$ ).
- Training Objective: The total loss is a weighted sum of binary cross-entropy (for OA) and standard cross-entropy (for KL):
  $L = \lambda_{OA} L_{OA}(\hat{y}_{OA}, y_{OA}) + \lambda_{KL} L_{KL}(\hat{y}_{KL}, y_{KL})$
- Hierarchy Definition: The binary OA label is derived from the KL grade ( $y_{OA} = \mathbb{I}(y_{KL} \geq 2)$ ), establishing a strict hierarchical relationship.
Experimental Design:
- Dataset: OAIZIB-CM (Knee MRI), with 383 training and 98 test subjects.
- Comparisons: The study compares three settings across three different 3D backbones:
  1. Single-OA: Only the binary head.
  2. Single-KL: Only the KL head.
  3. Dual-Head: Both heads trained simultaneously.
- Evaluation Metrics:
  - Predictive: AUC, Accuracy, F1-score.
  - Latent Geometry: Principal Component Analysis (PCA) to measure the alignment of the first principal component (PC1) with severity (Spearman correlation $\rho(PC1, KL)$ ) and the monotonicity of KL class centroids.
  - Anatomical Plausibility: Saliency–cartilage overlap (mass@ROI, top1@ROI, Dice@5/10) to verify if the model focuses on clinically relevant cartilage regions.

3. Key Contributions

Formulation: Reframed knee OA assessment as a hierarchical representation-learning problem, treating the coarse/fine label structure as an inductive bias for latent space organization.
Representation Probe: Utilized a simple dual-head model to systematically demonstrate that hierarchical supervision can reshape disease representations, not just improve classification scores.
Beyond Accuracy: Introduced a multi-faceted evaluation framework combining paired statistical testing, latent severity-axis geometry analysis, and anatomical plausibility checks (saliency overlap).
Architecture Dependence: Provided evidence that the benefits of hierarchical supervision are backbone-dependent, highlighting that not all architectures respond equally to this inductive bias.

4. Key Results

Predictive Performance:
- ResNet3D: The Dual-head model showed no significant improvement in binary OA metrics over Single-OA but achieved statistically significant gains in KL grading (Macro-AUC +0.12, Accuracy +0.21, Macro-F1 +0.21) compared to Single-KL.
- M3T: Dual-head improved OA AUC significantly and showed moderate gains in KL metrics.
- nnMamba: Showed no significant gains or even slight degradation, indicating that the benefit is not universal.
Latent Geometry (Severity Axis):
- For ResNet3D and M3T, the Dual-head approach induced a latent space where the primary axis (PC1) was strongly correlated with both KL severity and OA status. The KL class centroids were arranged in a more ordered, monotonic fashion.
- For nnMamba, the Dual-head setup resulted in a negative correlation between PC1 and KL labels, suggesting the latent organization was misaligned with the clinical hierarchy.
Anatomical Plausibility:
- ResNet3D and M3T Dual-head models exhibited stronger saliency overlap with cartilage regions compared to their single-task counterparts. This suggests that hierarchical supervision helps the model focus on clinically relevant anatomical structures.
- nnMamba showed a decline in cartilage overlap under Dual supervision, consistent with its lack of performance gains.

5. Significance and Conclusion

Inductive Bias: The study demonstrates that hierarchical supervision acts as a powerful structural prior. It stabilizes the learning of fine-grained severity by anchoring it to a robust coarse signal, preventing the model from collapsing into a single disease-presence axis or getting lost in noisy ordinal boundaries.
Interpretability: The method improves not just "black-box" accuracy but also the interpretability of the model. Better representations correlate with stronger anatomical alignment (focus on cartilage), which is crucial for clinical trust.
Architecture Sensitivity: A critical takeaway is that hierarchical supervision is not a "one-size-fits-all" solution. Its effectiveness depends heavily on the backbone architecture's capacity to utilize the shared representation.
Future Direction: The work suggests that future medical image analysis should move beyond flat classification benchmarks and consider the geometric organization of latent spaces and anatomical plausibility when designing training strategies for noisy, hierarchical clinical labels.