Technical Summary: LiNC (Lightweight Noise Correction)

Problem Statement

Label noise is a pervasive issue in medical imaging datasets, arising from inter-rater variability, annotation errors, and ambiguous clinical cases. This noise severely degrades the reliability and clinical effectiveness of machine learning models. While numerous methods exist for learning with noisy labels in natural images, medical imaging presents unique constraints: clean validation sets are expensive to acquire, data distributions shift frequently, and clinical workflows require interpretability regarding which labels are unreliable. Existing solutions often rely on multiple models, extensive hyperparameter tuning, or dataset-specific thresholds, making them difficult to integrate into resource-constrained clinical environments. The paper posits the question: Can label correction and noise detection be achieved "for free" during standard fine-tuning without requiring additional models or clean reference sets?

Methodology: LiNC

LiNC (Lightweight Noise Correction) addresses this by introducing a single trainable trust parameter ($\alpha_i$) for each training sample. The method operates within a standard training loop through three distinct phases:

1. Trust-Based Supervision

For each sample $i$, LiNC constructs a soft target distribution $q_i$ that interpolates between the model's current predictive distribution ($p_i$) and the observed label ($\tilde{y}_i$):
$$q_i(\alpha_i) = (1 - \alpha_i)\text{stopgrad}(p_i) + \alpha_i \mathbf{e}_{\tilde{y}_i}$$
Here, $\alpha_i \in (0, 1)$ controls the weight given to the observed label versus the model's prediction. The loss function is the cross-entropy between $q_i$ and $p_i$.

• When $\alpha_i = 1$, the method reverts to standard cross-entropy.
• When $\alpha_i = 0$, it becomes self-training using the model's prediction as the target.

2. Gradient-Driven Separation

The core theoretical insight (Theorem 3.1) is that the gradient of the loss with respect to $\alpha_i$ ($\partial L_i / \partial \alpha_i$) naturally separates clean and noisy samples during the early training phase (warm-up).

• Clean Samples: The model agrees with the observed label, resulting in a negative gradient that pushes $\alpha_i$ toward 1 (trusting the label).
• Noisy Samples: The model disagrees with the observed label, resulting in a positive gradient that pushes $\alpha_i$ toward 0 (trusting the model).
  This dynamic creates a separable distribution of trust values without access to ground-truth labels.

3. Three-Phase Training Schedule

LiNC executes a structured training schedule:

• Soft Warm-up: The model and trust parameters $\alpha_i$ are jointly optimized.
• Separation: After the warm-up, a 3-component Gaussian Mixture Model (GMM) is fitted to the learned $\alpha_i$ values. The components are ordered by their means and interpreted as:
  1. Noisy: Lowest mean (low trust).
  2. Ambiguous: Middle mean.
  3. Clean: Highest mean (high trust).
• Soft Correction: Samples identified as "noisy" are trained using the model's predictive distribution ($p_i$) as a soft target, while clean and ambiguous samples continue with the soft target interpolation.
• Hard Correction: Finally, the labels of the "noisy" cluster are replaced with the hard label ($\arg \max p_i$), and the model is trained using standard cross-entropy. Crucially, "ambiguous" samples are not relabeled, preventing over-correction of borderline cases.

Key Contributions

1. Per-Sample Trust Gradient: The authors derive a simple expression for the trust gradient, demonstrating how it induces separability between clean and noisy labels based on the model's agreement with the observed label.
2. Threshold-Free Separation: Instead of using hard-coded thresholds, LiNC employs an unsupervised 3-component GMM to separate samples into noisy, ambiguous, and clean categories. This explicitly models ambiguous cases, reducing the risk of correcting borderline samples that might actually be correct.
3. Lightweight Implementation: The method adds negligible asymptotic overhead. It requires only $O(N)$ additional memory (for the trust parameters) and maintains the $O(EN)$ runtime complexity of standard training. It does not require ensembles, second networks, or clean validation sets.

Experimental Results

The method was evaluated on ten 2D datasets from MedMNISTv2 under symmetric label noise ranging from 10% to 50%.

• Noise Detection: On the OrganSMNIST dataset with 20% noise, LiNC achieved an AUC of 0.9837 for detecting corrupted labels, significantly outperforming state-of-the-art baselines like VoG (0.9126) and AUM (0.8652).
• Accuracy Gains: LiNC consistently improved test accuracy across all noise levels. At 50% noise, the mean last-epoch accuracy across the ten datasets increased from 0.5882 (standard training) to 0.8023 (LiNC), a gain of 21.41 percentage points.
• Stability: Standard training exhibited significant performance degradation between the best and last epochs (e.g., a 23.06 point drop at 50% noise), whereas LiNC stabilized generalization, reducing this drop to only 3.56 points.
• Robustness: LiNC achieved higher last-epoch accuracy in 49 out of 50 dataset-noise combinations.

Significance and Claims

The paper claims that LiNC offers a practical, "plug-and-play" solution for noisy-label learning in medical imaging. Its significance lies in:

• Clinical Feasibility: By avoiding the need for clean validation sets, additional models, or complex tuning, it fits easily into existing clinical workflows.
• Interpretability: The learned trust scores provide an auditable signal. Low-trust cases can be flagged for expert re-review, and the "ambiguous" cluster identifies cases where diagnostic difficulty may exist rather than simple annotation error.
• Efficiency: It achieves state-of-the-art noise correction with minimal computational cost, making it scalable for large medical datasets.

The authors acknowledge limitations, noting that genuine clinical ambiguity (where multiple defensible labels exist) may be conflated with label noise, and that the method relies on the "early learning" behavior of deep networks. They suggest future work should focus on class-conditional trust distributions and mechanisms for human-in-the-loop review of low-confidence cases.