🤖 machine learning

XMix: Combating Extremely Noisy Labels via Local Smoothness in Self-Supervised Feature Space

XMix is a novel framework that leverages local smoothness in self-supervised feature spaces to estimate noise rates, select balanced clean samples, and generate reliable pseudo-labels, thereby significantly outperforming existing methods in handling extremely noisy labels without requiring prior noise knowledge.

Original authors: Chengqi Li, Yangdi Lu, Zhihao Shi, Wenbo He, Chamseddine Talhi, Nadjia Kara

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Chengqi Li, Yangdi Lu, Zhihao Shi, Wenbo He, Chamseddine Talhi, Nadjia Kara

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize animals. You show it thousands of pictures, but here's the catch: the labels on the pictures are written by a tired, distracted, or even mischievous human. Sometimes a picture of a cat is labeled "dog," and a picture of a car is labeled "airplane." This is the messy reality of "Learning with Noisy Labels." In the world of artificial intelligence, models usually need perfect data to learn, but in the real world, perfect data is rare and expensive. Scientists have been trying to build robots that can ignore the bad labels and learn from the good ones, but when the noise gets really high—like when 90% of the labels are wrong—most robots just give up or get confused.

The paper you're about to read tackles this exact problem. It introduces a new method called XMix. Think of it as a detective that doesn't just trust the written labels on the files. Instead, it looks at the pictures themselves to see which ones "look" like they belong together. If a picture of a cat looks very similar to other pictures of cats in the robot's memory, the detective assumes they are all cats, even if the labels say otherwise. This approach uses a concept called "local smoothness," which is a fancy way of saying: "Things that look alike usually belong in the same group." By using this logic, XMix tries to clean up the mess and teach the robot much better than previous methods, especially when the data is a total disaster.

The Problem: A Classroom Full of Pranksters

Imagine a classroom where a teacher is trying to teach students to identify different types of fruit. The teacher has a stack of flashcards, but a group of pranksters has swapped the labels. Some apples are labeled "bananas," and some oranges are labeled "grapes."

In the past, smart computer programs (called deep learning models) had a trick up their sleeve called the "memorization effect." They would study the cards and realize, "Hey, I'm really good at guessing the label for this specific card, but I'm terrible at guessing for that one." They assumed the ones they guessed well were the real ones (clean data) and the ones they struggled with were the pranksters' lies (noisy data).

But here's the trouble: when the pranksters go crazy and swap 90% of the labels, the computer gets confused. It can't tell the difference between a real apple and a banana labeled as an apple anymore. Also, the computer often ends up picking only one type of fruit to study, ignoring the others, which makes it bad at recognizing the whole variety. It needs a new strategy that doesn't rely on knowing exactly how many pranksters are in the room.

The Solution: XMix and the "Look-Alike" Rule

Enter XMix, the new detective. Instead of just looking at the written labels, XMix uses a special pair of glasses (a self-supervised feature encoder) to look at the visual details of the fruit. It knows that a red, round fruit with a stem looks a lot like other red, round fruits with stems.

Here is how XMix solves the three big problems:

1. Guessing the Noise Level Without a Cheat Sheet
Usually, these computer programs need to know exactly how many labels are wrong (e.g., "50% are wrong") to set their rules. If they guess wrong, they fail. XMix doesn't need this cheat sheet. It looks at a fruit and its closest "look-alike" neighbors in the image world. If the neighbors all have different labels, XMix calculates, "Wow, the noise must be really high here." It uses a math trick called "maximum likelihood" to estimate the noise level automatically. It's like a detective looking at a crime scene and saying, "Based on how many broken windows there are, I'd guess this was a riot," without needing a police report.

2. Finding More Good Students (Balanced & Expanded Selection)
When the noise is extreme, the old methods only find a few "clean" samples (good students) to study. XMix says, "Wait, if we found one good apple, and it looks exactly like these five other apples nearby, let's trust those five too!" It expands the group of trusted samples by including its neighbors.
Crucially, it also fixes the "class imbalance" problem. If the pranksters hid all the "banana" labels, the computer might stop studying bananas entirely. XMix notices this and says, "We need more bananas!" It looks even further out for banana look-alikes to make sure every fruit gets a fair chance to be studied. This ensures the robot learns a balanced diet of knowledge, not just one favorite fruit.

3. Better Guessing for the Unknown
Finally, for the cards it still isn't sure about (the noisy ones), XMix doesn't just guess randomly. It asks the neighbors: "What do you think this is?" It takes the opinions of the most trusted neighbors (the clean ones) and averages them to create a "pseudo-label" (a best-guess label). This is like asking a group of experts to vote on a mystery object. Because the experts are chosen based on how much they look like the object, their vote is much more reliable than a random guess.

What They Found: Beating the Odds

The researchers tested XMix on famous image datasets like CIFAR-10 and CIFAR-100, which contain pictures of cars, airplanes, animals, and more. They deliberately messed up the labels to create extreme scenarios:

90% Symmetric Noise: Imagine 90 out of 100 labels are completely random.
98% Noise: Almost everything is a lie.

In these brutal conditions, previous methods (like DivideMix and ProMix) started to crumble. For example, on CIFAR-10 with 90% noise, the old best method (DivideMix) only got about 76% accuracy. XMix, however, boosted this to 91.2%. On CIFAR-100 with 95% noise, the old method got 19.1%, while XMix reached 31.4%.

The paper shows that XMix doesn't just work a little bit better; it shines when the situation is most desperate. It successfully identified many more "true clean" samples than the old methods, sometimes doubling or tripling the number of good examples the robot could learn from.

The Verdict

XMix proves that you don't need to know the exact level of chaos in your data to clean it up. By trusting the visual similarities between images—using the "local smoothness" of the feature space—it can automatically adjust its strategy, find more good data, and teach the robot to ignore the noise.

The authors suggest that this method is a significant step forward, especially for real-world scenarios where data is messy and we don't have a manual telling us how bad it is. While it's not a magic wand that fixes everything perfectly (it still makes a few mistakes, especially when the noise is low and the expansion isn't needed), it consistently outperforms the current state-of-the-art methods in the most challenging, high-noise environments. It turns a chaotic classroom of pranksters into a place where learning can still happen.

Technical Summary: XMix

Problem Statement

Supervised deep learning models depend heavily on large, accurately labeled datasets. However, annotation errors are inevitable due to human limitations and automated systems, leading to the "Learning with Noisy Labels" (LNL) problem. While recent state-of-the-art methods address this via sample selection strategies (leveraging the memorization effect to filter clean data for semi-supervised learning), they face significant limitations:

Extreme Noise: They struggle when noise levels are very high (e.g., >80%), often failing to identify a sufficiently large clean subset.
Class Imbalance: Selected clean samples often become imbalanced across classes, as some classes are easier to learn than others.
Dependency on Prior Knowledge: Many methods require careful hyperparameter tuning or prior knowledge of the noise rate/distribution to remain effective.
Information Neglect: Existing approaches rely on the joint distribution of images and noisy labels, often ignoring rich, noise-agnostic information intrinsic to the images themselves (e.g., visual similarities).

Methodology: XMix

The authors propose XMix, a framework that leverages local smoothness in the self-supervised feature space to enhance all stages of the sample selection process without relying on potentially corrupted labels. The method utilizes a pretrained self-supervised encoder (DINOv2) to extract features and identify $k$ -nearest neighbors for each sample.

The framework operates through three primary mechanisms integrated into a standard sample selection baseline (e.g., DivideMix or ProMix):

1. Class-Wise Noise Rate Estimation

Instead of requiring prior noise knowledge, XMix estimates the noise rate adaptively:

Mechanism: It assumes that samples close in feature space share the same true label (smoothness assumption). It calculates the empirical probability of label matching among a sample's nearest neighbors.
Correction: Since observed labels are noisy, the raw matching rate is biased. The authors fit a quadratic correction model (calibrated on synthetic data) to adjust for this bias.
Usage: The resulting estimate is not intended to be a precise recovery of the true noise rate but is accurate enough to categorize the dataset into discrete noise regimes (low, moderate, high, extreme). This categorization automatically guides the selection of hyperparameters (thresholds and selection ratios) for the baseline model.

2. Balanced and Expanded Sample Selection

XMix expands the set of identified clean samples beyond the standard low-loss criterion:

Expansion: If a sample is identified as clean (low loss), its feature-space neighbors ( $G^k_i$ ) that share the same observed label are also added to the clean set ( $D^+_{clean}$ ).
Balancing: To address class imbalance, the method applies a different rule for underrepresented classes. For these classes, it considers $2k$ nearest neighbors instead of $k$ , allowing for the recovery of more examples from a broader neighborhood.
Effect: This strategy increases the volume of clean samples available for semi-supervised learning, particularly under extreme noise where standard methods fail to find enough clean data.

3. Nearest Neighbor Pseudo-Labeling

During the semi-supervised learning phase, XMix generates more reliable pseudo-labels for the unlabeled (noisy) data:

Mechanism: Instead of relying solely on the model's prediction, it aggregates predictions from the feature-space neighbors of an unlabeled sample.
Weighting: Predictions from neighbors identified as "clean" are weighted higher (weight = 2) than those from neighbors still considered "noisy" (weight = 1).
Integration: These weighted pseudo-labels are used in a MixMatch-style training loop, blending labeled and pseudo-labeled data to improve robustness.

Key Contributions

The paper outlines four main contributions:

Class-Wise Noise Rate Estimation: An adaptive method to estimate noise levels within feature neighborhoods using maximum likelihood, enabling automatic hyperparameter adjustment without prior noise knowledge.
Balanced and Expanded Sample Selection: A strategy to expand the clean sample set using feature neighbors, recovering more clean samples under extreme noise and mitigating class imbalance.
Nearest Neighbor Pseudo-Labeling: A mechanism to generate more accurate pseudo-labels by leveraging trustworthy local information in the feature space.
Empirical Validation: Demonstrations that XMix consistently boosts the performance of sample selection baselines across multiple LNL benchmarks, with particularly large improvements under severe label noise.

Experimental Results

The authors evaluated XMix on both synthetic and real-world noisy datasets (CIFAR-10, CIFAR-100, CIFAR-10N, CIFAR-100N, ANIMAL-10N, Mini-WebVision).

Extreme Noise Performance: On CIFAR-10 and CIFAR-100 with symmetric noise rates of 90% and 95%, XMix significantly outperforms baselines like DivideMix and UNICON. For instance, on CIFAR-10 with 90% noise, XMix (ProMix+) achieved 92.7% accuracy compared to DivideMix's 51.3%.
Real-World Benchmarks: XMix achieved state-of-the-art results on ANIMAL-10N (90.3% accuracy) and competitive results on Mini-WebVision and ILSVRC12.
Ablation Studies:
- Removing the Balanced & Expanded (B&E) selection or Nearest Neighbor (NN) pseudo-labeling led to noticeable performance degradation, especially under high noise.
- The performance gains were attributed primarily to the feature-space clustering and filtering mechanisms rather than the specific strength of the encoder (DINOv2 vs. SimCLR).
- The paper notes that at lower noise levels, the benefit of expansion is marginal, and the method may occasionally introduce false positives, though the overall impact remains positive.

Significance and Claims

The paper claims that XMix offers a simple yet robust solution to the LNL problem. Its primary significance lies in its ability to:

Operate effectively without prior knowledge of the noise level or distribution.
Systematically enhance sample selection by exploiting noise-agnostic visual patterns (local smoothness) rather than relying solely on loss values.
Provide superior performance in scenarios with extreme label noise where existing methods fail to converge or select sufficient clean data.
Be easily compatible with existing sample selection frameworks, acting as a plug-in enhancement to improve their resilience and balance.

The authors emphasize that while the noise rate estimation is a "coarse" guide for hyperparameter selection rather than a precise recovery of ground truth, it is sufficiently accurate to place datasets into the correct operational regimes, thereby removing a major bottleneck in current LNL research.