💻 computer science

CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels

The paper proposes CARE, a parameter-efficient framework that utilizes a class-adaptive expert consensus mechanism leveraging vision-language model signals to effectively address the compound challenges of long-tailed distributions and noisy labels by applying stricter agreement for tail classes and more permissive agreement for head classes.

Original authors: Mengke Li, Haiquan Ling, Lihao Chen, Yang Lu, Yiqun Zhang, Hui Huang

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mengke Li, Haiquan Ling, Lihao Chen, Yang Lu, Yiqun Zhang, Hui Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a new student (an AI model) how to identify animals from a giant photo album. But there are two big problems with this album:

The "Long Tail" Problem: The album is full of pictures of dogs and cats (the "head" classes), but there are only a few, blurry pictures of rare animals like the okapi or the pangolin (the "tail" classes).
The "Noisy Label" Problem: The person who wrote the names on the photos made mistakes. They labeled a dog as a cat, or a rare pangolin as a common squirrel.

Most previous methods tried to fix these problems separately. They either tried to balance the number of photos or tried to find and fix the wrong names. But the paper argues that doing this "one-size-fits-all" approach fails. It's like using the same strict rulebook for a student who knows everything about dogs and a student who is seeing a pangolin for the first time. The strict rules confuse the beginner, while the loose rules let the expert make careless mistakes.

The Solution: CARE (Class-Adaptive Rectification with Experts)

The authors propose a new method called CARE. Think of CARE not as a single teacher, but as a panel of three expert judges who vote on what is in each photo.

Here is how the three "experts" work:

The Text Expert: This judge has read every book about animals. It looks at the photo and says, "Based on the description of a 'pangolin', this looks like a pangolin." It uses language knowledge to guess the label.
The Image Expert: This judge is a visual artist who has studied thousands of photos. It looks at the shapes and colors and says, "Visually, this matches a pangolin."
The Original Label Expert: This is the original, flawed label written on the photo. Even though it's often wrong, it still has some useful hints, especially for common animals like dogs.

The Secret Sauce: "Class-Adaptive Consensus"

The magic of CARE isn't just having three judges; it's how they vote.

For Common Animals (The "Head"): There are thousands of dog photos. The judges can be a bit more relaxed. If the Text Expert and Image Expert agree it's a dog, they accept it, even if the Original Label said "cat." They don't need to be 100% sure because there is so much data to back them up.
For Rare Animals (The "Tail"): There are only a few pangolin photos. These are tricky. If the Original Label says "squirrel," the Text and Image experts need to be extremely confident and agree with each other to override that label. If they are even slightly unsure, CARE keeps the original label or treats it with caution.

The Analogy:
Imagine a town meeting to decide the name of a new street.

Head Classes (Common): If 100 people say "Main Street," and one person says "Oak Street," we trust the 100. We don't need a super strict vote count.
Tail Classes (Rare): If only 3 people are in the room, and 2 say "Oak Street" but the 3rd (the original label) says "Main Street," we can't just go with the majority. We need to be very careful. Maybe the 3rd person is right, or maybe the 2 are right. CARE says, "Let's only change the name if the 2 experts are shouting in agreement. If they are just whispering, we stick with the original or wait for more proof."

Why This Works Better

Previous methods tried to fix all the wrong names equally. This often made things worse for the rare animals because the AI would get confused by the noise and stop learning about them entirely.

CARE fixes this by:

Filtering out the noise: It ignores the "squirrel" label on the pangolin photo because the Text and Image experts strongly disagree with it.
Protecting the rare: It doesn't aggressively change the labels of the few rare photos unless the evidence is overwhelming. This prevents the AI from "over-correcting" and losing the little bit of truth it had.

The Results

The paper tested this on synthetic data (made-up noisy datasets) and real-world messy data (like photos from the internet).

The Outcome: CARE consistently beat the best existing methods.
The Gain: It improved accuracy by up to 3.0%. In the world of AI, that's a huge jump.
The Proof: When they looked at the "noise rates" (how many wrong labels were left), CARE managed to clean up the rare classes much better than other methods, which usually left the rare classes full of errors.

Summary

CARE is a smart system that knows when to be strict and when to be flexible. It uses the combined wisdom of text, images, and original labels to clean up messy data. Crucially, it treats rare items with extra care, ensuring that the AI learns to recognize the "long tail" of rare things without getting confused by the noise. It's a "plug-and-play" tool that makes AI more reliable when learning from imperfect, real-world data.

Technical Summary: CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels

1. Problem Definition

Real-world machine learning applications frequently face a compound challenge: long-tailed class distributions (where "tail" classes have few samples and "head" classes have many) co-occurring with noisy annotations (label errors). Existing approaches typically address these issues in isolation. Long-tail learning methods often assume clean labels, potentially amplifying mislabeling in scarce tail classes. Conversely, noisy-label learning methods often assume balanced classes, leading to the discarding of valuable tail samples or over-regularization of head classes.

A critical observation made by the authors is that class-agnostic label correction is insufficient. Standard methods often apply uniform noise rates or correction strategies across all classes. This fails to account for the reality that tail classes often exhibit higher noise levels due to data scarcity and annotation difficulty. Applying uniform correction can lead to:

Insufficient correction for tail classes, leaving them with high noise rates and poor performance.
Over-regularization for head classes, where valuable information is discarded or over-corrected.
Degraded overall performance, as evidenced by experiments where reducing the global noise rate did not translate to accuracy gains, particularly on tail classes.

2. Methodology: CARE Framework

The authors propose Class-Adaptive Rectification with Experts (CARE), a parameter-efficient framework designed to robustly learn from long-tailed noisy data without introducing additional trainable parameters for the correction mechanism itself. CARE leverages three complementary supervision sources derived from Vision-Language Models (VLMs), specifically CLIP:

2.1. Three Expert Sources

Text Expert (TE): Utilizes a frozen CLIP text encoder to generate semantic priors. It embeds class labels into a shared visual-semantic space, providing confidence scores based on text-image alignment without fine-tuning.
Image Expert (IE): Uses a CLIP image encoder fine-tuned via AdaptFormer (a parameter-efficient method) for the specific downstream task. This provides visual-semantic confidence scores tailored to the target domain.
Base Expert (BE): Relies on the original observed (noisy) labels. While potentially unreliable, these labels retain valuable class-indicative information, especially for head classes with abundant data.

2.2. Class-Adaptive Expert Consensus

The core innovation of CARE is a class-aware consensus mechanism that dynamically adjusts the correction strategy based on class frequency ( $n_c$ ).

Adaptive Top-K Voting: Instead of a global threshold, CARE selects the top- $K_c$ $K_{c}$ predictions from the Text and Image experts for each class $c$ $c$ . The value of $K_c$ $K_{c}$ is proportional to the class frequency ( $K_c \propto n_c$ $K_{c} \propto n_{c}$ ).
- Head Classes (High Frequency): Assigned a larger $K$ . This allows for more permissive agreement, preserving high-confidence labels and avoiding over-filtering.
- Tail Classes (Low Frequency): Assigned a smaller $K$ . This enforces stricter consensus, requiring stronger agreement between experts to validate a label, thereby suppressing noise-induced interference in data-scarce categories.
Frequency Accumulation: The framework aggregates confidence scores from all three experts (TE, IE, BE) into a cumulative frequency distribution $F_c$ $F_{c}$ .
- If the observed label appears in an expert's Top- $K$ list, that expert's confidence reinforces the label.
- If the observed label is not in the Top- $K$ , the expert contributes only its Top- $K$ predictions, preventing the reinforcement of potentially noisy labels.
Label Rectification: The corrected label for a sample is determined by the class with the highest accumulated frequency. This process iteratively refines the label distribution, reducing noise rates and approximating the intrinsic clean distribution.

2.3. Integration with Long-Tail Calibration

Once labels are rectified, CARE integrates with standard long-tail learning techniques, such as Logit Adjustment (LA), using the newly estimated class priors derived from the rectified labels. This ensures that the model is calibrated to the corrected, more accurate distribution.

3. Key Contributions

Identification of a Critical Gap: The authors demonstrate that merely lowering the global noise rate via class-agnostic methods fails to improve performance under long-tailed distributions. They highlight that tail classes require specific, stricter correction strategies to avoid performance degradation.
Parameter-Efficient Framework: CARE unifies noisy label correction and long-tail calibration without adding learnable parameters to the correction module. It leverages pre-trained VLMs and a lightweight fine-tuning strategy (AdaptFormer).
Class-Aware Consensus Mechanism: The proposal of a dynamic Top- $K$ consensus that adapts to class frequency. This theoretically balances the need for strict noise suppression in tail classes with the need to preserve signal in head classes.
Empirical Validation: Extensive experiments on synthetic (CIFAR-100-LTN) and real-world (WebVision-50, Food101N, ImageNet-LTNr) benchmarks demonstrate consistent outperformance over state-of-the-art methods.

4. Experimental Results

The paper reports significant performance gains across various noise types (symmetric, asymmetric, joint) and imbalance factors (IF = 10, 50, 100).

Synthetic Benchmarks (CIFAR-100-LTN):
- Under joint noise with IF=100 and NR=50%, CARE achieves 76.7% accuracy, surpassing CLIP+LA by 1.4% and CLIP+RLA by 10.0%.
- Under symmetric noise (IF=10, NR=60%), CARE reaches 79.2%, outperforming CLIP+LA by 3.2%.
Real-World Benchmarks:
- WebVision-50: CARE achieves 85.3% top-1 accuracy, the highest among compared methods.
- Food101N: CARE outperforms CLIP+RLA by a large margin (e.g., +6.9% at IF=100) and improves upon CLIP+LA by up to 0.4%.
- ImageNet-LTNr: CARE consistently improves performance over CE and LA baselines under both moderate and severe imbalance.
Ablation Studies:
- Removing any single expert (TE or IE) results in no improvement, confirming that collaborative consensus is crucial.
- The method effectively reduces noise rates across all class splits, particularly for tail classes (reducing tail noise from 72.8% to 53.3% in one experiment).
- The framework generalizes across different backbones (ResNet, MLP-Mixer) and VLMs (BLIP), indicating its robustness is driven by the consensus mechanism rather than a specific backbone architecture.

5. Significance and Claims

The paper claims that CARE addresses a fundamental limitation in current learning paradigms: the assumption that noise is uniform or that correction strategies should be class-agnostic. By introducing a class-adaptive expert consensus, CARE effectively disentangles intrinsic uncertainty (due to data scarcity) from annotation noise.

The authors emphasize that their approach:

Enhances Reliability: It produces more reliable label distributions, particularly for under-represented tail classes, which are often the bottleneck in real-world applications.
Improves Generalization: By preventing over-regularization of head classes and under-correction of tail classes, CARE yields more balanced supervision and better overall model generalization.
Practical Applicability: The method is "plug-and-play," compatible with various backbones and existing long-tail learning losses, making it a practical solution for learning from imperfect, real-world data where high-quality curation is infeasible.

The work concludes that effective learning from long-tailed noisy data requires not just noise reduction, but class-aware rectification that respects the specific statistical properties of each category.