🤖 machine learning

FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels

The paper proposes FedEFC, a novel federated learning framework that combines dynamic prestopping and a tailored loss correction mechanism to effectively mitigate the impact of noisy labels in heterogeneous data environments, supported by theoretical analysis and experimental results demonstrating significant performance improvements over existing methods.

Original authors: Seunghun Yu, Jin-Hyun Ahn, Joonhyuk Kang

Published 2026-07-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seunghun Yu, Jin-Hyun Ahn, Joonhyuk Kang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where your phone, your smartwatch, and your neighbor's tablet all want to learn how to recognize cats, dogs, and birds together, but none of them are willing to share their private photos. This is the heart of Federated Learning, a clever way for computers to learn as a team without ever seeing each other's personal data. Instead of sending photos to a central boss, the devices send only their "lessons learned" (mathematical updates) back and forth. It's like a group of students studying for a test in separate rooms, whispering their best study tips to a teacher who writes them on a blackboard, without anyone ever revealing their secret notes.

However, there's a catch: sometimes the students make mistakes. They might mislabel a picture of a cat as a dog, or a teacher might give them a wrong answer key. In the real world, data is messy and full of these "noisy labels." When you mix this messiness with the fact that every student has a different mix of subjects (some have 100 cat photos, others have none), the group can get confused and learn the wrong things. This paper tackles the tricky problem of how to keep this team learning effectively when everyone is working with messy, private, and uneven data.

The researchers, led by Seunghun Yu and colleagues from KAIST and Myongji University, propose a new method called FedEFC (Federated Learning Using Enhanced Forward Correction). Think of FedEFC as a smart study guide that helps the group avoid getting stuck on bad information. The method works in two distinct phases, like a two-step dance to keep the learning on track.

First, the team uses a technique called Prestopping. Imagine you are practicing a song, but you start making the same mistake over and over because you're trying too hard to memorize a wrong note. If you keep going, you'll just get worse. Prestopping is like a wise coach who watches the group's progress and says, "Stop right here!" It identifies the exact moment when the model is about to start memorizing the wrong labels (overfitting) and halts the training just before that happens. This ensures the model learns the general patterns from the good data before the noise takes over.

Once the training pauses at this "sweet spot," the second phase begins: Loss Correction. This is where the magic of "Forward Correction" happens. Usually, when a computer sees a label like "Dog" but the picture is actually a "Cat," it gets confused. FedEFC builds a special map, called a noise transition matrix, which acts like a decoder ring. It figures out the probability of a mistake happening (e.g., "Oh, 20% of the time, 'Dog' labels are actually 'Cats'"). Instead of throwing away the messy data, the method uses this map to mathematically adjust the learning process. It's as if the teacher whispers to the students, "Don't worry about the wrong label; here's how to fix your thinking so you learn the right thing anyway."

What makes this approach special is how it handles the fact that every student (or client) has a different set of data. The researchers didn't just guess; they ran extensive simulations using famous image datasets like MNIST (handwritten numbers), CIFAR-10, and CIFAR-100. They tested their method against other popular techniques in scenarios where data was unevenly distributed and labels were noisy.

The results show that FedEFC is quite robust. In their simulations, it consistently outperformed existing methods, especially when the data was very messy and uneven. For instance, in one specific test with highly uneven data, FedEFC showed a relative performance improvement of up to 41.64% over the best existing loss correction method. Even in extreme cases where 80% of the labels were wrong, FedEFC managed to keep the model learning better than the alternatives. The authors also provided a mathematical proof suggesting that, under their method, the learning goal with noisy labels becomes almost the same as if the data were perfectly clean.

In short, FedEFC offers a way for distributed teams to learn from messy, private data without needing to share the raw data or assume everyone has perfect information. By knowing when to stop and how to mathematically correct the mistakes, it helps the group reach a smarter conclusion, even when the starting information is far from perfect.

Technical Summary: FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels

1. Problem Statement

Federated Learning (FL) enables collaborative model training across decentralized clients without sharing raw data, offering strong privacy guarantees. However, FL is uniquely vulnerable to noisy labels (mislabeled data) due to two compounding factors:

Data Heterogeneity (Non-IID): Clients possess data with varying distributions, making global model convergence difficult.
Decentralized Constraints: Unlike centralized learning, FL clients cannot access other clients' data to verify labels or apply standard noise mitigation techniques that rely on global data access.

Existing methods for handling noisy labels in centralized settings often fail in FL because they require pretrained models (which may not exist), assume homogeneous data, or rely on feature similarity that risks privacy. Furthermore, standard FL algorithms (e.g., FedAvg) tend to overfit to noisy labels, severely degrading performance.

2. Methodology: FedEFC

The authors propose FedEFC, a two-phase framework designed to mitigate noisy label impacts in Non-IID FL settings without requiring inter-client data exchange or external pretrained models.

Phase 1: Prestopping (Dynamic Early Stopping)

The first phase addresses the risk of overfitting to noisy labels.

Mechanism: The central server monitors the aggregated training accuracy of participating clients. It tracks the global accuracy $A(t)$ and employs a "patience" parameter ( $\tau_p$ ) and a threshold ( $\gamma_{thr}$ ).
Logic: If the global accuracy fails to improve for $\gamma_{thr}$ consecutive rounds, the system identifies this point as the prestopping point ( $T_e$ ).
Rationale: Empirical observations suggest that model loss on noisy data initially decreases but eventually increases as the model begins to memorize mislabeled examples. Prestopping halts training just before this overfitting phase, preserving parameters that are near-optimal for the clean data distribution.

Phase 2: Enhanced Forward Loss Correction

After $T_e$ , clients switch to a robust loss correction strategy to continue training.

Noise Transition Matrix Estimation: Instead of relying on a pretrained model (which is often unavailable in FL), FedEFC utilizes the global model trained up to $T_e$ $T_{e}$ .
- Clients use the global model to predict labels for their local data.
- A count matrix is constructed by counting samples where the observed label matches the high-confidence prediction of the global model (based on a threshold $\tau_j$ ).
- This count matrix is converted into a noise transition matrix ( $Q^k_{\tilde{y}|y}$ ), representing the conditional probability of an observed noisy label given the true label.
Loss Correction: The standard cross-entropy loss is replaced with an enhanced forward correction loss. This modifies the loss function to account for the estimated noise transition matrix, effectively aligning the optimization objective with the clean data distribution.
Handling Imbalance: In cases of severe class imbalance (low $p$ ), a weighted noise transition matrix is applied to compensate for skewed label distributions.

3. Key Contributions

Enhanced Forward Correction for FL: The authors develop a forward correction method specifically tailored for heterogeneous FL. Unlike previous approaches that rely on model predictions for noise estimation (which can be unstable), FedEFC uses a count-based approach leveraging the global model at the prestopping point. This improves the stability and accuracy of the noise transition matrix estimation in Non-IID settings.
Prestopping Mechanism: A dynamic early stopping technique is introduced to prevent overfitting to noisy labels before the loss correction phase begins, ensuring the model starts Phase 2 with robust parameters.
Theoretical Alignment: The paper provides a theoretical proof (Theorem 1) demonstrating that under the proposed enhanced forward correction, the minimizer of the FL objective function with noisy labels is approximately equivalent to the minimizer with clean labels. This relies on the property of composite proper losses, showing that the noisy objective can be aligned with the clean objective if the estimated noise transition matrix is close to the true matrix.
Robustness in Non-IID Settings: The method is designed to function effectively without assuming any client has clean data or requiring additional privacy-compromising information (like feature data).

4. Experimental Results

The authors evaluated FedEFC on MNIST, CIFAR-10, and CIFAR-100 datasets under various Non-IID configurations (controlled by Dirichlet parameter $\alpha_{dir}$ and Bernoulli parameter $p$ ) and noise levels (controlled by noise fraction $\rho$ and sparsity $\zeta$ ).

Performance vs. Baselines: FedEFC consistently outperformed standard FL algorithms (FedAvg, FedProx, FedDitto) and state-of-the-art noise-robust methods (FedCorr, FedNoRo, RoFL, Confident Learning).
Heterogeneity: In highly Non-IID settings (e.g., $\alpha_{dir}=1.0, p=0.2$ ), FedEFC maintained superior performance. For instance, on CIFAR-100 under severe Non-IID conditions, it showed significant gains over existing loss correction methods.
Noise Robustness:
- Under moderate to high noise, FedEFC demonstrated the highest robustness.
- In extreme noise scenarios (e.g., $\rho=0.8$ ), FedEFC achieved the highest accuracy, outperforming the second-best method (FedNoRo) by over 2% in specific configurations.
- The paper notes a relative performance improvement of up to 41.64% over existing loss correction methods in certain heterogeneous settings.
Comparison to Clean Data: While FedEFC does not reach the performance of a model trained on perfectly clean data (the upper bound), it significantly narrows the gap compared to other noise-robust techniques.

5. Significance and Claims

The paper claims that FedEFC offers a practical and theoretically grounded solution to the noisy label problem in Federated Learning. Its primary significance lies in:

Privacy Preservation: It mitigates noise without requiring clients to share raw data or feature information.
Feasibility: It removes the dependency on pretrained models, which are often impractical in real-world FL deployments.
Adaptability: By combining dynamic prestopping with a refined loss correction mechanism, it addresses both the overfitting tendency of FL on noisy data and the instability of noise estimation in heterogeneous environments.

The authors conclude that FedEFC is a robust algorithm capable of ensuring reliable model performance in realistic FL scenarios characterized by both data heterogeneity and label noise.