💻 computer science

Unsupervised Incremental Learning Using Confidence-Based Pseudo-Labels

This paper proposes ICPL, an unsupervised class-incremental learning method that utilizes confidence-based pseudo-labels to enable models to learn novel classes from unlabeled data, achieving superior performance over state-of-the-art class-iNCD approaches while demonstrating practicality on fine-grained datasets and resource-constrained environments.

Original authors: Lucas Rakotoarivony

Published 2026-08-26

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Lucas Rakotoarivony

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, machines have become remarkably skilled at recognizing patterns, from identifying cats in photos to diagnosing diseases. However, these digital brains are often trained in a controlled environment where the list of things they need to know is fixed and complete. In the real world, the situation is far more fluid. New types of objects, animals, or situations appear constantly, and a system that cannot learn about them without being completely retrained is of limited use. This challenge is known as incremental learning. The goal is to teach a model new categories over time while remembering everything it learned before, a task complicated by a phenomenon called catastrophic forgetting, where learning new information causes the system to lose old knowledge. Traditionally, solving this has required humans to painstakingly label every new image with its correct name, a process that is slow, expensive, and often impossible to scale for the endless stream of data encountered in nature or industry.

A researcher at Thales has proposed a different way forward, one that removes the need for human labels entirely during these learning phases. They developed a method called ICPL, which allows an artificial intelligence to teach itself about new classes of objects using only raw, unlabeled images. Instead of waiting for a human to say, "This is a new type of bird," the system groups similar images together based on their visual features, creating its own temporary names for these groups. The researcher then introduced a safety mechanism to ensure the system only trusts its own guesses when it is very confident they are correct. By integrating this self-labeling technique into existing learning frameworks, they showed that a machine could continue to learn new things over a long period, retaining its memory of the past without human intervention.

The core of this work addresses a specific gap in how we test and build these systems. Previous methods for handling unlabeled data often treated the problem as a simple clustering task, where the computer just groups things together without caring if the groups match reality. The researcher argued that this approach is flawed for real-world applications because it allows a system to be "right" even if it confuses old classes with new ones. For instance, a system could mistakenly label a new type of car as an old type of car and vice versa, yet still receive a perfect score under older testing rules. To fix this, the researcher created a new evaluation protocol that locks the relationship between the computer's self-made groups and the actual real-world categories. This ensures that when the system is tested, it is truly being judged on its ability to distinguish between different things, not just on its ability to sort them into piles.

To generate these self-made labels, the researcher used a standard grouping algorithm that organizes images based on how similar they look in the computer's internal representation. Imagine the computer looking at a collection of unlabeled photos and arranging them into clusters where similar items sit close together. The researcher then calculated a confidence score for each image, measuring how tightly it fit into its assigned group. If an image was right in the center of a cluster, the system was highly confident in its label; if it was on the edge, the system was unsure. The method then discarded the uncertain images and only used the high-confidence ones to teach the model. This selective process was crucial, as using all the self-generated labels, including the noisy and incorrect ones, would have degraded the model's performance.

The results of this approach were tested on two major image datasets, one containing 100 categories of everyday objects and another with 100 categories of natural images. The researcher compared their unsupervised method against the standard supervised approach, where humans provide all the labels. They found that while the self-teaching method did result in a slight drop in accuracy compared to the human-labeled version, the difference was surprisingly small. In many scenarios, the performance loss was less than ten percent, a trade-off the researcher considered well worth the massive savings in time and labor required to label the data. More importantly, when they compared their method to other existing techniques designed to discover new categories without labels, their approach was significantly better. In long-term learning scenarios involving many incremental steps, their method outperformed the best previous unsupervised techniques by more than five percent in final accuracy.

The study also explored how this method holds up in more complex, real-world situations where the differences between categories are subtle, such as distinguishing between different breeds of dogs or models of cars. They tested the system on fine-grained datasets and found that it remained effective, though they noted that pre-training the model on a massive general dataset before starting the incremental learning actually hurt performance in these specific cases. This suggests that for highly specialized tasks, it is better for the model to learn from scratch using the new data rather than relying on broad, pre-existing knowledge. Furthermore, the researcher analyzed the computational cost of their method and discovered that it was not only faster to train but also required less computing power than the fully supervised approach. By filtering out the uncertain data, the system processed fewer images, reducing the energy and time needed for training by about a quarter compared to the standard method.

This work demonstrates that artificial intelligence can be designed to learn continuously from the raw data of the world without constant human supervision. By combining a simple clustering technique with a strict confidence filter and a more rigorous way of testing results, the researcher has shown a viable path toward machines that can adapt to new situations as they arise. The findings suggest that the dream of an AI that learns like a human—observing, grouping, and refining its understanding of new things without a teacher for every single example—is becoming a practical reality, particularly in environments where resources are limited and data is abundant but unlabeled.

Technical Summary: Unsupervised Incremental Learning Using Confidence-Based Pseudo-Labels (ICPL)

Problem Definition

Deep learning models typically require fixed class sets during training, failing to recognize novel classes that emerge in real-world streaming data. While Class-Incremental Learning (CIL) methods address this by learning new classes while retaining old knowledge, they rely on the unrealistic assumption that every incremental dataset is fully labeled by humans. Conversely, Class Incremental Novel Category Discovery (class-iNCD) methods attempt to learn from unlabeled data but are often ill-suited for handling a large sequence of incremental tasks over the long term.

The paper identifies a gap in Unsupervised Incremental Learning, where a model must learn new classes from raw, unlabeled images while preserving prior knowledge, all without human annotations. This task is challenging due to the dual risks of catastrophic forgetting and the introduction of noisy pseudo-labels.

Methodology: ICPL

The author proposes ICPL (Incremental Learning using Confidence-based Pseudo-Labels), a framework designed to replace human annotations with high-quality pseudo-labels generated from unlabeled data.

1. Pseudo-Label Generation

The core of ICPL involves a clustering-based approach to generate labels for new incremental tasks ( $k \ge 2$ ):

Feature Extraction: The feature extractor $m_{k-1}$ (trained on previous tasks) is used to generate embeddings for the unlabeled data of the current task.
Clustering: A KMeans algorithm is applied to these embeddings to cluster the data, assuming images of the same class are close in the embedding space.
Confidence Scoring: To mitigate the noise inherent in unsupervised clustering, the author introduces a confidence selection mechanism. They calculate a distance matrix between samples and cluster centers. A confidence score is derived by applying a softmax transformation to the inverse of these distances.
Selection: Only samples with confidence scores above a threshold $\alpha$ (set to 0.85 in experiments) are selected to form the pseudo-labeled dataset $\hat{D}_k$ for training.

2. Integration with CIL

ICPL is not a standalone algorithm for preventing forgetting but a wrapper that enables existing supervised CIL methods to operate in an unsupervised setting. The pseudo-labels $\hat{y}$ simply replace the ground-truth labels $y$ in popular CIL algorithms, including:

Data-centric: Replay (using a rehearsal buffer).
Algorithm-centric: iCaRL (using knowledge distillation) and WA (Weight Adjustment).
Model-centric: FOSTER (using feature boosting and compression).

The framework also incorporates data augmentation (e.g., AutoAugment, MixUp) and recalculates pseudo-labels every $\tau$ epochs (e.g., $\tau=10$ ) to refine the labels as the model improves.

3. Evaluation Protocol

The paper argues that standard "cluster accuracy" (using Hungarian assignment to match clusters to classes) is inadequate for incremental learning because it allows a model to achieve 100% accuracy by confusing old and new classes (e.g., predicting all new classes as old classes and vice versa).

Static Encoding: The author proposes a static encoding evaluation protocol. A mapping between pseudo-clusters and ground-truth classes is established once and fixed. New associations are only added when new classes are introduced. This ensures that misclassification between tasks is penalized, providing a more rigorous and realistic metric for unsupervised incremental learning.

Key Contributions

ICPL Framework: A simple yet effective method enabling unsupervised incremental learning by generating confidence-based pseudo-labels, eliminating the need for manual annotation in incremental steps.
Integration and Analysis: The method is integrated into state-of-the-art CIL approaches (Replay, iCaRL, WA, FOSTER), with a detailed analysis of performance degradation compared to fully supervised baselines.
Superiority over Class-iNCD: The work demonstrates that ICPL significantly outperforms existing class-iNCD and Generalized Category Discovery (GCD) methods in long-term incremental learning scenarios.
Efficiency and Robustness: The method is shown to be computationally efficient and robust across various datasets, including fine-grained ones, and adaptable to different clustering algorithms.

Experimental Results

The author evaluated ICPL on CIFAR100, ImageNet100, and fine-grained datasets (Stanford Cars, FGVC-Aircraft, Oxford-IIIT Pet).

Comparison with Supervised Learning: While unsupervised learning naturally incurs a performance drop compared to fully supervised CIL, ICPL limits this degradation. On average, the final accuracy drop is approximately 10.27%. In specific configurations (e.g., Base50 Inc10 on CIFAR100), FOSTER + ICPL achieved an average accuracy of 65.68%, surpassing the supervised WA method (62.96%).
Comparison with Class-iNCD: ICPL outperforms state-of-the-art class-iNCD methods (FRoST, GM) and GCD methods (DCCL, CMS) by more than 5% in final accuracy across various settings. Notably, class-iNCD methods struggled significantly in long-term incremental setups (e.g., Base0 Inc5), whereas ICPL maintained high accuracy.
Fine-Grained Datasets: The method proved effective on fine-grained datasets, demonstrating practical relevance. Interestingly, pretraining on ImageNet slightly degraded performance on these specific datasets, suggesting that general features may obscure subtle distinctions required for fine-grained classification.
Computational Complexity: Contrary to intuition, the unsupervised approach reduced computational cost. By selecting only high-confidence samples (approx. 70% of data) and using efficient clustering, ICPL reduced TFLOPs by 24% and training time compared to the supervised baseline.

Significance and Claims

The paper positions ICPL as a practical solution for real-world applications where continuous learning from unlabeled data is necessary. The author claims that:

Practicality: By removing the dependency on human annotation for every incremental task, ICPL makes incremental learning feasible for resource-constrained and large-scale environments.
Evaluation Rigor: The proposed static encoding protocol addresses a critical flaw in previous evaluation methods, ensuring that models are truly learning distinct classes rather than merely clustering data.
Baseline Establishment: The author hopes ICPL serves as a new baseline for unsupervised incremental learning, demonstrating that simple clustering combined with confidence selection can outperform complex, specialized category discovery methods in long-term scenarios.

The work concludes that while unsupervised incremental learning presents challenges regarding label noise, the ICPL approach effectively balances performance, computational efficiency, and the elimination of manual labeling requirements.