💻 computer science

Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD

This paper benchmarks transfer learning, active learning, and pseudo-labeling on the Ground-based Cloud Dataset (GCD) and finds that while ImageNet-pretrained transfer learning achieves high label efficiency, uncertainty-based active learning and high-confidence pseudo-labeling offer only marginal and inconsistent improvements over a strong supervised baseline.

Original authors: Esther Bou Dagher, Viktoriya Bu-Dager, Boguslaw Zegarlinski

Published 2026-10-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Esther Bou Dagher, Viktoriya Bu-Dager, Boguslaw Zegarlinski

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The sky above us is never truly empty; it is a shifting tapestry of water droplets and ice crystals that dictates our weather, powers our solar panels, and guides our aircraft. To understand these patterns, scientists have long relied on satellites and weather balloons, but there is a unique value in looking up from the ground. Ground-based cameras capture the texture and movement of clouds in high resolution, offering a local perspective that satellites cannot see. However, teaching a computer to recognize these clouds is a difficult task. While a human can often distinguish a fluffy white cumulus from a flat, gray stratus, the lines between them are frequently blurred. Clouds mix, transition, and change shape, making it hard for an artificial intelligence to learn without a massive amount of human guidance. In the world of machine learning, this guidance comes in the form of labeled data, where experts manually identify the cloud type in thousands of images. This process is slow, expensive, and requires specialized knowledge, creating a bottleneck for researchers who want to build better weather models.

A team of researchers set out to solve this problem by asking a simple but critical question: how much human labeling is actually necessary to teach a computer to classify clouds accurately? They focused on a specific challenge known as "label efficiency," which explores whether clever learning strategies can reduce the need for manual work. Instead of inventing a new type of artificial intelligence architecture, they tested three practical approaches using a standard, powerful computer vision model that had already learned to recognize general objects from a massive database of everyday images. The first approach was straightforward: feed the model a small, carefully chosen set of labeled clouds and let it learn. The second, called active learning, involved letting the model pick the specific images it was most confused about, so a human could label only those tricky cases. The third approach, known as pseudo-labeling, had the model guess the labels for unlabeled images it felt confident about, and then use those guesses as if they were real facts to teach itself further.

The researchers conducted their experiment using a large, public collection of ten thousand sky images, divided into seven distinct cloud categories ranging from clear skies to complex mixed clouds. They started by training their model on just one percent of the available labeled images, a mere hundred pictures, and gradually increased the amount of data up to the full set. They repeated this process five times with different random selections to ensure their results were consistent. The findings revealed that the standard approach, where the model learns from a random but representative sample of labeled images, was already remarkably efficient. With only one percent of the labels, the model achieved an accuracy of roughly sixty-three percent. As they added more data, the performance climbed quickly, reaching over seventy-two percent accuracy with just forty percent of the labels. By the time they used the full dataset, the accuracy was only slightly higher, at about seventy-three percent. This suggested that the model had already learned the essential visual features of clouds from the small initial set and did not need the rest of the data to perform well.

When the researchers compared this standard method against the more complex strategies, the results were surprisingly modest. The active learning approach, which tried to be smart by selecting only the most confusing images, did not significantly outperform the simple random sampling. While it did manage to find and label some of the most difficult cloud types, such as the messy "mixed" clouds that contain elements of several different categories, the overall improvement in accuracy was small. Similarly, the pseudo-labeling method, which tried to expand the training set with the model's own confident guesses, produced reliable labels for the easy-to-identify clouds but failed to help much with the difficult ones. The model tended to guess correctly on clear skies and simple cloud shapes, but it avoided the ambiguous, mixed scenes where errors were most likely to occur. Consequently, adding these extra "self-taught" examples did not solve the core problem of distinguishing the most visually similar and confusing cloud types.

The study ultimately concluded that for ground-based cloud classification, the most effective strategy is often the simplest one: using a strong, pre-trained model and a carefully selected, small fraction of human-labeled data. The complex methods of asking the model to choose its own lessons or teaching it with its own guesses offered only marginal benefits over the baseline. The real bottleneck was not a lack of data or a need for smarter algorithms, but the inherent difficulty of the visual task itself. The clouds that confused the model the most were those that naturally look alike to the human eye, such as the boundary between stratocumulus and cumulonimbus. These results suggest that while artificial intelligence can learn a great deal from very little human help, the final push to perfect accuracy on the most ambiguous cases will require more targeted solutions rather than just more data or more complex learning tricks.

Technical Summary: Label-Efficient Learning for Ground-Based Sky-Image Classification

Problem Statement
Accurate ground-based cloud classification is critical for atmospheric monitoring, solar-energy forecasting, aviation weather assessment, and climate observation. However, reliable annotation of sky images is time-consuming and requires domain expertise, particularly when distinguishing visually similar or mixed cloud types (e.g., Stratocumulus vs. Cumulonimbus, or Mixed scenes). While deep learning has improved classification performance, most research focuses on architectural novelty rather than the annotation cost required to achieve useful performance. There is a lack of systematic empirical benchmarks comparing label-efficient strategies—specifically active learning and pseudo-labeling—against strong supervised transfer-learning baselines under constrained annotation budgets.

Methodology
The study utilizes the Ground-based Cloud Dataset (GCD), comprising 10,000 training images and 9,000 independent test images across seven sky-type groups (Cumulus, Altocumulus, Cirrus, Clear sky, Stratocumulus, Cumulonimbus, and Mixed). The dataset exhibits significant class imbalance.

The experimental framework employs a frozen ImageNet-pretrained ResNet50 as a common backbone to ensure a controlled comparison, isolating the effects of learning strategies from architectural variations. The authors benchmark three strategies across label budgets ranging from 1% to 100% of the training set, repeated over five random seeds:

Supervised Transfer Learning: A baseline using class-stratified random sampling of the labeled subset.
Uncertainty-Based Active Learning: A pool-based strategy where a model trained on an initial labeled subset selects the most uncertain samples (based on $1 - \max_c p(y=c|x)$ ) from the unlabeled pool for annotation. This process iterates from 1% to 40% budgets.
High-Confidence Pseudo-Labeling: A semi-supervised approach where a model trained on the labeled subset assigns pseudo-labels to unlabeled samples with confidence $\ge 0.95$ . These are added to the training set for a second stage of training. A "continued-supervised" control is used to isolate gains from pseudo-labels versus additional optimization steps.

Performance is evaluated using overall accuracy, balanced accuracy, macro-F1, weighted-F1, and per-class F1 scores. Diagnostic analyses include pseudo-label quality (accuracy, coverage, and selection bias) and active-learning query enrichment (comparing queried samples against the remaining unlabeled pool distribution).

Key Contributions

Quantification of Label Efficiency: The authors establish the supervised label-efficiency curve for a frozen ResNet50 on GCD, demonstrating how performance scales with annotation volume.
Systematic Benchmarking: The study provides a controlled comparison of stratified random sampling, uncertainty-based active learning, and high-confidence pseudo-labeling under matched label budgets and training protocols.
Diagnostic Analysis: The work goes beyond aggregate metrics to analyze why strategies succeed or fail, specifically examining pseudo-label reliability and the specific cloud groups targeted by active learning queries.
Class-Balanced Evaluation: The study emphasizes metrics (macro-F1, balanced accuracy) that reveal performance on minority and visually ambiguous classes, which overall accuracy often obscures.

Results

Supervised Baseline: Supervised transfer learning is highly label-efficient. Test accuracy rises sharply from $0.635 \pm 0.018$ at 1% labels to $0.730 \pm 0.002$ at 40% labels, approaching the full-label performance of $0.735 \pm 0.003$ . Class-balanced metrics improve more gradually, indicating that additional labels benefit difficult classes even after overall accuracy plateaus.
Active Learning: Uncertainty sampling provides small, inconsistent improvements over the supervised baseline. While it successfully queries difficult groups (e.g., "Mixed" and the Stratocumulus/Cumulonimbus boundary) and improves macro-F1 at specific budgets, it does not yield large aggregate gains in overall accuracy.
Pseudo-Labeling: High-confidence pseudo-labels are highly reliable (accuracy $0.946 $–$ 0.977$) but exhibit significant selection bias. They are heavily biased toward easy, visually distinctive classes (e.g., Clear sky) and under-represent difficult, ambiguous groups (e.g., Mixed). Consequently, pseudo-labeling does not consistently outperform an update-matched continued-supervised control, suggesting gains are often due to additional optimization rather than the pseudo-labels themselves.
Per-Class Performance: "Clear sky" is easily classified ( $F1 \approx 0.97$ ), while "Mixed," "Altocumulus," and "Stratocumulus" remain challenging. Active learning improves performance on "Mixed" more than other methods, but errors persist due to visual ambiguity and overlap between classes.

Significance and Claims
The paper claims that for ground-based cloud classification on GCD, a strong supervised transfer-learning baseline is already highly effective, requiring only 20–40% of the training labels to approach full-label performance. The study concludes that simple active learning and semi-supervised strategies provide only limited additional gains over this strong baseline.

The authors argue that the primary challenge is not merely the quantity of labeled data, but the difficulty of improving performance on visually ambiguous and mixed cloud groups. They suggest that while active learning successfully targets these difficult regions, the queried samples may represent redundant boundary cases or insufficient information to overcome the strong feature representations provided by ImageNet pretraining. Similarly, pseudo-labeling fails to improve class-balanced metrics because it reinforces the model's confidence on easy examples rather than addressing the hard cases.

The paper posits that future improvements will likely require more targeted approaches, such as class-aware pseudo-labeling, diversity-aware active learning, or hybrid strategies specifically designed for visually ambiguous sky-type groups, rather than simple application of standard label-efficient techniques.

More like this