💻 computer science

Enhancing Hyperspectral Image Prediction with Contrastive Learning in Low-Label Regime

This study demonstrates that a two-stage contrastive learning framework significantly enhances both single-label and multi-label hyperspectral image classification performance in low-label regimes, maintaining robust accuracy even with 50% less training data by leveraging self-supervised pre-training and a streamlined architecture that adapts the encoder to classifier features.

Original authors: Salma Haidar, José Oramas

Published 2026-02-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Salma Haidar, José Oramas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize different types of terrain—like forests, roads, and fields—from satellite photos. Usually, to teach a robot, you need a massive library of photos where a human has carefully drawn boxes around every tree and road and labeled them. This is like hiring a team of experts to spend years labeling every single pixel in a photo. It's expensive, slow, and often impossible to get enough labeled data.

This paper proposes a clever workaround: teach the robot to learn on its own first, then give it a quick lesson.

Here is how their method works, broken down into simple steps:

1. The "Self-Teaching" Phase (Contrastive Learning)

Before the robot ever sees a labeled photo, the researchers let it look at a mountain of unlabeled satellite images.

The Analogy: Imagine showing the robot two slightly different photos of the same patch of ground (maybe one is flipped or rotated). The robot's job is to figure out, "Hey, these two look like the same thing!"
The Goal: By doing this millions of times with different pairs, the robot learns the intrinsic patterns of the world. It learns that "trees look like trees" and "roads look like roads" without anyone ever telling it the names. It builds a strong internal map of what things look like.

2. The "Quick Lesson" Phase (Fine-Tuning)

Once the robot has this strong internal map, the researchers give it a small amount of labeled data (just 50% or even less of what is usually needed).

The Analogy: Now, the robot is like a student who already knows how to read and write (from the self-teaching phase). The teacher just needs to show them a few flashcards with the names of the objects. Because the robot already understands the shapes and textures, it learns the names very quickly.
The Twist: The researchers found that if they let the robot "re-learn" its internal map slightly while learning the names (a process they call "CL-tune"), it gets even better. It's like the student adjusting their understanding of "tree" to perfectly match the teacher's specific definition of "oak tree."

3. The Two Types of Tests

The researchers tested this on two different ways of looking at the data:

Single-Label (The "Center Pixel" Game): They look at a tiny square of the image and ask, "What is the main thing in the very center?" (e.g., "This is a road").
Multi-Label (The "What's in the Box?" Game): They look at the same tiny square and ask, "What everything is in this box?" (e.g., "This box has a road, some grass, and a shadow"). This is harder because real-world scenes are messy and mixed.

The Big Results

Half the Data, Same Results: Even when they cut the amount of labeled training data in half (or more), their robot performed just as well as other robots trained on all the data.
Better than the Old Way: Their method beat traditional methods that tried to learn everything from scratch using only labeled data.
The "Magic" of Context: When they visualized what the robot was "thinking," they saw something surprising. Even though they never told the robot about geography or location, the robot naturally grouped things that belong together. For example, it realized that "shadows" often appear near "buildings" and "grass" is near "trees." It learned these hidden connections just by looking at the patterns in the unlabeled data.

Why This Matters

The paper argues that this approach is a game-changer for hyperspectral imaging (which sees more colors than the human eye can see). Because labeling this data is so hard and expensive, being able to get great results with half the labeled data means we can deploy these technologies much faster and cheaper in the real world.

In short: Instead of forcing a student to memorize a dictionary before they can read, this method teaches the student to recognize the shapes of letters first, so they only need a tiny dictionary to learn the words.

Technical Summary: Enhancing Hyperspectral Image Prediction with Contrastive Learning in Low-Label Regimes

Problem Statement

Hyperspectral image (HSI) analysis faces significant challenges due to the scarcity of labelled data, the high dimensionality of spectral information, and the complexity of training deep learning models. Traditional supervised learning approaches often require extensive, labour-intensive annotation, which is costly and prone to errors. Furthermore, existing methods predominantly focus on single-label, pixel-level classification, often neglecting the contextual influence of neighbouring pixels. This limitation is particularly problematic for mixed spectral signals where a single pixel may contain multiple materials, or for applications like land cover classification where identifying multiple objects within a scene is crucial. While self-supervised learning (SSL) and contrastive learning (CL) have shown promise in computer vision, their application to HSI remains underexplored, particularly for patch-level multi-label classification tasks under data-constrained regimes.

Methodology

The paper proposes a streamlined, two-stage classification framework that leverages self-supervised contrastive learning to enhance performance in low-label scenarios.

Stage 1: Self-Supervised Pre-training
The authors train a base encoder and a projection network using unlabelled data.

Data Augmentation: For each input patch, two augmented views (positive pairs) are generated via random horizontal and vertical flips.
Architecture: The encoder consists of fully connected layers with ReLU activation and dropout. It maps the augmented views into hidden representations ( $h_1, h_2$ ). A projection head (two fully connected layers) maps these to vector embeddings ( $z_1, z_2$ ).
Objective: The model is trained using the Normalised Temperature-scaled Cross Entropy Loss (NT-Xent). This loss function maximizes the similarity between the embeddings of the two augmented views of the same patch while minimizing similarity with embeddings from other patches (negative pairs) within the batch.
Outcome: The encoder learns to extract invariant and discriminative features from the unlabelled data without explicit class supervision.

Stage 2: Supervised Fine-tuning
The projection head is discarded, and the pre-trained encoder is integrated with a classifier.

Training Strategy: The authors evaluate two scenarios:
- CL-tune: The encoder layers are unfrozen, allowing the weights to be retrained alongside the classifier using labelled data.
- CL-freeze: The encoder layers are frozen, and only the classifier is trained.
Tasks: The framework is evaluated on two tasks:
- Multi-label Classification: Patches are assigned labels indicating the presence of multiple classes (mixed materials).
- Single-label Classification: Patches are assigned a label based solely on the centre pixel, following standard practices.
Loss Functions: Binary Cross Entropy with Logits Loss is used for multi-label tasks, and Cross-Entropy Loss is used for single-label tasks.

Key Contributions

Robustness to Limited Data: The study demonstrates that the contrastive learning-based method remains robust even when the amount of labelled training data is reduced by up to 50%. The method maintains competitive performance comparable to fully supervised techniques trained on full datasets.
Patch-Level Multi-Label Classification: The work addresses the under-explored area of patch-level multi-label classification in HSI. By training on small patches containing subsets of a scene, the method identifies multiple materials within a patch, capturing spatial context and mitigating the impact of mixed spectral signals.
Single-Label Performance without Pre-processing: The approach achieves superior single-label classification accuracy compared to traditional supervised methods and other self-supervised baselines, notably without requiring complex pre-processing techniques like Principal Component Analysis (PCA) or morphological profiles.
Qualitative Insights into Representations: Through t-SNE visualisation, the authors reveal that the contrastive learning-based encoder captures contextual and transitional features (e.g., spatial proximity and material similarities) even without explicit training on such information. This suggests the model implicitly uncovers spatial information inherent in the data.

Experimental Results

The methodology was validated on four public datasets: Pavia University (PaviaU), Salinas, Houston 2013, and Houston 2018.

Multi-Label Performance: The CL-tune strategy significantly outperformed the CL-freeze strategy and three fully supervised baselines (Iterative, Joint, and Cascade schemes). On PaviaU and Salinas, CL-tune improved accuracy by 1.73%–4.37% and 1.25%–2.46% respectively over the best supervised baselines. Crucially, when trained on only 50% of the labelled data, the CL-tune model achieved accuracy nearly identical to fully supervised models trained on 100% of the data.
Single-Label Performance: The CL-based classifier outperformed established methods including HSI-CNN, TRI-CNN, SSCL, and 3D-CNN. Notably, it achieved higher accuracy than the 3D-CNN method while utilizing a significantly smaller dataset (approx. 9% of the total labelled samples for PaviaU) and a simpler architecture without PCA.
Data Reduction Impact: While performance naturally declined with reduced data, models with larger hidden representation sizes (64 vs. 32) showed greater resilience, particularly on datasets with high class diversity (PaviaU, Houston 2013).
Temperature Parameter: Experiments confirmed that a temperature scaling parameter ( $T$ ) of 0.1 yielded optimal accuracy, balancing model confidence and generalisation.

Significance and Claims

The paper claims that its primary significance lies in demonstrating that a streamlined, minimalist architecture enhanced by self-supervised contrastive learning can effectively overcome the limitations of labelled data scarcity in hyperspectral imaging.

The authors assert that their method offers a viable alternative to complex deep learning structures (like ResNet) and heavy pre-processing pipelines. By enabling the retraining of the encoder alongside the classifier (CL-tune), the model adapts its feature extraction capabilities to the specific requirements of the downstream task, leading to improved classification performance. Furthermore, the qualitative analysis suggests that contrastive learning inherently fosters the learning of representations that encode spatial and contextual relationships, enhancing the model's ability to distinguish between classes even in complex, mixed-patch scenarios. This approach is presented as a robust solution for real-world applications where acquiring extensive labelled datasets is impractical.