Spectral-Spatial Contrastive Learning Framework for Regression on Hyperspectral Data

Technical Summary: Spectral-Spatial Contrastive Learning Framework for Regression on Hyperspectral Data

1. Problem Statement

Hyperspectral imagery (HSI) provides rich, high-dimensional data by capturing hundreds of contiguous spectral bands, allowing for the identification of unique material signatures. While self-supervised contrastive learning has revolutionized representation learning for classification tasks, its application to regression tasks (such as hyperspectral unmixing or pollution concentration prediction) remains under-explored.

Existing methods often focus solely on the spectral dimension (wavelength variations), neglecting the critical spatial information (structural relationships and context) inherent in HSI. The challenge lies in designing a framework that can simultaneously leverage both spectral and spatial features to improve regression accuracy and robustness in real-world scenarios.

2. Methodology

The authors propose a model-agnostic spectral-spatial contrastive learning framework designed to enhance various 3D-based backbone architectures (3D-CNNs and Transformers).

• Data Processing: To manage high dimensionality, the HSI cube is divided into smaller patches. For regression, the target label for a patch is defined as the average label of the pixels within that patch.
• The Contrastive Framework:
  • Augmentation: Each patch $P$ is transformed into a different "view" $eP$ using a specialized toolbox $\Phi_{transform}$.
  • Feature Extraction: Both the original and augmented patches are passed through a shared 3D backbone ($\Phi_w$) to extract latent features.
  • Regression Head: A network ($g_\theta$) maps these features to continuous regression labels.
  • Contrastive Loss for Regression: Unlike classification (where positive pairs belong to the same class), the authors define positive pairs in the regression context using a radius-based approach. Two samples $i$ and $j$ are considered a positive pair if the $\ell_2$-norm of their regression labels is within a specific radius $r$ ($|y_i - y_j| \leq r$).
  • Joint Optimization: The model is trained using a total loss function: $L_{total} = L_{Regression} (MSE) + \alpha L_{Contrastive}$ (Cross-Entropy based).
• Augmentation Toolbox:
  • Spectral Transformations: Spectral shift, spectral flipping, scattering (Hapke’s model), atmospheric compensation, elastic distortion, band erasure, and band permutation.
  • Spatial Transformations: Rotation, elastic deformation, flipping, and translation.
  • Spectral-Spatial: A combination of both spatial and spectral transforms.

3. Key Contributions

1. Spectral-Spatial Toolbox: The creation of a comprehensive set of data augmentation techniques specifically tailored for the unique properties of hyperspectral data.
2. Regression-Centric Contrastive Framework: A novel approach to applying contrastive learning to regression tasks by utilizing a radius-based similarity metric for positive pair selection.
3. Model-Agnostic Design: The framework is compatible with various state-of-the-art 3D backbones, including 3D-CNNs and Vision Transformers.

4. Results

The framework was evaluated using five benchmark 3D-based backbones on both synthetic data (simulating non-linear mixing and noise) and the real-world Samson dataset (soil, tree, and water endmembers).

• Performance Gains: In all tested models, the proposed framework significantly outperformed the "Baseline" (models trained with regression loss only).
• Incremental Improvement: The results demonstrated a clear hierarchy of effectiveness: Baseline < Spectral Contrastive < Spatial Contrastive < Spectral+Spatial Contrastive.
• Architecture Performance: The Spectral-Spatial Transformer consistently achieved the highest $R^2$ scores and lowest Mean Absolute Errors (MAE), proving that transformers are highly effective for HSI tasks when paired with contrastive learning.

5. Significance

This research bridges the gap between self-supervised learning and hyperspectral regression. By integrating spatial context with spectral signatures, the framework provides a more holistic way to learn representations. This has significant implications for remote sensing applications where labeled data is scarce, such as environmental monitoring, mineralogy, and precision agriculture. The work sets a foundation for future research into domain adaptation to help these models generalize to entirely unseen geographic or atmospheric conditions.