Super-résolution non supervisée d'images hyperspectrales de télédétection utilisant un entraînement entièrement synthétique

Technical Summary: Unsupervised Hyperspectral Single-Image Super-Resolution via Fully Synthetic Training

Problem Statement
Hyperspectral images (HSIs) provide rich spectral information across hundreds of wavelengths, making them valuable for applications such as vegetation monitoring, land use analysis, and astrophysics. However, due to optical constraints, hyperspectral sensors typically possess lower spatial resolution compared to multispectral counterparts. The task of Hyperspectral Single-Image Super-Resolution (SISR) aims to enhance spatial resolution while preserving spectral fidelity.

Current SISR methods predominantly rely on supervised learning, which requires paired low-resolution (LR) and high-resolution (HR) ground truth data. A significant bottleneck exists in this domain: high-resolution ground truth HSIs are rarely available in practice. Consequently, supervised approaches are limited by the lack of suitable training datasets. While some methods utilize transfer learning or specific convolutional architectures (e.g., 3D convolutions), they remain constrained by the unavailability of real-world HR references.

Methodology
The authors propose a novel unsupervised approach that circumvents the need for real HR ground truth by training a neural network entirely on synthetic data. The method integrates hyperspectral unmixing with a stochastic image generation model. The process consists of three main stages:

1. Hyperspectral Unmixing:
   The input low-resolution HSI ($HSI_{LR}$) is decomposed into two components using a linear mixing model:

   • Endmembers ($S$): Spectral signatures of pure materials, extracted using the Volume Minimum algorithm.
   • Abundance Maps ($A_{LR}$): Spatial maps representing the proportion of each material in every pixel, obtained via least squares estimation.
     The assumption is that $HSI_{LR} = \sum S \cdot A_{LR}$.

2. Synthetic Data Generation (Dead Leaves Model):
   To generate training pairs, the authors employ the "Dead Leaves" model, a stochastic process originally introduced by Matheron to model porous media and later adapted for natural images.

   • Generation Process: Synthetic abundance maps are created by sequentially superimposing random rectangular shapes ("leaves") at random positions and orientations. This process mimics the statistical properties of natural scenes, particularly non-Gaussian statistics.
   • Realism Constraints: To ensure the synthetic data reflects the specific scene, the spatial geometry (shape, position, rotation) of the leaves is kept consistent across all material dimensions ($z$). The intensity values ($V$) for each leaf are randomly sampled from the actual abundance maps ($A_{LR}$) of the target scene.
   • Training Pairs: High-resolution synthetic abundance maps ($A_{DL,HR}$) are generated directly. Corresponding low-resolution maps ($A_{DL,LR}$) are created by applying a simulated Point Spread Function (PSF)—specifically a Gaussian blur followed by bicubic downsampling—to the HR maps.

3. Network Training and Reconstruction:

   • Architecture: The authors utilize MCNet (Mixed 3D/2D Convolutional Network), a state-of-the-art CNN designed for HSI super-resolution. It combines 2D convolutions for spatial feature extraction and 3D convolutions for joint spatial-spectral processing. The network is adapted to accept $N$ channels, corresponding to the number of unmixing materials.
   • Training: The network is trained in a supervised manner using the synthetic pairs $(A_{DL,LR}, A_{DL,HR})$ with an $L1$ loss function.
   • Inference: During the actual super-resolution phase, the real $A_{LR}$ is fed into the trained network to predict super-resolved abundance maps ($A_{SR}$). The final high-resolution HSI is reconstructed by recombining the predicted abundances with the original endmembers: $HSI_{SR} = \sum S \cdot A_{SR}$.

Key Contributions

• Unsupervised Framework: The paper introduces a fully unsupervised SISR pipeline that eliminates the dependency on unavailable high-resolution ground truth data.
• Synthetic Training Strategy: It demonstrates the efficacy of training deep learning models on synthetic data generated via the Dead Leaves model, specifically tailored to replicate the statistical properties of hyperspectral abundance maps.
• Integration of Unmixing: Unlike previous synthetic approaches, this method fully integrates the hyperspectral unmixing step into the super-resolution workflow, ensuring that spectral consistency is maintained throughout the process.
• Improved Architecture Adaptation: The method adapts the MCNet architecture to operate on unmixing components rather than raw spectral cubes, optimizing the network for the specific task of abundance map enhancement.

Experimental Results
The method was evaluated on the Urban dataset (captured by the HYDICE sensor), comprising 307 $\times$ 307 pixels and 162 usable spectral bands. A super-resolution factor of 4 was applied.

• Training Setup: 5,000 synthetic abundance map pairs were generated. The network was trained for 200 epochs using the Adam optimizer.
• Comparisons: The proposed method (labeled MCNet-DL) was compared against:
  • Bicubic interpolation.
  • State-of-the-art supervised methods: MCNet, SSPSR, and HSISR.
  • Note: To ensure a fair comparison, the supervised methods were trained on 16 patches of the Urban dataset, with a leave-one-out cross-validation strategy, giving them optimal conditions that are often impractical in real-world scenarios.
• Performance Metrics: The evaluation used Peak Signal-to-Noise Ratio (PSNR), Spectral Angle Mapper (SAM), and Relative Global Dimensionless Synthesis Error (ERGAS).
• Quantitative Findings: MCNet-DL outperformed all competing methods across all metrics:
  • PSNR: 26.69 dB (vs. 25.56 dB for MCNet).
  • SAM: 14.53° (lower is better; vs. 16.52° for MCNet).
  • ERGAS: 7.60 (lower is better; vs. 8.72 for MCNet).
• Qualitative Findings: Visual inspection of reconstructed patches showed that the proposed method recovered significantly more spatial details compared to bicubic interpolation and other SOTA methods, with reconstructions appearing more precise than the standard MCNet applied directly to HSI patches.

Significance and Claims
The paper claims that the proposed approach effectively addresses the critical limitation of data scarcity in hyperspectral super-resolution. By leveraging the Dead Leaves model to generate statistically relevant synthetic abundance maps, the authors demonstrate that a network trained entirely on synthetic data can outperform supervised methods trained on real, albeit artificially partitioned, data.

The authors position this work as an evolution of their previous research [26], highlighting that the full integration of hyperspectral unmixing results in a complete, efficient, and truly unsupervised super-resolution method. The results suggest that synthetic data generation is a viable and powerful strategy for training deep learning models in remote sensing applications where ground truth is inaccessible.