HyVIC: A Metric-Driven Spatio-Spectral Hyperspectral Image Compression Architecture Based on Variational Autoencoders

1. Problem Statement

Hyperspectral Imaging (HSI) generates massive data volumes due to high spatial and spectral resolution, posing significant challenges for storage and transmission in remote sensing (RS). While learning-based compression methods have improved upon traditional transform coding (e.g., JPEG2000, PCA), existing deep learning approaches often suffer from two main limitations:

• Inadequate Spatio-Spectral Modeling: Most models adapt architectures designed for natural images (which rely heavily on spatial features) without explicitly addressing the unique, distinct redundancies inherent in HSIs (both spatial and spectral).
• Rigid Architectures: Existing Variable Autoencoder (VAE) models often lack mechanisms to dynamically balance spatial and spectral feature learning. They typically rely on fixed architectural designs that cannot adapt to varying compression requirements or sensor characteristics, leading to suboptimal Rate-Distortion (RD) performance.

2. Methodology: The HyVIC Architecture

The authors propose HyVIC (Hyperspectral Variational Image Compression), a novel metric-driven VAE architecture specifically designed for HSI. The model is built upon the "mean & scale hyperprior" framework but introduces adjustable spatio-spectral components to explicitly model HSI redundancies.

Core Components

The architecture consists of four main modules:

1. Adjustable Spatio-Spectral Encoder ($E_\Phi$): Transforms the input HSI into a latent representation. It sequentially stacks spectral processing (1x1 convolutions for pixel-wise spectral feature extraction) and spatial processing blocks (2D convolutions with GDN activation). The number of spatial stages ($S$) is adjustable.
2. Spatio-Spectral Hyperencoder ($E^H_\Psi$): Processes the latent representation to generate a "hyperlatent" ($Z$). This captures the spatio-spectral redundancies within the latent space to serve as side information for the entropy model.
3. Spatio-Spectral Hyperdecoder ($D^H_\Gamma$): Estimates the mean ($\hat{\mu}$) and scale ($\hat{\sigma}$) parameters of a Gaussian Mixture Model (GMM) entropy model based on the quantized hyperlatent. This allows for image-dependent entropy coding.
4. Adjustable Spatio-Spectral Decoder ($D_{\Phi'}$): Reconstructs the HSI from the quantized latent. It mirrors the encoder, using Inverse GDN (IGDN) and adjustable spatial stages to recover spatial and spectral details.

Metric-Driven Hyperparameter Selection

A key innovation is the metric-driven strategy to select hyperparameters (specifically the number of spatial stages $S$, latent channels $M$, and kernel size $k$).

• Instead of arbitrary selection, the authors use the Bjøntegaard Delta PSNR (BD-PSNR) metric to systematically evaluate different configurations.
• This strategy optimizes the trade-off between spatial and spectral feature learning, ensuring the model performs well across a wide range of Compression Ratios (CRs).

Training Objective

The model is trained end-to-end to minimize a Rate-Distortion (RD) loss:
$$L_{RD} = R + \lambda \cdot D$$
Where $R$ is the bitrate (entropy of latent and hyperlatent) and $D$ is the distortion (Mean Squared Error). The Lagrange multiplier $\lambda$ controls the trade-off between compression efficiency and reconstruction quality.

3. Key Contributions

1. Novel Architecture: Introduction of HyVIC, the first VAE-based HSI compression model with explicitly adjustable spatial and spectral blocks to balance feature learning.
2. Metric-Driven Optimization: Development of a systematic strategy using BD-PSNR to select hyperparameters, proving that the balance between spatial and spectral learning is critical for HSI fidelity.
3. Comprehensive Evaluation: Extensive experiments on two large-scale benchmark datasets (HySpecNet-11k and MLRetSet) comparing HyVIC against 12 state-of-the-art methods (including GANs, Transformers, and traditional VAEs).
4. Practical Guidelines: Derivation of insights suggesting that HSI compression benefits from reduced spatial receptive fields (smaller kernels) and reduced spatial downsampling compared to natural image compression, due to the specific nature of RS data.

4. Experimental Results

The proposed model was evaluated on HySpecNet-11k (EnMAP satellite data) and MLRetSet (airborne data).

• Performance Gains: HyVIC achieved a BD-PSNR improvement of up to 4.66 dB over the previous best-performing model (HyCASS) across the entire RD curve.
• Rate-Distortion Performance:
  • At high compression ratios (CR > 32), HyVIC significantly outperformed traditional methods (JPEG2000, PCA) and other learning-based models, maintaining high spatial and spectral fidelity.
  • At low CRs, traditional linear methods remained competitive, but HyVIC still demonstrated superior adaptability.
• Ablation Studies:
  • Kernel Size: A small kernel size ($k=3$) was found to be optimal, outperforming larger kernels ($k=5, 7$) by 1.0–2.8 dB in PSNR while reducing computational complexity by ~65–85%. This suggests spatial redundancies in RS data are localized.
  • Spatial Stages: A balanced configuration (e.g., $S=2$) was found to be superior. $S=0$ (spectral only) failed at high CRs, while excessive spatial stages ($S=3,4$) degraded performance at low CRs by over-compressing spatial details.
• Visual and Spectral Fidelity:
  • HyVIC preserved spatial structures (edges, boundaries) better than models relying solely on spatial downsampling (which caused blurring).
  • It accurately reconstructed detailed spectral signatures (including spikes in SWIR bands) where other models introduced smoothing or offsets.

5. Significance and Future Directions

• Paradigm Shift: The paper challenges the direct transfer of natural image compression architectures to HSI. It demonstrates that HSI requires a distinct architectural balance, favoring smaller spatial receptive fields and higher transform capacities to preserve spectral integrity.
• Generalization: The metric-driven selection strategy proved effective across different sensors (satellite vs. airborne) and resolutions, indicating strong generalization capabilities.
• Future Work: The authors suggest exploring more advanced entropy models to capture latent dependencies and developing interactive Large Language Model (LLM) agents to guide compression based on user context.

In conclusion, HyVIC sets a new state-of-the-art for learning-based HSI compression by explicitly addressing the spatio-spectral nature of hyperspectral data through an adjustable, metric-optimized VAE architecture.