Spectral Compressive Imaging via Chromaticity-Intensity Decomposition

Technical Summary: Spectral Compressive Imaging via Chromaticity-Intensity Decomposition

Problem Statement
Coded Aperture Snapshot Spectral Imaging (CASSI) is a promising architecture for capturing hyperspectral images (HSIs) in a single shot. However, the reconstruction of HSIs from CASSI measurements constitutes a severely ill-posed inverse problem due to two primary challenges:

1. Entangled Information: The compressive measurement fuses spatial and spectral information, making the separation of spatial structures and spectral signatures difficult. Existing deep learning methods often rely on implicit feature learning without clear physical decomposition, while optimization-based methods struggle with complex textures and noise.
2. Illumination Sensitivity: Captured radiance inherently entangles intrinsic surface reflectance with scene illumination. This coupling makes reconstruction sensitive to lighting variations, limiting the recovery of lighting-invariant spectral reflectance. While intrinsic image decomposition (separating reflectance and illumination) is established in RGB imaging, it has not been effectively incorporated into CASSI reconstruction.

Methodology
The authors propose a Chromaticity-Intensity Decomposition (CID) framework that factorizes the hyperspectral image cube $X$ into a spatially smooth intensity map $I$ and a spectrally variant chromaticity cube $C$ ($X = C \odot I$). In this formulation, $C$ represents the lighting-invariant reflectance enriched with high-frequency spatial details and local spectral sparsity, while $I$ captures the global illumination structure.

Based on this decomposition, the authors introduce CIDNet, a deep unfolding network designed for a dual-camera CASSI system (where an intensity image, e.g., from a PAN or RGB camera, is available). The methodology comprises three core components:

1. Degradation-Aware Noise Modeling:
   The authors formulate the reconstruction as a Maximum A Posteriori (MAP) estimation problem under an anisotropic Gaussian noise model. Unlike traditional methods that assume homoscedastic (uniform) noise, CIDNet employs a Dual Noise-Estimation Module (DNEM). This module learns a spatially adaptive noise covariance map ($\Sigma$) and a denoising strength parameter at each iteration stage. This allows the network to dynamically adjust gradient projection steps and proximal denoising operations to handle spatially varying noise patterns inherent in the imaging process.

2. Hybrid Spatial-Spectral Transformer (HSST):
   To reconstruct the chromaticity cube, which exhibits structured sparsity and localized spectral correlations, the authors design an asymmetric UNet backbone.

   • Encoder (Spatial): Utilizes a Swin Transformer with window-based local spatial attention (Spa-LWSA) to capture high-frequency spatial textures.
   • Decoder (Spectral): Employs a Sparse TopK Spectral Attention (Spec-TKSA) mechanism. Unlike dense spectral attention, this module restricts attention to the $K$ most relevant spectral neighbors within local windows. This enforces sparsity and locality, reflecting the physical property that reflectance spectra often have localized support. A multi-ratio strategy aggregates attention maps from different sparsity levels to capture both dominant and complementary spectral correlations.

3. Unfolding Framework:
   The network unrolls an optimization algorithm (based on Half-Quadratic Splitting) where each stage consists of a learnable noise estimation, an analytical data-consistency step (incorporating the intensity-guided mask), and a learned proximal denoiser. This structure bridges model-based interpretability with data-driven expressiveness.

Key Contributions

• Novel Decomposition Model: The paper proposes the first chromaticity-intensity decomposition model specifically tailored for CASSI, explicitly separating lighting-invariant chromaticity from smooth intensity to improve robustness against illumination changes.
• CIDNet Architecture: Development of a deep unfolding network integrating a hybrid spatial-spectral Transformer (HSST) and a degradation-aware, spatially-adaptive noise estimation module (DNEM).
• Sparse Spectral Attention: Introduction of a TopK spectral attention mechanism that effectively models the sparse and local nature of chromaticity spectra, reducing computational complexity while preserving spectral fidelity.
• Dual-Validation: The decomposition paradigm is validated through both traditional optimization-based solvers and the proposed deep unfolding framework in a dual-camera CASSI setting.

Experimental Results
Extensive experiments were conducted on synthetic datasets (CAVE and KAIST) and real-world dual-camera CASSI data.

• Quantitative Performance: CIDNet achieved state-of-the-art performance on the KAIST simulation dataset. The 9-stage version (CIDNet-9stg) attained an average PSNR of 44.12 dB and SSIM of 0.991, significantly outperforming existing methods like In2SET-9stg, SSR-9stg, and DAUHST-9stg.
• Chromaticity and Intensity Fidelity: The method demonstrated superior performance in reconstructing both the chromaticity and intensity components, with CIDNet achieving the best chromaticity metrics (PSNR 35.81 dB, SSIM 0.93) among dual-camera algorithms.
• Real-World Validation: Reconstructions on real-world data (Ninja scene) confirmed the model's ability to restore fine details and spectral accuracy compared to baseline methods like PIDS and iterative CID-TV.
• Ablation Studies: Experiments confirmed that the intensity-guided mask, the HSST architecture, and the DNEM module each contribute significantly to the final performance, with the full model showing a substantial improvement over baseline configurations.

Significance
The paper argues that explicitly decomposing hyperspectral images into chromaticity and intensity offers a physically interpretable solution to the ill-posed CASSI reconstruction problem. By focusing on learning the chromaticity (which is invariant to lighting and spectrally sparse) rather than the raw HSI, the method achieves better spectral fidelity and robustness. The work highlights the efficacy of combining physics-aware decomposition with hybrid attention mechanisms (spatial and spectral) to address the specific challenges of compressive spectral imaging, setting a new benchmark for both spectral reconstruction and chromaticity fidelity.