A 3-D Log-Gabor Feature Similarity Index for Quality Assessment of Pansharpened Images

Technical Summary: A 3-D Log-Gabor Feature Similarity Index for Quality Assessment of Pansharpened Images

Problem Statement
The quality assessment of pansharpened images remains a significant challenge in remote sensing, primarily due to the unavailability of a high-resolution multispectral (HRMS) reference image for full-scale evaluation. Existing Image Quality Assessment (IQA) metrics often evaluate spectral and spatial qualities separately, failing to account for the critical inter-band relationships inherent in multispectral data. Furthermore, many traditional metrics rely on pixel-wise differences that do not align well with the Human Visual System (HVS), which is sensitive to structural patterns, edges, and textures rather than raw intensity values. While deep learning approaches have emerged, they can suffer from feature redundancy and loss functions that ignore relationships between fusion results. Consequently, there is a need for a perceptually relevant metric that simultaneously analyzes spatial and spectral information within a unified framework.

Methodology
The paper proposes a new metric, 3-D FSIM, which extends the Feature Similarity (FSIM) index—originally designed for grayscale and RGB images—to the domain of multispectral pansharpening. The methodology is grounded in the HVS's sensitivity to low-level features, specifically Phase Congruency (PC) and Gradient Magnitude (GM).

1. 3-D Log-Gabor Filtering: Unlike conventional approaches that process spectral bands independently, the proposed method employs a bank of 3-D log-Gabor filters. These filters operate across the spatial (x, y) and spectral (z) dimensions simultaneously. The filter transfer function is defined by radial frequency magnitude and angular orientation (elevation and azimuth), allowing the extraction of structural features that capture both spatial details and spectral correlations.
2. Feature Extraction:
   • 3-D Phase Congruency (PC): Computed by analyzing the even-symmetric and odd-symmetric responses of the 3-D log-Gabor filters across multiple scales and orientations. PC identifies perceptually significant structures (edges, corners) and is invariant to changes in brightness or contrast.
   • 3-D Gradient Magnitude (GM): Calculated using specific 3-D gradient operators to capture local intensity variations and contrast, complementing the structural information provided by PC.
3. Similarity Computation: The metric computes the similarity between the reference (or original low-resolution MS) and the distorted (pansharpened) images. Instead of averaging band-wise scores, the method treats feature responses at each spatial location as vectors along the spectral dimension. Similarity maps for PC and GM are generated using vector inner products and Euclidean norms.
4. Pooling: The final quality score is obtained by pooling the local similarity maps, weighted by the maximum phase congruency response. This ensures that perceptually significant regions contribute more heavily to the final score.

Key Contributions

• 3-D Framework: The primary contribution is the extension of the FSIM metric into a three-dimensional framework. This eliminates the need for independent band-by-band computation and averaging, thereby preserving and exploiting inter-band spectral correlations.
• Perceptual Alignment: By utilizing 3-D log-Gabor filters to extract PC and GM, the metric aligns closely with HVS perception, focusing on structural and textural features rather than simple pixel differences.
• Unified Assessment: The method provides a single score that simultaneously evaluates spatial fidelity and spectral consistency, addressing the limitations of metrics that assess these qualities in isolation.

Results and Performance
The proposed metric was evaluated using a database of pansharpened IKONOS images derived from four distinct regions (Coliseum, Road, Urban, River). The dataset included images fused via six different techniques (MRA and CS categories) and subjected to various distortions (blur and additive white Gaussian noise). Performance was benchmarked against subjective Mean Opinion Scores (MOS) and Differential MOS (DMOS) provided by human observers.

• Correlation Metrics: The 3-D FSIM achieved the highest correlation with subjective scores among all tested metrics. Specifically, it outperformed existing state-of-the-art methods (including SAM, ERGAS, UIQI, Q4, SSIM, and the original 2-D FSIM) in Spearman Rank-Order Correlation Coefficient (SROCC: 0.782), Kendall Rank-Order Correlation Coefficient (KROCC: 0.574), and Pearson Linear Correlation Coefficient (PLCC: 0.853).
• Error Metrics: The proposed method demonstrated the lowest Root Mean Square Error (RMSE) of 4.471, indicating minimal deviation from human subjective ratings.
• Ranking Consistency: In case studies involving specific fusion methods (e.g., PCA, ATWT, BDSD, PNN), the 3-D FSIM rankings closely matched the DMOS-based ground truth, showing superior agreement compared to metrics like SAM, QNR, and Ds, which exhibited significant deviations.
• Statistical Significance: F-tests confirmed that the performance advantage of 3-D FSIM over most competing metrics is statistically significant at the 95% confidence level.

Significance and Claims
The authors claim that the 3-D FSIM metric provides a more robust, reliable, and perceptually consistent tool for assessing the quality of pansharpened images. By integrating spatial and spectral information through a 3-D log-Gabor framework, the metric effectively captures distortions that affect both structural details and spectral consistency. The paper asserts that this approach overcomes the limitations of band-wise evaluation and offers a superior alternative to existing IQA metrics for full-resolution pansharpening quality assessment, particularly in scenarios where a high-resolution reference image is unavailable.