💻 computer science

3D Fourier-based Global Feature Extraction for Hyperspectral Image Classification

This paper proposes Hybrid GFNet (HGFNet), a novel hyperspectral image classification architecture that combines 3D convolutions with three complementary frequency transforms (1D spectral, 2D spatial, and 3D spatial-spectral) and an Adaptive Focal Loss to efficiently model global spatial-spectral dependencies while addressing scalability issues and class imbalance.

Original authors: Muhammad Ahmad

Published 2026-03-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Muhammad Ahmad

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to identify different types of trees in a massive forest, but you can't just look at them with your eyes. Instead, you have a special pair of "super-glasses" that see hundreds of different colors of light (spectral bands) for every single leaf, not just the three colors (Red, Green, Blue) our eyes see. This is Hyperspectral Imaging.

The goal is to build a computer program (an AI) that can look at these super-detailed images and correctly label every pixel as "Oak," "Pine," "Grass," or "Water." This is called Hyperspectral Image Classification.

However, this is incredibly hard for computers because:

Too much data: There are so many colors and pixels that the computer gets overwhelmed (the "curse of dimensionality").
Confusing neighbors: Some trees look identical in color but are different species, while others look different but are the same species.
Missing clues: Sometimes the computer only has a few examples of rare trees to learn from, making it easy to get confused.

The Problem with Current AI

Current AI detectives use two main strategies, but both have flaws:

The "Zoom-In" Detective (3D CNNs): This AI looks very closely at small neighborhoods to see fine details. It's great at spotting texture, but it has "tunnel vision." It can't see the big picture or understand how a whole forest connects.
The "Zoom-Out" Detective (Transformers): This AI looks at the whole image at once to understand the big context. But it's like trying to read a library of books at the speed of light; it gets too slow and expensive as the image gets bigger. Also, it often ignores the unique "color fingerprints" of the materials.

The Solution: HGFNet (The Hybrid Detective)

The authors of this paper created a new AI called HGFNet. Think of it as a detective who wears a special pair of glasses that can instantly switch between "micro-vision" and "frequency-vision."

Here is how it works, using simple analogies:

1. The "3D Convolution" (The Local Detective)

First, the AI looks at small patches of the image, like a detective examining a single leaf and the dirt around it. It uses 3D Convolutional Layers to understand the texture and the immediate neighbors.

Analogy: Imagine you are trying to identify a song by listening to a 5-second clip. You can hear the specific instruments and the rhythm. This part of the AI does the same for the local area.

2. The "Fourier Transform" (The Frequency Detective)

This is the paper's big innovation. Instead of just looking at the image as a picture, HGFNet turns the image into a soundwave or a musical chord.

The Magic Trick: In music, a complex chord is made of different notes (frequencies). In an image, complex patterns are made of different "frequencies" (smooth gradients vs. sharp edges).
The 3D Twist: Most AI only looks at the "spatial" frequencies (left-to-right patterns). HGFNet is special because it also looks at Spectral Frequencies (the color patterns).
- Analogy: Imagine you have a smoothie with 100 different fruits blended together. A normal AI tries to taste the whole smoothie. HGFNet uses a special machine to separate the smoothie back into its individual fruit juices (frequencies). It can then say, "Ah, I taste a lot of strawberry (a specific spectral frequency), so this must be a strawberry field," even if the leaves look like apple leaves.

3. The "Global Filter" (The Noise Cancellation)

Once the image is turned into frequencies, the AI acts like a high-tech noise-canceling headphone.

It keeps the "loud" frequencies that represent real trees and buildings.
It mutes the "static" frequencies that represent random noise or sensor errors.
Analogy: If you are trying to hear a friend speak at a noisy party, you ignore the clinking glasses and focus on the voice. HGFNet does this for the whole image, cleaning it up so the important patterns stand out.

4. The "Adaptive Focal Loss" (The Fair Teacher)

Hyperspectral datasets are often unbalanced. You might have 1,000 photos of "Oak" trees but only 5 photos of "Rare Pine." A normal AI would ignore the Rare Pine because it's easier to just guess "Oak" and get a high score.

The Fix: HGFNet uses a special scoring system called Adaptive Focal Loss.
Analogy: Imagine a teacher grading a test. If a student gets a common question right, the teacher gives 1 point. But if the student gets a rare, difficult question right, the teacher gives 10 points. This forces the AI to pay extra attention to the rare, hard-to-find classes so it doesn't ignore them.

The Results

The authors tested this new detective on three different real-world forest datasets.

The Competition: They compared HGFNet against other top AI models (like Transformers and Mamba).
The Outcome: HGFNet won almost every time. It was more accurate, it made fewer mistakes on rare trees, and the maps it produced looked much cleaner (less "salt-and-pepper" noise).

Summary

HGFNet is a smart new way to classify hyperspectral images. It combines:

Local focus (looking at small details),
Global frequency analysis (turning the image into sound/frequencies to see hidden patterns),
Smart scoring (making sure the AI doesn't ignore rare items).

It's like giving the computer a superpower to see the "music" hidden inside the colors, allowing it to identify materials with incredible precision, even when the data is messy or incomplete.

1. Problem Statement

Hyperspectral Image Classification (HSIC) faces three primary challenges:

Scalability and Complexity: Existing Transformer-based models suffer from quadratic computational complexity ( $O(n^2)$ ) due to self-attention mechanisms, making them difficult to scale for high-dimensional spectral data.
Inadequate Spectral Modeling: Recent Fourier transform-based methods typically apply 2D spatial FFTs, treating spectral channels as static features. This ignores critical inter-band spectral dependencies and wavelength-specific frequency structures inherent in hyperspectral data.
Class Imbalance: Real-world hyperspectral datasets often suffer from severe class imbalance (few labeled pixels for minority classes), leading to biased optimization and poor discrimination for underrepresented categories.
Local vs. Global Trade-off: 3D CNNs capture local spatial-spectral structures well but have limited receptive fields. Transformers capture global context but struggle with local detail and computational cost.

2. Proposed Methodology: Hybrid GFNet (HGFNet)

The authors propose HGFNet, a hybrid architecture that integrates localized 3D convolutional feature extraction with frequency-domain global filtering. The framework consists of three main components:

A. Local Feature Extraction (3D CNN Stem)

The input HSI cube is processed through three consecutive 3D convolutional layers equipped with Gaussian Error Linear Unit (GeLU) activations.
Purpose: To capture fine-grained local spatial-spectral structures, spectral gradients, and neighborhood context, providing a strong inductive bias for local mixing patterns.

B. Global Frequency Filtering (GFNet Blocks)

Instead of standard self-attention, HGFNet utilizes Global Frequency Network (GFNet) blocks to model long-range dependencies efficiently ( $O(n \log n)$ via FFT). The core innovation is the introduction of three complementary 3D Fourier transforms tailored to hyperspectral data:

Spectral Fourier Transform (SFT): A 1D FFT applied along the spectral axis to encode wavelength-wise frequency variations and inter-band dependencies.
Spatial Fourier Transform (SpFT): A 2D FFT applied over spatial dimensions to capture global spatial patterns and texture regularities.
Spatial-Spectral Fourier Transform (SSFT): A 3D FFT applied jointly across spatial and spectral dimensions. This is the key contribution, modeling coupled frequency structures arising from the interaction between material spectra and spatial organization.

Mechanism: Features are normalized, transformed via FFT, filtered by learnable frequency masks (to suppress noise and retain discriminative components), passed through a Feed-Forward Network (FFN), and reconstructed via Inverse FFT. Residual connections ensure stable gradient flow.

C. Adaptive Focal Loss (AFL)

To address class imbalance, the standard Focal Loss is replaced with an Adaptive Focal Loss (AFL).

Unlike standard focal loss which uses global parameters, AFL dynamically adjusts the class-specific weighting ( $\alpha_i$ ) and focusing parameter ( $\gamma_i$ ) based on class frequency and training dynamics.
This ensures that minority and hard-to-classify samples receive appropriate emphasis without excessively down-weighting well-represented classes.

3. Key Contributions

High-Dimensional Fourier Generalization: The first application of a joint 3D FFT (SSFT) to hyperspectral imagery, explicitly modeling the coupling between spatial layout and spectral variation, rather than treating them separately.
Hybrid Local-Global Architecture: A novel design that fuses the local inductive bias of 3D CNNs with the efficient global context of Fourier filtering, overcoming the limitations of pure CNNs (limited receptive field) and pure Transformers (quadratic complexity).
Imbalance-Aware Optimization: The integration of Adaptive Focal Loss specifically tuned for the spectral separability and spatial distribution characteristics of HSIC tasks.
State-of-the-Art Performance: Demonstrated superior performance across multiple benchmarks, validating the efficacy of explicit frequency-domain modeling over attention-based or wavelet-based baselines.

4. Experimental Results

The model was evaluated on three benchmark datasets: Indian Pines (IP), WHU Hi HanChuan (HC), and WHU Hi HongHu (HH).

Quantitative Performance:
- HC Dataset: HGFNet achieved 99.24% Overall Accuracy (OA), 98.72% Average Accuracy (AA), and 99.11% Kappa, outperforming SpectralFormer, Hybrid ViT, WaveFormer, and Mamba-based models.
- IP Dataset: Achieved 98.46% OA and 98.24% Kappa, significantly outperforming the second-best method (WaveMamba).
- HH Dataset: Achieved 99.31% OA and 99.13% Kappa.
Ablation Studies:
- SSFT vs. Single Transforms: The joint 3D FFT (SSFT) consistently outperformed models using only SFT or SpFT, proving that spatial and spectral frequency cues are complementary.
- Loss Function: The proposed AFL consistently yielded higher accuracy than standard Focal Loss (SFL), particularly for minority classes.
- Baselines: HGFNet outperformed Transformer-based (MHSA, SpectralFormer) and Wavelet-based (WaveFormer, WaveMamba) approaches, indicating that explicit 3D frequency modeling provides a stronger inductive bias for HSIC.
Qualitative Analysis: Visual classification maps showed HGFNet produced more spatially coherent regions with fewer "salt-and-pepper" noise artifacts and clearer boundaries compared to competitors.

5. Significance and Future Directions

Significance: This work bridges the gap between local feature extraction and efficient global dependency modeling in hyperspectral imaging. By leveraging the computational efficiency of FFTs and the specific 3D nature of HSI data, it offers a scalable alternative to Transformers that handles spectral redundancy and class imbalance effectively.
Limitations & Future Work:
- Stationarity Assumption: The reliance on Fourier transforms assumes signal stationarity, which may struggle with highly non-stationary variations (e.g., complex illumination or atmospheric effects). Future work could explore adaptive spectral bases or hybrid Fourier-Wavelet formulations.
- Computational Cost: While efficient, 3D convolutions can still be memory-intensive for ultra-high-resolution scenes. Future research may investigate depthwise separable 3D convolutions.
- Extreme Imbalance: While AFL helps, extremely skewed distributions may require meta-learning or curriculum-based reweighting strategies.

In conclusion, HGFNet represents a significant advancement in HSIC by effectively unifying local spatial-spectral cues with global frequency-domain modeling, setting a new state-of-the-art benchmark for accuracy and robustness.