💻 computer science

Voronoi-guided Bilateral 2D Gaussian Splatting for Arbitrary-Scale Hyperspectral Image Super-Resolution

This paper proposes GaussianHSI, a novel framework that leverages Voronoi-guided bilateral 2D Gaussian splatting and a spectral detail enhancement module to achieve flexible, arbitrary-scale hyperspectral image super-resolution while preserving both spatial adaptability and spectral fidelity.

Original authors: Jie Zhang, Jinkun You, Shi Chen, Yicong Zhou

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jie Zhang, Jinkun You, Shi Chen, Yicong Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a blurry, low-resolution photo of a landscape taken from a satellite. You want to zoom in to see the individual trees, cars, and buildings clearly. But there's a catch: this isn't just a normal photo; it's a Hyperspectral Image (HSI). Instead of just seeing Red, Green, and Blue, this camera sees hundreds of different "colors" (wavelengths) of light, like a super-powerful prism. This helps scientists identify what materials are on the ground (e.g., is that a healthy crop or a dry one?).

The problem is that these images are often blurry. Traditional methods to sharpen them are like trying to fit a square peg in a round hole: they work well if you want to zoom in exactly 2x or 4x, but if you want to zoom in 3.5x or 7x, they break or require you to build a completely new machine for each specific zoom level.

This paper introduces GaussianHSI, a new way to sharpen these images that works at any zoom level you want. Here is how it works, explained through simple analogies:

1. The Old Way vs. The New Way

The Old Way (Pixel Grid): Imagine trying to reconstruct a blurry photo by guessing the color of every single tiny square (pixel) on a grid. If you want to zoom in, you have to guess the colors of new squares that didn't exist before. It's rigid and often looks blocky.
The New Way (2D Gaussian Splatting): Instead of a grid of squares, imagine the image is made of thousands of soft, glowing, fuzzy clouds (called Gaussians) floating in space. Each cloud has a specific shape, size, color, and transparency.
- To see the image at any size, you don't guess new pixels; you just look at how these clouds overlap at that specific spot. It's like looking at a painting through a magnifying glass: the details are already there in the "clouds," you just need to focus on them.

2. The Problem with Just "Clouds"

If you just let all the clouds float around and mix, it gets messy.

The "Too Many Clouds" Problem: If you try to calculate the color of one tiny spot by looking at every cloud in the entire image, it takes forever and wastes energy.
The "Wrong Neighbors" Problem: In a normal photo, neighbors are usually similar. But in a hyperspectral image, a patch of water might be right next to a patch of concrete. They are close in space, but their "colors" (spectral data) are totally different. If you mix them up, the result looks wrong.

3. The Solution: GaussianHSI's Two Superpowers

The authors built a system with two main tricks to fix these problems:

Trick A: The "Voronoi Guide" (Smart Selection)

Imagine you are at a party, and you want to know who is talking to you.

Old Method: You shout to everyone in the building. Most people ignore you, but you still hear them, creating noise.
GaussianHSI Method: It uses a Voronoi Guide. Think of this like drawing invisible fences around each "cloud." If you are standing inside a specific fence, only the clouds inside that fence are allowed to talk to you.
- This is adaptive. If the clouds are stretched out (like a long road), the fence stretches with them. If they are round (like a tree), the fence is round. This ensures you only listen to the clouds that actually matter for your specific spot, making the process fast and accurate.

Trick B: The "Bilateral Filter" (The Double-Check)

Once you've selected the relevant clouds, you need to mix their colors.

Old Method: You just mix them based on how close they are.
GaussianHSI Method: It uses a Bilateral Filter. This is like a strict bouncer at a club.
1. Distance Check: "Are you close to me?" (Geometric relevance).
2. Soul Check: "Do you look like the original blurry photo?" (Feature consistency).
- Even if a cloud is right next to you, if its "soul" (spectral signature) doesn't match the low-resolution input, the bouncer kicks it out. This prevents the "water next to concrete" mistake, ensuring the sharp details stay true to the original data.

Trick C: The "Spectral Detail" Booster

Sometimes, the cloud method is great for shapes but a bit fuzzy on the specific "colors" (spectral bands).

So, the system has a parallel assistant (the Spectral Detail Enhancement module). While the main system focuses on the shapes and structures, this assistant focuses purely on sharpening the colors and chemical signatures, then they combine their results for a perfect picture.

Why is this a big deal?

Arbitrary Scale: You can zoom in by 1.5x, 4.3x, or 10x, and the system works perfectly without needing to be retrained. It's like having a universal zoom lens.
Efficiency: By using the "Voronoi Guide," it ignores irrelevant data, making it faster.
Accuracy: By checking both distance and "soul" (spectral consistency), it doesn't blur the unique chemical signatures that make hyperspectral images so valuable for science.

In a Nutshell

GaussianHSI is like upgrading from a pixelated, blocky video game to a smooth, high-definition 3D world where you can zoom in infinitely without losing quality. It uses smart "fences" to pick the right details and a "double-check" system to ensure the colors stay true, making it the ultimate tool for sharpening blurry satellite images.

1. Problem Statement

Hyperspectral Images (HSIs) contain rich spectral information crucial for remote sensing but often suffer from limited spatial resolution due to hardware trade-offs. Hyperspectral Image Super-Resolution (HSI-SR) aims to reconstruct high-resolution (HR) HSIs from low-resolution (LR) observations while preserving spectral fidelity.

Current deep learning approaches face two main limitations:

Fixed-Scale Constraints: Most existing methods rely on fixed-scale upsampling operations. They require separate training and storage for each target scale, lacking flexibility for arbitrary-scale reconstruction.
Discrete vs. Continuous Representation: While recent arbitrary-scale methods (e.g., SQformer) use discrete token representations, they struggle with continuity. Conversely, 2D Gaussian Splatting offers a continuous representation ideal for arbitrary scales but has not been effectively adapted to HSIs. Direct application of existing Gaussian Splatting methods fails because:
- They often rely on fixed Euclidean rasterization, which lacks adaptivity to diverse spatial structures in remote sensing.
- They aggregate Gaussians based solely on geometric proximity, ignoring the fact that spatially adjacent pixels in HSIs may have distinct spectral properties.
- They lack specific mechanisms to preserve the critical spectral fidelity required in HSI tasks.

2. Methodology: GaussianHSI

The authors propose GaussianHSI, a framework based on 2D Gaussian Splatting designed for arbitrary-scale HSI super-resolution. The architecture consists of an encoder, two parallel branches (Spatial and Spectral), and a decoder.

A. Overall Architecture

Encoder: Extracts latent features from the LR HSI using a fully connected layer followed by residual convolutional blocks.
Feature Splitting: The encoded features are split into two streams:
- Spatial Stream: Processed by the Voronoi-Guided Bilateral 2D Gaussian Splatting (VBGS) module for spatial reconstruction.
- Spectral Stream: Processed by the Spectral Detail Enhancement (SDE) module to refine spectral details.
Decoder: Concatenates the outputs of both branches and uses self-attention and fully connected layers to reconstruct the final HR HSI.

B. Core Module 1: Voronoi-Guided Bilateral 2D Gaussian Splatting (VBGS)

This module addresses the spatial reconstruction challenge by moving beyond fixed rasterization.

Initialization & Parameterization: Instead of regressing Gaussians from scratch, the model initializes Gaussian centers at pixel locations and predicts parameters (center coordinates, covariance matrix $\Sigma$ , and feature/color) using CNNs. This ensures stable optimization.
Voronoi-Guided Selection:
- To avoid the high computational cost of aggregating all Gaussians for every pixel, the method selects the top- $k$ most relevant Gaussians.
- Unlike fixed-radius rasterization, relevance is determined by Mahalanobis distance, which accounts for the anisotropic scale and orientation of each Gaussian.
- This creates a dynamic, Voronoi-like partitioning where each target pixel is associated with the most spatially relevant Gaussian candidates, adapting to varying object sizes and shapes.
Reference-Aware Bilateral Aggregation:
- Standard aggregation relies only on geometric proximity. VBGS introduces a bilateral weighting mechanism.
- It computes a weight based on cosine similarity between the Gaussian's feature and the upsampled LR reference feature at the target location.
- This ensures that the reconstruction respects both geometric relevance and spectral/feature consistency, preventing the mixing of dissimilar spectral signatures from nearby but distinct objects.

C. Core Module 2: Spectral Detail Enhancement (SDE)

Recognizing that Gaussian spatial reconstruction alone may not fully recover complex spectral variations, a parallel SDE module is introduced.

It takes the upsampled spectral features and processes them through local patches.
It employs alternating Spatial MLPs (for token mixing across locations) and Spectral MLPs (for channel mixing within the spectral vector) to refine spectral details before fusion.

3. Key Contributions

First Arbitrary-Scale Gaussian HSI Framework: GaussianHSI is the first work to introduce 2D Gaussian Splatting into the domain of arbitrary-scale HSI super-resolution.
Voronoi-Guided Gaussian Selection: A novel strategy that uses Mahalanobis distance to dynamically select the most relevant Gaussians for each pixel, enabling adaptive modeling of diverse spatial structures without fixed support constraints.
Reference-Aware Bilateral Aggregation: A mechanism that modulates Gaussian contributions using low-resolution features as a reference, ensuring that spatial reconstruction maintains spectral consistency.
Spectral Detail Enhancement: A dedicated module that complements the spatial Gaussian reconstruction to ensure high-fidelity spectral recovery.

4. Experimental Results

The method was evaluated on three benchmark remote sensing datasets: Houston, Pavia, and Chikusei.

Single-Scale Performance: GaussianHSI outperformed state-of-the-art methods (including MCNet, MSDformer, HSRMamba, and SQformer) across various scales ( $\times2, \times4, \times8$ $\times 2, \times 4, \times 8$ ).
- Example: On the Pavia dataset at $\times4$ , it improved PSNR by 0.31 dB and reduced SAM (Spectral Angle Mapper) by 0.005 compared to the second-best method.
Arbitrary-Scale Performance: Compared against SQformer (the only other arbitrary-scale HSI method) and general image SR methods (SRNO, AnyTSR, GaussianSR).
- GaussianHSI demonstrated superior out-of-distribution generalization, performing well on unseen scales ( $\times6, \times8$ ) after training only on $\times2$ to $\times4$ .
- It achieved the best balance of spatial detail (high PSNR/SSIM) and spectral fidelity (low SAM).
Ablation Studies:
- Removing VBGS or SDE resulted in significant performance drops, confirming the necessity of both adaptive spatial selection and spectral enhancement.
- Sensitivity analysis showed that selecting top-16 Gaussians ( $k=16$ ) offered the optimal trade-off between reconstruction quality and computational efficiency.

5. Significance

This paper bridges the gap between the continuous representation capabilities of Gaussian Splatting and the specific demands of hyperspectral imaging. By introducing Voronoi-guided selection and bilateral weighting, the authors solve the adaptivity and spectral consistency issues that prevented previous Gaussian methods from working on HSIs. The ability to perform arbitrary-scale reconstruction with a single model significantly reduces the computational overhead and storage requirements associated with training multiple scale-specific models, making it highly practical for remote sensing applications where target resolutions can vary dynamically.