🤖 AI

Beyond Visual Fidelity: Benchmarking Super-Resolution Models for Large-Scale Remote Sensing Imagery via Downstream Task Integration

This paper introduces GeoSR-Bench, a novel benchmark dataset that evaluates super-resolution models for remote sensing by directly linking image enhancement to downstream Earth monitoring tasks, revealing that traditional fidelity metrics often fail to predict or even negatively correlate with actual task performance.

Original authors: Zhili Li, Kangyang Chai, Zhihao Wang, Xiaowei Jia, Yanhua Li, Gengchen Mai, Sergii Skakun, Dinesh Manocha, Yiqun Xie

Published 2026-05-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhili Li, Kangyang Chai, Zhihao Wang, Xiaowei Jia, Yanhua Li, Gengchen Mai, Sergii Skakun, Dinesh Manocha, Yiqun Xie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a blurry, low-quality photo of a city taken from a high-flying satellite. You want to see the details: the individual houses, the winding roads, the specific types of trees. Super-Resolution (SR) is like a magic tool that tries to sharpen that blurry photo, turning it into a crisp, high-definition image.

For a long time, scientists tested these magic tools by asking: "Does the new photo look mathematically similar to a real high-definition photo?" They used strict rulers called PSNR and SSIM to measure pixel-by-pixel accuracy. If the numbers were high, the tool was considered "good."

The Problem:
This paper argues that measuring how "mathematically perfect" a photo looks isn't enough. It's like judging a chef solely by how perfectly they chopped an onion, without ever tasting the soup they made. Just because a sharpened image looks nice on a ruler doesn't mean it actually helps a computer solve real-world problems, like counting cars, mapping flood zones, or identifying crop types.

The Solution: GeoSR-Bench
The authors built a new testing ground called GeoSR-Bench. Instead of just checking if the photo looks sharp, they put the sharpened photos to work in real "downstream tasks."

Think of it this way:

Old Way: You sharpen a photo and ask, "Does this look like a high-res photo?"
New Way (GeoSR-Bench): You sharpen a photo, then immediately hand it to a computer program and ask, "Can you now count the number of houses in this image?" or "Can you tell me how tall these trees are?"

The Experiment
The researchers gathered a massive library of satellite images from around the world, covering everything from giant forests to dense cities. They created two main "levels" of difficulty:

Coarse to Medium: Turning a very blurry, 500-meter-wide view (like looking at a city from a plane) into a 30-meter view (like looking from a helicopter).
Medium to High: Turning a 10-meter view (like a drone) into a 0.6-meter view (like a bird's-eye view from a very low altitude).

They tested 9 different types of "magic tools" (including GANs, Transformers, and Diffusion models) against 10 different real-world tasks, such as:

Finding roads and buildings.
Mapping where crops are growing.
Estimating how much carbon trees are storing.

The Big Surprise
The results were eye-opening. The authors found that the tools that scored the highest on the "mathematical rulers" (PSNR/SSIM) were often the worst at solving the real-world tasks.

The Analogy: Imagine two artists. Artist A paints a picture that is mathematically perfect but looks a bit blurry and smooth. Artist B paints a picture that has some "noise" or texture but captures the sharp edges of a building perfectly.
- The "Mathematical Ruler" loves Artist A because the pixels match the original perfectly.
- The "Real-World Task" (like a robot trying to find the building) loves Artist B because the edges are clear, even if the pixels aren't a perfect match.

In many cases, the correlation was even negative. This means the tool that looked "best" on paper actually made the computer's job harder or led to wrong answers.

The Takeaway
The paper concludes that we can't just rely on "visual fidelity" (how pretty or mathematically accurate the image looks) to judge these AI models. If we want these tools to help with disaster response, urban planning, or farming, we need to test them on the actual jobs they are supposed to do.

What They Did Next
To fix this, the authors released their entire dataset, code, and the trained models to the public. They want other scientists to stop just chasing "pretty pictures" and start building AI that is actually useful for solving Earth-monitoring problems.

In a Nutshell:
Don't just ask if the picture looks sharp. Ask if the picture helps you do the job. The paper proves that the "sharpest" looking image isn't always the most useful one.

1. Problem Statement

Super-resolution (SR) techniques for remote sensing have traditionally been evaluated using visual fidelity metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). These metrics measure pixel-level accuracy or perceptual similarity to a ground-truth high-resolution image.

However, the paper argues that these metrics are insufficient for remote sensing applications because:

Misalignment with Utility: High visual fidelity does not necessarily translate to improved performance in real-world downstream tasks (e.g., land cover classification, infrastructure mapping, biomass estimation).
Spurious Details: Generative models (like GANs) may produce visually sharp but spectrally incorrect details ("hallucinations") that degrade the performance of downstream analytical models.
Lack of Benchmarks: Existing SR benchmarks (e.g., DIV2K, SEN2VENµS) focus on natural images or synthetic downscaling and lack integration with specific Earth observation tasks.

2. Methodology: GeoSR-Bench

The authors introduce GeoSR-Bench, the first benchmark dataset designed to evaluate SR models based on their utility in downstream tasks.

A. Dataset Construction

The dataset comprises ~36,000 locations globally, featuring spatially co-located and temporally aligned image pairs across two cross-platform SR tasks:

Coarse-to-Medium: MODIS (500m) $\rightarrow$ Landsat-8 (30m).
Medium-to-High: Sentinel-2 (10m) $\rightarrow$ NAIP (0.6m).

Data Processing & Quality Control:

Coincident Pairing: Used Apache Sedona to perform spatio-temporal joins across millions of images, ensuring pairs are acquired within a 1-day window.
Stratified Sampling: To avoid bias toward homogeneous landscapes (e.g., forests), the dataset uses stratified sampling to ensure an equal proportion of urban and non-urban patches.
Filtering: Rigorous cloud/snow masking (using QA bands) and manual visual inspection to remove temporal inconsistencies (e.g., construction, floods) and spectral misalignments.
Spectral Alignment: Used surface reflectance products and upsampled bands to match resolutions (e.g., Sentinel-2 20m/60m bands upsampled to 10m).

B. Downstream Task Integration

For each of the two SR tasks, the benchmark includes 5 downstream datasets (10 total), covering pixel-level classification and regression:

Classification: River segmentation, Urban land cover, Cropland Data Layers (CDL), Building footprints, Road detection, Vegetation cover.
Regression: Gross Primary Production (GPP) estimation, Canopy Height Model (CHM) estimation.
Labels: Derived from high-quality sources (e.g., Microsoft Building Footprints, ESA WorldCover, USDA NAIP) and aligned with the coincident image pairs.

C. Evaluation Protocol

The evaluation follows a two-stage process:

SR Training: 9 representative SR models (covering Transformers, Neural Operators, GANs, and Diffusion) are pre-trained on the SR datasets.
Downstream Evaluation:
- SR models are fine-tuned for specific downstream tasks.
- Downstream task models (U-Net, SegFormer, Mask2Former) are trained on the generated SR images.
- Performance is compared against: (a) Original Low-Resolution (LR) inputs, (b) Original High-Resolution (HR) ground truth, and (c) Generated SR outputs.

3. Key Contributions

GeoSR-Bench Dataset: A large-scale, quality-controlled dataset integrating SR tasks with 10 diverse downstream applications across global and US-specific regions.
Task-Integrated Evaluation Framework: A protocol that moves beyond PSNR/SSIM to measure the "real utility" of SR by quantifying improvements in land cover segmentation, infrastructure mapping, and biophysical variable estimation.
Comprehensive Benchmarking: Evaluation of 9 state-of-the-art SR models across 270 experimental settings (2 SR tasks $\times$ 9 models $\times$ 3 downstream models $\times$ 5 tasks).
Open Source: Full release of the dataset, trained models, and evaluation code to facilitate future research.

4. Key Results

The experiments yielded several critical findings:

Weak Correlation between Fidelity and Utility:
- Traditional metrics (PSNR/SSIM) often do not correlate with downstream task performance. In many competitive scenarios (top-k models), the correlation is weak or even negative.
- Models with the highest PSNR (often Transformer-based or Neural Operators using L1 loss) do not always yield the best downstream results.
- Conversely, models with lower PSNR (e.g., GAN-based SeD) sometimes outperform others in downstream tasks because they recover fine spatial textures critical for object boundaries (e.g., roads, buildings), despite spectral inaccuracies.
Performance Gaps:
- While SR models generally improve over the original Low-Resolution inputs, a significant performance gap remains between SR outputs and the original High-Resolution ground truth.
- For the MODIS-to-Landsat-8 task, Transformer and Neural Operator models led in downstream performance.
- For the Sentinel-2-to-NAIP task, the GAN-based SeD model showed the strongest gains (up to 30% improvement over LR baseline) due to its ability to recover fine-grained textures.
Sub-Pixel Limitations:
- Experiments on the TreeFinder dataset (detecting individual trees smaller than the Sentinel-2 pixel size) showed that SR provides minimal utility (0-4% gain) for sub-pixel tasks. The spectral dilution in the coarse input makes it impossible to accurately recover the precise location of small targets, regardless of the SR model's visual fidelity.

5. Significance

Paradigm Shift: The paper challenges the community to stop optimizing SR models solely for visual metrics. It argues that downstream task performance should be the primary metric for evaluating SR in Earth observation.
Guidance for Model Selection: The findings suggest that selecting an SR model based on PSNR/SSIM can be misleading. Researchers and practitioners must evaluate models based on the specific application (e.g., choosing GANs for infrastructure mapping where edge sharpness is critical, even if PSNR is lower).
Future Direction: The results highlight the need for new SR architectures and loss functions that are explicitly designed to preserve task-relevant features (e.g., semantic boundaries, spectral consistency) rather than just pixel-level accuracy.

In conclusion, GeoSR-Bench establishes that "better looking" images do not always mean "better data" for analysis, necessitating a new era of task-integrated super-resolution development.