💻 computer science

Beyond GSD-as-Token: Continuous Scale Conditioning for Remote Sensing VLMs

This paper introduces ScaleEarth, a parameter-efficient fine-tuning framework for remote sensing vision-language models that treats ground sampling distance (GSD) as a continuous conditioning variable to dynamically route computation via CS-HLoRA, while simultaneously predicting GSD from visual features and leveraging a newly constructed scale-aware dataset to achieve state-of-the-art performance across diverse Earth-system tasks.

Original authors: Song Zhang, Yanlong Chen, Yilin Li, Yining Chen, Zili Yi, Xiaowei Zhang, Yawei Li

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Song Zhang, Yanlong Chen, Yilin Li, Yining Chen, Zili Yi, Xiaowei Zhang, Yawei Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Zoom" Confusion

Imagine you are looking at a city from two different heights:

From a drone hovering 10 feet up: You can see individual cars, the color of a roof, and a person walking a dog.
From a satellite 10 miles up: You can't see cars or people. You only see a patch of green (a park) or a gray grid (a neighborhood).

In the world of Remote Sensing (taking pictures of Earth from above), the distance from the ground is called GSD (Ground Sampling Distance). The problem is that current AI models (Vision-Language Models) are bad at handling this.

Old Way 1 (The Blind Model): The AI looks at the picture and tries to guess what it is, ignoring the height entirely. It might try to count individual cars in a blurry satellite photo, which is impossible.
Old Way 2 (The Text Token): The AI is told, "This photo is taken from 10 miles up," but it just treats that sentence like any other word in a story. It doesn't actually change how it thinks or processes the image. It's like telling a chef, "This is a spicy dish," but the chef still tastes it as if it were mild.

The Solution: ScaleEarth

The authors built a new system called ScaleEarth. Instead of just reading the height as a word, they made the height a physical dial that actually changes the AI's brain while it's thinking.

Think of the AI's brain as a Swiss Army Knife with different tools:

Tiny Tools: For seeing fine details (cars, people, roof tiles).
Medium Tools: For seeing structures (roads, buildings, blocks).
Big Tools: For seeing the big picture (forests, cities, coastlines).

ScaleEarth has a special mechanism that automatically picks the right tools based on how high the camera is.

If the camera is close (low GSD), it unlocks the "Tiny Tools" and locks the "Big Tools."
If the camera is far (high GSD), it locks the "Tiny Tools" (because they would just see noise) and unlocks the "Big Tools."

This happens automatically and smoothly, not by switching between different AI models, but by adjusting the same model's settings in real-time.

How They Built It (The Three Parts)

1. The "Smart Dial" (CS-HLoRA)

This is the core invention. The researchers created a "gate" inside the AI that controls which parts of its brain are active.

The Analogy: Imagine a dimmer switch for a lightbulb. Instead of just turning the light on or off, you can slide the switch to exactly the right brightness.
How it works: The AI takes the physical distance (GSD) as a number. It uses a mathematical formula to decide exactly how much to "turn on" the detail-focused parts of its brain versus the big-picture parts. This allows the AI to adapt perfectly to any zoom level.

2. The "Guessing Eye" (SSE-U)

Sometimes, the photo doesn't come with a label saying how high the camera was.

The Analogy: Imagine you are handed a photo without a caption. You look at the trees and buildings and say, "Based on how small the trees look, I'd guess this photo was taken from about 1,000 feet up. I'm pretty sure, but maybe I'm off by a little bit."
How it works: The system has a small helper module that looks at the image and estimates the height and how confident it is. If the image is very clear, it guesses accurately. If the image is blurry or weird, it says, "I'm not sure," and the system defaults to a safe, middle-ground setting so it doesn't make a wild guess.

3. The "Training Manual" (GeoScale-VQA)

To teach the AI this new skill, the researchers created a massive new textbook called GeoScale-VQA (1.5 million questions and answers).

The Analogy: Instead of just showing the AI a picture of a car and asking "What is this?", they showed it the same picture at different zoom levels.
- At close range: "What color is the car?" (Answer: Red).
- At far range: "What kind of area is this?" (Answer: A parking lot).
The Loop: They made sure the questions asked matched the zoom level. You wouldn't ask "What color is the car?" if the photo was too blurry to see the car. This created a perfect feedback loop where the AI learned that Zoom Level = Different Questions = Different Answers.

The Results

When they tested this new system:

It beat all other remote-sensing AI models on standard tests.
It was much better at answering questions that required understanding the "scale" (e.g., counting cars vs. counting neighborhoods).
It worked even when the height information was missing, thanks to the "Guessing Eye."

Summary

ScaleEarth is like giving a remote-sensing AI a pair of smart glasses.

Old AIs put on the same glasses for everything, so they either squint too hard at blurry photos or miss the big picture in detailed ones.
ScaleEarth automatically adjusts the lenses based on how far away the object is. It knows when to look for tiny details and when to step back and look at the landscape, all without needing a human to tell it what to do.

Technical Summary: Beyond GSD-as-Token: Continuous Scale Conditioning for Remote Sensing VLMs

1. Problem Statement

Remote Sensing Vision-Language Models (RS-VLMs) face a fundamental mismatch with their natural-image counterparts due to the extreme variability in Ground Sampling Distance (GSD). While natural images typically exist within a stable scale range, RS imagery spans over two orders of magnitude, from sub-decimeter aerial imagery (~~0.06 m/pixel) to decameter satellite products (~~10 m/pixel or coarser).

Existing RS-VLMs fail to address this physical reality effectively through two primary limitations:

Scale-Agnostic Fine-Tuning: These models use static parameters across all resolutions, forcing the model to collapse distinct visual regimes (e.g., fine textures vs. blurred regions) into a single averaged representation, degrading both fine-detail recognition and coarse regional reasoning.
GSD-as-Token: Current approaches inject GSD metadata as discrete text tokens (e.g., "0.4 m resolution"). This method tokenizes the scalar as text, inducing no parameter-level adaptation and failing entirely when sensor metadata is missing or unreliable.

Neither approach treats GSD as a continuous physical variable that should dynamically govern the model's internal computation path.

2. Methodology: ScaleEarth

The authors propose ScaleEarth, a parameter-efficient fine-tuning framework built on the Qwen3-VL-8B backbone. The core innovation is treating GSD not as passive metadata, but as a continuous conditioning variable that drives the model's computation. The framework consists of three main components:

A. CS-HLoRA (Continuous Scale-Conditioned Hyper-LoRA)

CS-HLoRA is the central adaptation mechanism. It modifies the standard Low-Rank Adaptation (LoRA) update path by inserting a GSD-driven gate between the down-projection and up-projection matrices.

Mechanism: The update is defined as $\Delta W(s) = \frac{\alpha_{lora}}{r} B \text{diag}(h(s)) A$ , where $s = \log_{10}(\text{GSD})$ .
Gating Function: The gate $h_k(s) = \sigma(\alpha(\tau_k - s))$ is a sigmoid function that softly selects LoRA rank dimensions based on the input scale.
Physical Tier Initialization: The 64 rank dimensions are initialized into three physically motivated tiers based on the minimum mapping unit theory:
- Object Ranks (0–10): Active at high resolution (GSD < 1 m) for fine-grained details.
- Structure Ranks (11–31): Active at mid-to-low resolution (1–10 m) for layout and structural relations.
- Semantic Ranks (32–63): Active across the full scale range for landscape-level semantics.
Continuous Routing: As GSD varies, the active LoRA dimensions shift smoothly, replacing the averaged policy of scale-agnostic fine-tuning with a scale-dependent adaptation path without modifying the pretrained backbone.

B. SSE-U (Heteroscedastic Scale Estimation)

To ensure the framework operates without reliance on sensor metadata at deployment, ScaleEarth includes a lightweight heteroscedastic sub-head, SSE-U.

Function: It predicts the GSD ( $\hat{\mu}$ ) and its uncertainty ( $\hat{\sigma}^2$ ) directly from visual features (pooled ViT features).
Training: It is trained via Gaussian negative log-likelihood (NLL) on samples with exact GSD annotations.
Inference: If metadata is missing, the model uses the predicted GSD if the uncertainty is low; otherwise, it falls back to a neutral anchor scale. This allows for uncertainty-aware scale-conditioned decoding.

C. GeoScale-VQA Dataset

To provide matching supervision, the authors constructed GeoScale-VQA, a 1.5M-sample scale-layered RS-VQA corpus.

Construction: It combines existing datasets (RSVQA-HR/LR, RSVQA-xBEN, GeoLLaVA-8K, PatternNet, MtSCCD) with newly generated samples.
Scale-Aware Generation: Question-answer pairs are generated conditioned on the image's GSD tier (High: <0.2m, Mid: 0.2–1.0m, Low: >1.0m). High-tier prompts emphasize object details, mid-tier focuses on spatial layout, and low-tier targets landscape semantics.
Quality Control: A two-stage filtering process (syntactic checks followed by Claude-supervised quality screening) ensures factual consistency and scale alignment.

D. Training Pipeline

ScaleEarth employs a two-stage training strategy:

Stage 1 (RS-Domain SFT): Full-parameter supervised fine-tuning on RS-GPT4V to close the natural-image-to-RS distribution gap.
Stage 2 (Scale-Conditioned Adaptation): The backbone is frozen (4-bit quantized). CS-HLoRA adapters and the SSE-U head are jointly trained on GeoScale-VQA. The objective combines the VLM loss with a GSD estimation loss ( $\mathcal{L} = \mathcal{L}_{VLM} + \lambda_{gsd}\mathcal{L}_{nll}$ ).

3. Key Contributions

CS-HLoRA: A novel fine-tuning method that modulates the LoRA low-rank subspace through a GSD-driven gate with physically grounded tier initialization. It enables a unified model to adapt computation to physical scale without architectural modification or separate resolution-specific adapters.
SSE-U: A heteroscedastic scale-estimation head that predicts GSD and uncertainty from visual features alone, reducing dependence on metadata and supporting uncertainty-aware decoding.
GeoScale-VQA: A 1.5M-sample scale-aware RS-VQA corpus that forms a closed method-data loop, where supervision and parameter-efficient adaptation are driven by the same physical scalar (GSD).

4. Experimental Results

The model was evaluated on two comprehensive benchmarks: XLRS-Bench (ultra-high-resolution perception and reasoning) and OmniEarth-Bench (Earth-system cognition across six spheres).

XLRS-Bench: ScaleEarth achieved a new state-of-the-art average accuracy of 57.4%, outperforming the strongest specialized baseline (GeoEyes, 54.2%) by +3.2% and the best open-source generalist (Qwen3-VL-235B-A22B, 51.1%) by +6.3%, despite using only an 8B backbone. Significant gains were observed in scale-sensitive tasks like Regional Land Use Classification (78.6%) and Object Motion State (66.1%).
OmniEarth-Bench: ScaleEarth achieved an average accuracy of 40.71% on multiple-choice tasks, exceeding the strongest zero-shot open-source baseline (InternVL3-72B, 33.26%) by +7.45 percentage points and the strongest closed-source baseline (Claude-3.7-Sonnet, 29.07%) by +11.64 percentage points.
Ablation Studies: Removing CS-HLoRA reduced accuracy by 4.2%, while removing scale-aware supervision caused a 3.8% drop. Randomizing the physical tier initialization further decreased performance by 3.5%, confirming the utility of the cartographic prior.
GSD-Spoofing Tests: When the GSD input was systematically perturbed, ScaleEarth exhibited a smooth, bell-shaped accuracy curve peaking at the true GSD. In contrast, discrete bucket-based routing (MoE-LoRA) showed sharp discontinuities at bucket boundaries, and standard LoRA or text-token injection showed flat responses, proving that the gains stem from continuous physical conditioning rather than just added capacity.

5. Significance and Claims

The paper argues that the closed loop established by ScaleEarth among architecture, supervision, and data suggests a broader design principle for RS-VLMs: physical variables that fundamentally distinguish overhead imagery from natural images should shape the model directly and systematically, rather than being discarded or inserted merely as auxiliary text prompts.

The authors claim that ScaleEarth demonstrates that:

Treating GSD as a continuous conditioning variable allows a single model to dynamically route computation across object-, structure-, and semantic-level tiers.
This approach outperforms both scale-agnostic models and those that treat scale as a discrete text token or bucketed routing signal.
The framework remains robust in deployment scenarios where sensor metadata is missing, thanks to the uncertainty-aware fallback provided by SSE-U.

The work positions itself as a step toward more reliable, resolution-aware remote sensing AI, particularly for tasks requiring fine-grained perception in ultra-high-resolution scenarios or cross-scale reasoning in complex Earth-system tasks.