💻 computer science

Finding Change in Satellite Archives from Text: How to Combine Before-and-After Images Efficiently

This paper presents a controlled evaluation of eight fusion module designs for efficiently matching natural language queries to bi-temporal satellite image archives, demonstrating that a training-free two-stage search significantly reduces query costs while maintaining performance, and revealing that memory-bound linear models like Mamba offer no speed advantage over attention mechanisms at typical vision scales.

Original authors: Simon Roy, Mark Bong, Giovanni Beltrame

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Simon Roy, Mark Bong, Giovanni Beltrame

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of a crime scene, your office is the entire planet, and your evidence is a massive library of satellite photos taken from space. Every day, new pictures arrive, showing cities growing, forests shrinking, or storms rolling in. The problem? There are too many photos to look at one by one. You need a way to ask a simple question in plain English, like "Show me where a new building popped up," and have the computer instantly find the exact "before" and "after" pictures that match your description. This is the world of Earth observation, where scientists use special computer brains called "vision-language models" to understand both images and words. These models are like super-smart translators that can look at a photo and a sentence and decide if they are talking about the same thing. But here's the catch: to find the right photos, the computer has to compare two images at once (the "before" and the "after") and figure out exactly what changed between them. Doing this math for thousands of photo pairs is slow and expensive, like trying to solve a giant puzzle while running a marathon. If the computer is too slow, you can't ask your questions in real-time, and the whole system becomes useless for things like checking storm damage or tracking urban growth.

So, a team of researchers set out to find the fastest, most efficient way to make this "change-finding" magic happen. They treated the problem like a cooking competition, testing eight different "fusion modules"—the special kitchen tools that mix the "before" and "after" images together to spot the differences. They wanted to know: which tool gives the best taste (accuracy) without burning too much fuel (computing power)? They tested these tools on two famous photo libraries: one full of high-resolution city photos from Texas (LEVIR-CC) and another with lower-resolution land images from Dubai (Dubai-CC).

First, they looked at the "fancy" tools. One popular idea in the tech world right now is using a new type of engine called Mamba, which is supposed to be incredibly fast at reading long lists of information. The researchers thought, "Maybe this new engine will zoom through our photo pairs!" But when they actually ran the tests, the Mamba engine didn't win the race. Even though it looked great on paper, it got stuck in traffic caused by the computer's memory. At the size of the photo patches they were using (196 pieces per image), the old-school "attention" method—which is like having a team of detectives look at every pair of clues at once—was actually faster because it could use the computer's parallel power more efficiently. The new Mamba engine only started to show its speed advantage if the photos were huge, like a massive stack of tiles, which isn't the standard size for these tasks.

Next, they tested a clever trick called "compression." Imagine you have two thick books of clues (the before and after images). Instead of reading every single page of both books, you quickly summarize them into a thinner, lighter notebook before you start comparing them. The researchers built a tool called "Temporal Bottleneck Fusion" (TBF) that does exactly this. They found that this tool was a winner. It made the computer 1.6 times faster and used 2.3 times fewer memory resources than the heavy, unsummarized version, while still finding the right photos almost as well as the best possible method. The only tiny cost was a very small drop in how perfectly it described the change (a difference so small it's almost invisible to the naked eye).

Finally, the team discovered a "two-step" strategy that was even better. Instead of checking every single photo pair with the expensive, high-quality tool, they used a super-cheap, lightning-fast "difference filter" first. This filter is like a metal detector that quickly sweeps through the library and picks out the top 25 most likely suspects. Then, they only used the expensive, high-quality tool to re-check those 25. This approach slashed the cost of finding an answer by 10 to 15 times! It was so effective that it found the right photos just as often as checking every single pair, but in a fraction of the time.

In the end, the paper suggests that for finding changes in satellite photos, we don't need the newest, flashiest engines like Mamba for standard-sized images. Instead, the best approach is to use a smart, compressed version of the classic attention method, or even better, to use a two-step process where a cheap filter does the heavy lifting first. This means we can build systems that answer our questions about the changing Earth almost instantly, without needing super-computers for every single search.

Technical Summary: Finding Change in Satellite Archives from Text

Problem Definition
Operational Earth observation (EO) increasingly requires answering natural language queries such as "find the image pairs where a new building appeared." This necessitates searching archives of bi-temporal (before-and-after) satellite image pairs and ranking them based on how well they match a text description of a change. While vision-language models like CLIP can embed images and text in a shared space, the specific challenge of change retrieval requires fusing two temporal views before matching them to text. In realistic deployments, the expensive vision backbone is computed once per frame during ingestion, but the fusion module must run at query time for every candidate pair formed by the requested region and time interval. Consequently, the speed and memory footprint of this fusion module largely dictate the cost and feasibility of interactive search, particularly on constrained platforms.

Methodology
The authors conduct a controlled comparison of eight fusion architectures designed to combine "before" and "after" image features. The study utilizes a fixed, frozen CLIP ViT-B/16 encoder to extract patch features ( $L=196$ patches, $D_{in}=768$ ), which are projected to a width of $D=320$ before fusion. All models are trained using the same recipe (AdamW, symmetric InfoNCE objective) over ten random seeds on two benchmarks: LEVIR-CC (high-resolution urbanization) and Dubai-CC (Landsat 7 tiles).

The evaluated architectures fall into three families:

Attention-Based:
- Concatenation Transformer: Concatenates features along the channel axis ( $L \times 2D$ ) and processes them with a standard Transformer. This doubles the feature width, quadrupling the cost of linear and feed-forward layers.
- TFF (Text-ITSR): Applies cross-attention between difference features and original bi-temporal features.
State-Space Models (SSM/Mamba):
- Concatenation Mamba: Replaces the Transformer with Mamba blocks at width $2D$ .
- Bottleneck Mamba: Compresses features to width $D$ before Mamba blocks.
- Interleaved Mamba: Interleaves temporal tokens into a single sequence of length $2L$ .
Learned Compression:
- Temporal Bottleneck Fusion (TBF): Applies a "concat-then-reduce" pattern. An MLP projects concatenated features back to width $D$ before the self-attention mechanism, reducing the quadratic cost of the attention layers while preserving global interaction.
Baselines: Simple element-wise subtraction and an MLP fusion (channel mixing without token interaction).

Additionally, the authors propose a training-free two-stage retrieval cascade: a cheap subtraction model shortlists candidates, followed by a re-ranking step using an attention-based module (TBF or Concatenation Transformer) on the top- $N$ pairs.

Key Results

Efficiency vs. Accuracy Trade-off: The Temporal Bottleneck Fusion (TBF) emerges as the most favorable operating point. It reduces parameters by 2.3 $\times$ and latency by 1.6 $\times$ compared to the full Concatenation Transformer, with a negligible drop in change-only BLEU-1 score (0.007 difference).
Mamba Performance: Contrary to theoretical expectations of linear-time complexity, Mamba variants did not offer wall-clock speed benefits at standard vision transformer sequence lengths ( $L=196$ ). The selective scan is memory-bandwidth bound and sequential, whereas attention maps efficiently to parallel tensor cores. Mamba only becomes faster than attention at sequence lengths exceeding $L \approx 400$ .
Cascaded Retrieval: The proposed two-stage search (Subtraction $\to$ TBF) achieves 10–15 $\times$ reduction in query cost on LEVIR-CC while matching or exceeding the recall of full-fusion baselines. On the smaller Dubai-CC dataset, it achieved comparable recall at roughly 2.6 $\times$ lower cost.
Robustness: The relative performance ranking of the fusion modules remained consistent when tested with alternative frozen backbones (GeoRSCLIP and SigLIP 2), confirming the findings are not specific to the CLIP architecture.

Significance and Claims
The paper claims to provide the first controlled, statistically grounded comparison of attention, bottleneck, and SSM-based fusion for text-based change retrieval. Its primary contributions are:

Practical Efficiency: Demonstrating that a simple compression strategy (TBF) offers a superior efficiency-quality balance for standard satellite tile sizes, reducing computational load without significant accuracy loss.
Hardware Reality Check: Highlighting that theoretical complexity advantages of SSMs (like Mamba) do not translate to latency benefits at typical vision sequence lengths due to hardware memory bottlenecks.
Deployment Strategy: Validating that a training-free, multi-stage retrieval cascade can drastically reduce the cost of searching large archives while maintaining high recall, making interactive text-based change retrieval more feasible for operational systems.

The authors conclude that for standard patch counts, optimized attention (specifically with bottleneck compression) should remain the default, and that fusion need not be computed for every candidate pair in an archive.

Technical Summary: Finding Change in Satellite Archives from Text

More like this