💻 computer science

CraterBench-R: Instance-Level Crater Retrieval for Planetary Scale

This paper introduces CraterBench-R, a large-scale benchmark for instance-level planetary crater retrieval, and proposes a scalable two-stage pipeline combining self-supervised Vision Transformers with a novel instance-token aggregation method to achieve high accuracy while significantly reducing storage and computational costs.

Original authors: Jichao Fang, Lei Zhang, Michael Phillips, Wei Luo

Published 2026-04-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jichao Fang, Lei Zhang, Michael Phillips, Wei Luo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a planetary archaeologist trying to organize a massive library of photos taken from space. These photos show the surface of Mars, covered in thousands of impact craters.

The Problem: The "Lost in Translation" Library
Right now, scientists have great AI tools that can find craters (like a librarian who can point to a book on a shelf). But they struggle to match them.

The Scenario: You take a photo of a specific crater from one angle. You want to find that exact same crater in a different photo taken from a different angle, with different lighting, or zoomed in/out.
The Failure: Current AI is like a person who only remembers the "general vibe" of a book. If you ask, "Show me the book about the red dragon," the AI might show you any book with a dragon on the cover, even if it's a different story. It loses the specific details that prove it's the same crater.

The Solution: CraterBench-R
The authors created a new "test library" called CraterBench-R. It contains 25,000 unique craters, each with multiple photos (different zoom levels and lighting). They manually checked every single match to ensure the test is fair. This is the "exam" they used to test their new AI methods.

The Discovery: Don't Summarize, Remember the Details
The researchers tested many AI models. They found a surprising truth:

The Old Way (Pooling): Most AI tries to shrink a whole image into one single "summary number" (like writing a one-sentence summary of a 300-page novel). This works okay for simple things, but for craters, it throws away too much detail. It's like trying to recognize a friend by only remembering "they have a nose."
The New Way (Token Matching): Instead of one summary, the AI keeps hundreds of tiny "notes" (tokens) describing different parts of the crater (the rim, the shadow, the texture).
The Result: Using these hundreds of notes is like recognizing a friend by their smile, their walk, and their voice. It works much better.

The Innovation: The "Smart Filing System"
Here's the catch: Keeping hundreds of notes for every single crater in a planetary database is too heavy. It's like trying to carry a library in your backpack; it's too slow and takes up too much space.

The team invented a clever trick called Instance-Token Aggregation. Think of it like this:

The Problem: You have 196 sticky notes describing a crater. You can't carry all of them.
The Old Solution: Throw them all away and keep just one (too much detail lost).
The New Solution (Aggregation): You pick the 16 most important notes (the "seeds"). Then, you look at the other 180 notes and say, "Hey, this note is similar to Seed #1, so I'll attach it to Seed #1."
The Magic: You now have a tiny, super-dense "cluster" of 16 notes that still holds all the important details of the original 196. It's like compressing a high-resolution photo into a smaller file without losing the face.

The Final Pipeline: The Two-Step Hunt
To make this fast enough for real-world use, they built a two-step search engine:

Step 1 (The Net): Use a fast, simple search to pull up a "shortlist" of maybe 100 potential matches. (This is like casting a wide net to catch any fish that might be the one you want).
Step 2 (The Inspection): Take those 100 candidates and use the "Smart Filing System" (the 16-note clusters) to do a deep, detailed comparison.
- Result: This finds the right crater 89–94% of the time, but it's incredibly fast (milliseconds per search).

Why This Matters
This isn't just about craters. It's a new way to search through massive databases of images (like satellite photos of Earth, medical scans, or security footage) where you need to find the exact same object even if the lighting or angle changes.

In a Nutshell:
The authors realized that summarizing a crater kills its identity. Instead, they taught the AI to keep a "cheat sheet" of the most important details, compress it smartly, and use a two-step process to find matches. It's like upgrading from a blurry Polaroid to a high-definition, searchable 3D model of the universe.

1. Problem Statement

While deep learning has advanced crater detection (locating craters and estimating size), it fails to address critical scientific workflows that require instance-level retrieval. These workflows include:

Catalog deduplication: Identifying the same crater across overlapping image footprints.
Cross-observation matching: Linking views of the same physical crater across different scales, contexts, and illumination conditions.
Morphological analog discovery: Grouping craters by degradation state or shape.

Challenges:

Visual Complexity: Planetary craters exhibit extreme variations in degradation (erosion, infilling), illumination, and scale.
Representation Bottleneck: Standard Vision Transformers (ViTs) collapse spatial information into a single global vector (e.g., CLS token), losing critical local morphological details.
Data Scarcity: Unlike terrestrial datasets, planetary datasets often have very few views per identity (often only 2), making supervised metric learning (fine-tuning) ineffective or even detrimental.
Scalability: Retaining all patch tokens (e.g., 196 tokens) for "late interaction" matching provides high accuracy but is computationally and storage-prohibitive at a planetary scale (millions of images).

2. Methodology

The authors propose a comprehensive pipeline involving a new benchmark, a novel aggregation method, and a two-stage retrieval system.

A. CraterBench-R Benchmark

Dataset: Curated from Mars CTX imagery and the Robbins catalog.
Scale: 25,000 crater identities, 50,000 gallery images (2× and 3× context crops), and 5,000 manually verified multi-view queries.
Task: Strict instance-level retrieval (matching the exact same physical crater ID).
Relevance: Uses a "cluster-tolerant" definition where a query is relevant if it shares a crater ID with the gallery image, handling cases where small craters overlap.

B. Baseline Diagnosis

The authors evaluated 30 frozen backbones (CNNs, generic SSL ViTs, and domain-specific models).

Finding 1: Self-supervised ViTs (specifically DINO family) significantly outperform CNNs and supervised models.
Finding 2: Domain pretraining is crucial. MarsDINO (trained on Mars imagery) outperformed generic large models (e.g., DINOv3-7B) despite having far fewer parameters.
Finding 3: Supervised fine-tuning degrades performance. Due to the scarcity of positive views (only 2 per crater), fine-tuning with losses like SupCon or ArcFace disrupted the pre-trained token structure, lowering retrieval accuracy.

C. Instance-Token Aggregation (The Core Innovation)

To bridge the gap between single-vector efficiency and dense-token accuracy without training, the authors propose a training-free, deterministic compression scheme:

Seed Selection: Select $K$ $K$ seed tokens from the $N=196$ $N = 196$ patch tokens.
- Strategy A (Attention): Select top- $K$ based on CLS-to-patch attention weights (prioritizing salient regions).
- Strategy B (FPS): Farthest-Point Sampling in cosine space (prioritizing diversity).
Assignment: Assign remaining tokens to the nearest seed based on cosine similarity (hard assignment).
Aggregation: Create a new instance token by combining the seed with the mean of its assigned cluster tokens:
$z_k = \ell_2 \left( t_{s_k} + \frac{1}{\max(|C_k|, \epsilon)} \sum_{i \in C_k} t_i \right)$
- Key Advantage: This "residual" form preserves the identity of the seed (unlike K-means centroids which smooth out details), retaining local morphological structure.

D. Two-Stage Retrieval Pipeline

To make dense matching feasible at scale:

Stage 1 (Shortlisting): Use a single-vector index (FAISS) with a standard pooling method (CLS or GeM) to retrieve a small candidate set (Top- $S$ ).
Stage 2 (Reranking): Apply Late Interaction (ColBERT-style max-similarity) using the compressed Instance Tokens ( $K \ll 196$ ) only on the shortlisted candidates.

3. Key Contributions

CraterBench-R: The first curated instance-level retrieval benchmark for planetary science, featuring 25k identities and rigorous evaluation protocols.
Baseline Diagnosis: Demonstrated that single-vector pooling imposes a low accuracy ceiling and that supervised fine-tuning fails in low-view planetary regimes, establishing the need for frozen, token-level approaches.
Instance-Token Aggregation: A novel, training-free method to compress ViT patch tokens into $K$ discriminative tokens, preserving local structure better than classical clustering (K-means/VLAD).
Scalable Pipeline: A practical two-stage system that recovers 89–94% of full late-interaction accuracy with millisecond latency and significantly reduced storage.

4. Key Results

Model Performance:
- MarsDINO (ViT-B/16) achieved the best baseline performance (mAP = 0.553), outperforming the 7B-parameter generic DINOv3 model.
- Token Matching: Retaining just 64 attention-selected tokens via late interaction achieved mAP = 0.716 (DINO), surpassing the best single-vector baseline by +16.3 points.
Aggregation Efficacy:
- Instance-Token Aggregation significantly outperformed raw token selection and classical clustering.
- At $K=16$ , aggregation improved mAP by +17.9 points over raw attention selection.
- At $K=64$ , it matched the accuracy of using all 196 tokens while using 3× less storage.
- It outperformed per-image K-means at operational budgets ( $K \ge 32$ ) because it preserves seed identity rather than smoothing into a centroid.
Two-Stage System:
- With a shortlist size $S=100$ and $K=32$ tokens, the system recovered 89% of the exhaustive late-interaction accuracy.
- Latency: ~3.6 ms per query (including FAISS search and reranking).
- Robustness: The method is highly robust to INT8 quantization (4× compression) with negligible accuracy loss.

5. Significance and Impact

Scientific Workflow Enablement: This work shifts crater analysis from simple detection to robust retrieval, enabling automated catalog deduplication and cross-mission matching, which are essential for planetary geology.
GeoAI Generalization: The methodology (late interaction + deterministic token compression) is domain-agnostic. It offers a blueprint for scaling instance-level retrieval for Earth Observation tasks (e.g., change detection, scene deduplication) where large-scale embedding corpora exist but fine-tuning data is scarce.
Efficiency vs. Accuracy Trade-off: The paper proves that one does not need massive models or supervised fine-tuning to achieve high accuracy in specialized domains. Instead, engineering the matching strategy (preserving spatial tokens via aggregation) yields superior results.
Open Science: The benchmark and code are publicly available, setting a new standard for evaluating planetary retrieval systems.