Technical Summary: JL1-CC&QA

Problem Statement

Remote sensing change detection (CD) has traditionally focused on binary change detection (BCD), which produces pixel-level masks indicating where changes occurred but fails to explain what changed or why. While Semantic Change Detection (SCD) and Building Damage Assessment (BDA) have introduced categorical labels and damage scales, these remain encoded as discrete numerical indices within closed taxonomies, lacking the richness of natural language. Furthermore, existing benchmarks rarely unify binary masks, natural language change descriptions (captions), and interactive question-answering (QA) pairs on the same image set. This gap limits the development and evaluation of multi-task learning systems and vision-language models (VLMs) capable of deep, language-grounded change understanding.

Methodology

The paper introduces JL1-CC&QA, a multi-task benchmark extending the existing JL1-CD dataset. The construction relies on a three-stage annotation pipeline designed to balance scalability with factual reliability:

1. Source Data: The foundation is JL1-CD, comprising 5,000 bi-temporal image pairs (0.5–0.75 m GSD) from the Jilin-1 satellite, covering diverse Chinese provinces and change types (anthropogenic and natural).
2. Annotation Pipeline:
   • Stage 1 (Generation): A multi-modal Large Language Model (LLM), specifically Kimi-K2.6, generates candidate annotations.
     • For Change Captioning (CC), the model receives the pre-event image, post-event image, binary change mask, and spatial metadata (e.g., change area ratio, region description) to generate five diverse captions per pair.
     • For Change Question Answering (QA), the generation is context-enriched by incorporating the selected captions and metadata from the CC stage. The model generates five QA pairs per image, covering eight distinct question types (existence, description, location, magnitude, temporal comparison, causation, relative comparison, and visual detail).
   • Stage 2 (Vision-Grounded Judging): A second LLM call acts as a judge, evaluating generated text against the original image pair and mask.
     • Captions are scored (1–10) on accuracy, specificity, spatial correctness, naturalness, and informativeness.
     • QA pairs are scored on answer accuracy, question quality, completeness, and redundancy.
     • Items scoring below 7 are rejected; top-scoring candidates are retained.
   • Stage 3 (Human Verification): Domain experts review a subset of the retained data to verify factual accuracy and identify systematic errors, ensuring the automated pipeline produces reliable annotations at scale.

Key Contributions

The paper presents three primary contributions:

1. JL1-CC&QA Benchmark: A unified resource providing binary change masks, 17,021 quality-verified change captions (JL1-CC), and 20,060 question-answer pairs (JL1-QA) across 5,000 image pairs. This enables joint training and evaluation across CD, CC, and QA tasks.
2. Scalable Annotation Pipeline: A reproducible framework coupling multi-modal LLM generation with vision-grounded LLM judging and human verification. This approach effectively mitigates hallucination (e.g., by prohibiting exact numerical percentages in QA answers unless visually derivable) while maintaining high throughput.
3. Comprehensive Statistics and Release: The authors provide detailed dataset statistics, including vocabulary sizes, word lengths, and score distributions, and release all data and code to support community research.

Results and Dataset Statistics

• JL1-CC: From 25,000 generated captions, 17,021 were retained (68.1% pass rate). The selected captions have a mean length of 26.2 words and a vocabulary of 7,458 unique tokens. The judge score distribution shows clear discrimination, with 39.8% of captions scoring 9–10.
• JL1-QA: From 24,995 generated QA pairs, 20,060 were retained (80.3% pass rate). The higher pass rate compared to CC is attributed to the context enrichment from change captions, which reduced hallucinations. The dataset covers eight question types, with YES/NO (21.2%), WHERE (18.3%), and WHAT (16.9%) being the most frequent.
• Data Characteristics: The dataset exhibits a long-tailed distribution of Change Area Ratio (CAR), with a mean of 9.7% and a median of 2.9%, mirroring real-world imbalance. It covers diverse land-cover transformations including buildings, roads, vegetation, and water bodies.

Significance

The authors position JL1-CC&QA as a valuable resource to advance multi-task change understanding in remote sensing. By unifying binary masks, captions, and QA pairs on the same image set, the benchmark addresses the limitation of current datasets that remain silent on the semantic content of changes. The paper argues that this language-grounded approach is essential for training and evaluating increasingly capable architectures, including foundation models, agents, and interactive systems that require reasoning about surface changes in natural language. The dataset is intended to facilitate the transition from purely pixel-level detection to comprehensive, interactive change interpretation.