💻 computer science

STAR-IOD: Scale-decoupled Topology Alignment with Pseudo-label Refinement for Remote Sensing Incremental Object Detection

This paper proposes STAR-IOD, a novel framework for Remote Sensing Incremental Object Detection that addresses intra-class scale variations and missing annotations through a Subspace-decoupled Topology Distillation module and a Clustering-driven Pseudo-label Generator, achieving state-of-the-art performance on newly constructed DIOR-IOD and DOTA-IOD datasets.

Original authors: Yaoteng Zhang, Qing Zhou, Junyu Gao, Qi Wang

Published 2026-05-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yaoteng Zhang, Qing Zhou, Junyu Gao, Qi Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a student to recognize objects in aerial photos (like those taken by satellites or drones). In the real world, new types of objects appear over time. You teach the student to spot "ships" first, then later you teach them to spot "airplanes."

The problem with traditional AI students is that when they learn about airplanes, they often forget how to spot ships. This is called "catastrophic forgetting." It's like a student who, after studying for a history exam, suddenly forgets everything they learned in math class.

This paper, STAR-IOD, introduces a new teaching method designed specifically for remote sensing (aerial) images to stop this forgetting. It tackles two main problems that make aerial photos tricky:

1. The "Size" Problem: The Same Object, Different Scales

In a normal photo of a street, a car is usually the same size. But in an aerial photo, a ship can look tiny if it's far away, or huge if it's close up.

The Old Way: Previous methods tried to teach the student to see "ships" as one single, uniform concept. This was like trying to fit a tiny toy boat and a massive cruise liner into the same mental box. The result was confusion; the student couldn't distinguish between them well, and when learning new things, they got mixed up.
The STAR-IOD Solution (The "Sorting Hat"): The authors created a module called Subspace-decoupled Topology Distillation (STD). Imagine a teacher who doesn't just say "Here is a ship." Instead, they sort the ships into three separate bins: Small, Medium, and Large.
- The student learns the relationships between ships only within their own size bin.
- This prevents the "tiny ship" from confusing the "giant ship." By keeping these size groups separate, the student can remember the specific "shape" of a small ship and a large ship without them blurring together. This helps the student keep their old knowledge (ships) sharp while learning new things (planes).

2. The "Missing Label" Problem: The Silent Background

In these aerial photos, when you introduce a new category (like "planes"), the old photos you have don't have labels for the planes. But they do have ships.

The Old Way: Because the new photos don't have labels for the old ships, the AI assumes any ship it sees is just "background noise" (like clouds or water). It thinks, "Oh, that's just a blob, not a ship." This causes the student to actively unlearn how to spot ships.
The STAR-IOD Solution (The "Smart Detective"): The authors built a module called Clustering-driven Pseudo-label Generator (CPG).
- Imagine a detective who looks at the student's guesses. Instead of using a rigid rule like "Only trust guesses with 80% confidence," the detective looks at the pattern of all the guesses.
- The detective notices a split: some guesses are clearly "real objects" (high confidence) and some are clearly "noise" (low confidence).
- Using a mathematical trick called K-Means clustering, the detective automatically draws a line between the "real" and the "noise" for each specific type of object.
- This allows the system to say, "Ah, even though this wasn't labeled, the student is 90% sure it's a ship, and the pattern matches other real ships. Let's treat it as a ship and teach the student to keep that knowledge." This fixes the "missing label" issue without needing human annotators to go back and label old photos.

The Results

The researchers tested this new "teacher" (STAR-IOD) on two new datasets they created (DIOR-IOD and DOTA-IOD), which are like standardized exams for aerial object detection.

The Score: Their method beat the current best methods (State-of-the-Art) by 1.7% to 2.1% in overall accuracy.
The Win: Most importantly, it stopped the student from forgetting the old classes. While other methods forgot how to spot ships when learning about planes, STAR-IOD kept spotting ships accurately.

Summary

Think of STAR-IOD as a super-organized teacher who:

Sorts students by height (size) so they don't get confused by how big or small an object looks.
Acts as a smart detective to figure out which unlabeled objects are actually real targets, ensuring the student doesn't forget what they already know.

This approach ensures that as the AI learns new things, it doesn't lose its memory of the old things, making it much more reliable for real-world aerial monitoring.

Technical Summary: STAR-IOD

Problem Statement

Remote Sensing Incremental Object Detection (RS-IOD) addresses the challenge of training detectors on continuous data streams where new object categories are introduced sequentially. Unlike static detection paradigms, RS-IOD models face catastrophic forgetting, where learning new classes causes the model to degrade performance on previously learned categories.

The paper identifies two specific, critical challenges in RS-IOD that existing methods often overlook:

Intra-class Scale Variations: Remote sensing objects (e.g., ships, planes) exhibit extreme scale diversity due to varying sensor altitudes and imaging angles. Existing methods often force these scale-variant features into a unified representation, which blurs inter-class boundaries and disrupts the topological structure of knowledge, hindering effective knowledge transfer.
Missing Annotations and Noisy Pseudo-labels: In incremental settings, instances of old classes in new data lack annotations and are treated as background. While pseudo-labeling is used to recover these instances, remote sensing scenarios (characterized by complex backgrounds, dense objects, and occlusions) generate noisy pseudo-labels. Fixed confidence thresholds fail to distinguish reliable targets from noise, introducing erroneous supervisory signals that accelerate forgetting.

Additionally, the field lacks standardized datasets and evaluation protocols for RS-IOD, preventing fair comparisons.

Methodology: STAR-IOD

The authors propose STAR-IOD (Scale-decoupled Topology Alignment with Pseudo-label Refinement), a framework built upon the Grounding DINO architecture. It integrates two core modules to address the identified challenges:

1. Subspace-decoupled Topology Distillation (STD)

To mitigate the negative impact of scale variations, STD enforces structured knowledge alignment by decoupling features into scale-specific subspaces.

Scale-adaptive Instance Partitioning (SIP): Predicted bounding boxes are partitioned into three disjoint subspaces (small, medium, large) based on their area thresholds ( $\tau_s, \tau_m$ ).
Context-injected Prototype Aggregation: For each scale subspace, class prototypes are constructed using confidence-weighted aggregation of pseudo-labeled old-class instances. A background anchor node is also injected to model object-background dependencies.
Topological Relation Alignment: Within each scale subspace, the method constructs inter-class topological matrices representing the semantic distances between class prototypes. Instead of rigid geometric alignment, it models these relationships as probabilistic distributions (using a Boltzmann-like distribution) and minimizes the Kullback-Leibler (KL) divergence between the teacher's and student's affinity distributions. This ensures the student preserves the relational reasoning logic of the teacher within specific scale contexts.

2. Clustering-driven Pseudo-label Generator (CPG)

To address missing annotations and label noise, CPG replaces static confidence thresholds with a data-driven, adaptive mechanism.

Dynamic Score Maintenance: A class-specific score bank maintains prediction scores from the teacher model to ensure statistical stability for clustering.
Clustering-guided Threshold Estimation: The module treats threshold estimation as a 1D clustering problem. It applies the K-Means algorithm to the confidence scores of each class to identify a bimodal distribution (low-confidence noise vs. high-confidence targets).
Adaptive Thresholding: The threshold for each class is dynamically set as the lower bound of the high-confidence cluster. This allows the system to filter out noisy pseudo-labels and retain reliable supervision signals tailored to the specific confidence distribution of each class and incremental stage.
Box De-duplication: A spatial filtering step removes pseudo-labels that significantly overlap with ground-truth annotations to prevent duplicate supervision.

Loss Function

The total optimization objective combines the standard detection loss ( $\mathcal{L}_{detr}$ ), the subspace-decoupled topology distillation loss ( $\mathcal{L}_{STD}$ ), and a correspondence response distillation loss ( $\mathcal{L}_{CRD}$ ) that aligns classification and regression responses between teacher and student.

Key Contributions

Novel Framework (STAR-IOD): A unified framework that explicitly addresses intra-class scale variations through subspace-decoupled topology distillation and mitigates pseudo-label noise via clustering-driven threshold estimation.
Subspace-Decoupled Topology Distillation: A method that aligns inter-class topological structures within explicitly decoupled scale subspaces, preventing the distortion of relational knowledge caused by scale shifts.
Clustering-driven Pseudo-label Generator: An adaptive module that dynamically determines class-specific confidence thresholds using K-Means clustering, effectively reducing bias from noisy supervision in remote sensing scenarios.
Benchmarks and Protocols: The construction of two standardized RS-IOD datasets, DIOR-IOD and DOTA-IOD, along with standardized incremental splitting strategies and evaluation pipelines to facilitate reproducible research.

Experimental Results

The authors evaluated STAR-IOD on the newly constructed DIOR-IOD and DOTA-IOD datasets, comparing it against state-of-the-art (SOTA) methods including CL-DETR, MD-DETR, ERD, and GCD.

Performance Gains: STAR-IOD outperformed the second-best method (GCD) by 1.7% mAP on DIOR-IOD and 2.1% mAP on DOTA-IOD.
Forgetting Mitigation: The method demonstrated superior retention of old classes. On DIOR-IOD, it achieved a 40.7% mAP on old classes (Task 2), a significant improvement over competitors. On DOTA-IOD, it maintained robust performance across multi-step incremental settings where other methods suffered severe catastrophic forgetting (e.g., dropping to <1% mAP on old classes).
Scale Robustness: The framework showed balanced improvements across small, medium, and large object scales, validating the effectiveness of the scale-decoupling mechanism.
Ablation Studies: Experiments confirmed that removing either the STD or CPG modules led to significant performance degradation. Specifically, the scale-adaptive partitioning improved overall mAP by 2.1%, and the adaptive thresholding outperformed fixed thresholds and other dynamic baselines (e.g., Consistent Teacher, LPL).

Significance and Claims

The paper claims that STAR-IOD provides a robust solution to the specific challenges of remote sensing incremental detection, where traditional methods fail due to scale diversity and annotation scarcity. By decoupling scale variations and refining pseudo-labels through clustering, the framework achieves a better balance between stability (retaining old knowledge) and plasticity (learning new classes).

The authors emphasize that the release of DIOR-IOD and DOTA-IOD establishes a necessary foundation for fair comparison and future investigation in this field. While the method significantly alleviates catastrophic forgetting, the authors modestly acknowledge a limitation: the framework struggles with semantically similar categories (e.g., Helicopters vs. Planes) where text embeddings in the underlying Grounding DINO architecture are highly proximate, leading to confusion. They suggest that future work could focus on enhancing text encoder discriminability to address this specific failure case.