Tri-path DINO: Feature Complementary Learning for Remote Sensing Multi-Class Change Detection

1. Problem Statement

Multi-Class Change Detection (MCD) in remote sensing aims to identify not only where changes have occurred but also what specific semantic categories those changes belong to (e.g., distinguishing between "building damage" and "new construction"). While crucial for applications like disaster assessment and urban monitoring, MCD faces significant challenges:

• Fine-Grained Complexity: Changed areas are often small, and the features distinguishing different change categories are subtle (e.g., varying degrees of structural damage).
• Data Scarcity: High-quality, detailed annotations for MCD are scarce compared to Binary Change Detection (BCD).
• Model Limitations: Traditional Siamese networks often lack the capacity to model long-range dependencies and capture both coarse semantic context and fine structural details simultaneously. Furthermore, while powerful pre-trained foundation models (like DINOv3) exist, they have not been effectively adapted to explicitly model the complementary features required for fine-grained MCD.

2. Methodology: Tri-path DINO

The authors propose Tri-path DINO, a novel architecture designed to adapt pre-trained foundation models for MCD through a three-path complementary feature learning strategy. The framework consists of three main components:

A. Backbone: Pre-trained DINOv3 (Main Path)

• Base: Utilizes the DINOv3 model as a shared Siamese backbone to extract robust, coarse-grained semantic representations from bi-temporal images ($T_1, T_2$).
• Adaptation: To avoid catastrophic forgetting and adapt to the remote sensing domain, LoRA (Low-Rank Adaptation) and Adapter modules are inserted into the frozen DINOv3 weights.
• Feature Processing: The output features ($S_1, S_2$) are processed by a lightweight CNN feature extractor. This path focuses on coarse-grained targets and global semantic context. The CNN processes the difference features and concatenated features to refine local details.

B. The "Third Path" (Auxiliary Path)

• Purpose: Designed to capture fine-grained features and model long-range dependencies that the CNN backbone might miss.
• Mechanism: It acts as a complementary Siamese encoder using a Transformer architecture.
• Input: It aggregates intermediate features from the C2, C3, and C4 layers of the DINO backbone. These features are summed to form Query ($Q$), Key ($K$), and Value ($V$) matrices.
• Output: A Multi-Head Self-Attention mechanism processes these inputs to generate fine-grained feature maps ($f_{Tr}$), focusing on structural details and subtle variations.

C. Enhanced Decoder: Multi-Level Hybrid Attention (MLHA)

• Integration: The features from the Main Path ($f_{CNN}$) and the Third Path ($f_{Tr}$) are concatenated ($f_{fuse}$).
• Architecture: The decoder employs a Multi-Level Hybrid Attention module with two stages:
  1. Initial Fusion: Uses parallel convolutions with different kernel sizes (3×3, 5×5, 7×7) to adaptively capture contextual information under varying receptive fields.
  2. Final Fusion: Integrates Channel Attention, Spatial Attention, and Local Context attention mechanisms to refine the features before generating the final pixel-level multi-class change map.

D. Loss Function

To address class imbalance and the scarcity of changed pixels, the model uses a composite loss function:
$$L_{total} = \alpha L_{focal} + \beta L_{Dice} + (1 - \alpha - \beta) L_{Lovasz}$$

• Focal Loss: Focuses on hard-to-classify samples.
• Dice Loss: Optimizes the overlap between prediction and ground truth.
• Lovász Loss: A differentiable surrogate for the Jaccard index to refine segmentation boundaries.

3. Key Contributions

1. Novel Architecture: Proposed Tri-path DINO, which explicitly models complementary coarse-grained (via DINOv3+CNN) and fine-grained (via Transformer) features to solve the MCD task.
2. Dataset Introduction: Introduced the Gaza-change dataset, a new benchmark for infrastructure damage assessment containing six specific semantic categories of change (e.g., building damage, new camps) in the Gaza region.
3. Empirical Validation: Demonstrated that MCD offers a practical balance between efficiency and performance compared to Binary Change Detection (BCD) and full Semantic Change Detection (SCD), particularly for time-sensitive disaster assessment.
4. Interpretability: Provided Grad-CAM visualizations confirming that the main path focuses on semantic context while the auxiliary path focuses on structural details, validating the synergistic design.

4. Experimental Results

The method was evaluated on two datasets: the newly created Gaza-change dataset and the classic SECOND dataset.

• Performance on Gaza-change:
  • Achieved 96.45% Overall Accuracy (OA) and 71.35% mIoU.
  • Outperformed the previous state-of-the-art (MC-DiSNet) by 0.12% in OA and 0.71% in mIoU.
  • Showed significant leads in SeK (13.02) and Fscd (62.58), metrics critical for handling imbalanced change classes.
• Performance on SECOND:
  • Achieved the highest scores in mIoU (71.62%), SeK (20.61), and Fscd (55.44).
  • Successfully detected fine details (e.g., gaps between buildings) that other methods missed.
• Ablation Studies: Confirmed that both the "Third Path" (Transformer) and the "MLHA" decoder module are essential, with the full model outperforming variants where either component was removed.
• Visualization: Grad-CAM heatmaps showed the main path activating on large semantic regions (e.g., entire buildings) while the third path highlighted local, previously unattended areas, proving the complementary nature of the design.

5. Significance

This work bridges the gap between powerful self-supervised foundation models and the specific, demanding requirements of remote sensing change detection. By introducing a complementary learning strategy, Tri-path DINO effectively handles the dual challenge of small target sizes and subtle inter-class differences. The proposed framework offers a robust, interpretable, and annotation-efficient solution for critical real-world applications, particularly post-disaster damage assessment, where rapid and accurate identification of specific damage types is vital for humanitarian response.