💻 computer science

Lightweight SAR Ship Detection via Contrastive Distillation

This paper introduces SURGE, the first transformer-based knowledge distillation framework for SAR ship detection that leverages contrastive relational geometry to significantly boost the performance of lightweight detectors, enabling them to surpass their teacher models on SSDD and HRSID benchmarks.

Original authors: Surendar Devasundaram, Saber Latibari Banafsheh, Abhijit Mahalanobis

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Surendar Devasundaram, Saber Latibari Banafsheh, Abhijit Mahalanobis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Heavy Expert" vs. The "Lightweight Apprentice"

Imagine you are trying to spot ships in a foggy ocean using radar (SAR). Radar is great because it sees through clouds and darkness, but the images it produces are very "noisy" and grainy, like a static-filled TV screen.

To find ships in this mess, you need a very smart computer brain (a deep learning model).

The Problem: The smartest brains (called "Teachers") are huge, heavy, and slow. They are like a giant supercomputer that needs a massive power plant to run. You can't put them on a small drone or a satellite that needs to work in real-time.
The Alternative: You can use a smaller, lighter brain (a "Student"). It's fast and efficient, but it's not very smart. It often misses ships or confuses waves for ships because it can't see the complex patterns in the noisy radar images.
The Goal: We want the small, fast brain to be as smart as the big, slow one, without actually making it heavy.

The Old Way: "Copy the Answer Sheet"

Previously, researchers tried to teach the small brain by showing it the big brain's "answer sheet."

They would say, "Look at this specific pixel; the big brain says 'ship' here, so you say 'ship' too."
The Flaw: This is like a student memorizing the final answers on a test without understanding why they are right. The small brain learns to copy local details (like a specific pixel's brightness) but fails to understand the relationships between objects. It doesn't learn that "ships usually have a certain shape relative to the water" or "these two blobs are likely a ship and its wake."

The New Solution: SURGE (The "Geometry Teacher")

The authors propose a new framework called SURGE. Instead of just copying answers, SURGE teaches the small brain to understand the geometry and relationships of the scene, just like the big brain does.

Think of it like this:

The Old Way: The teacher points to a dot and says, "This is a ship."
The SURGE Way: The teacher says, "Look at this group of dots. Notice how they are arranged? They form a long line with a specific curve. That pattern is what makes a ship. Don't just look at the dots; look at how they relate to each other."

How SURGE Works (The "Shared Map" Analogy)

The Teacher's Map: The big, heavy teacher looks at the radar image and finds potential ships. It creates a list of "candidate regions" (boxes around things that look like ships).
The Student's Map: The small student looks at the same image and tries to find ships too.
The Alignment: SURGE takes the teacher's list and the student's list and lines them up perfectly, even if the two computers see the image slightly differently.
The "Contrastive" Lesson: This is the secret sauce.
- Imagine the teacher and student are both holding a map of the ocean.
- The teacher points to a ship and says, "This is a ship."
- SURGE asks the student: "Is this other thing you found similar to the ship I found, or is it different?"
- The student learns to push "non-ships" far away from "ships" in its mental map and pull "ships" closer together, preserving the shape of the neighborhood.
- It's not about matching exact pixels; it's about learning the topology (the layout) of the objects.

Why This is Special

It Works for Everyone: Whether the computer brain is a "Two-Stage" detector (like a careful inspector who first finds candidates, then checks them), a "One-Stage" detector (a quick scanner), or a "Transformer" (a modern AI that looks at the whole picture at once), SURGE speaks their language. It doesn't require changing the internal wiring of these different computers.
First of Its Kind: This is the first time this specific "relationship-learning" method has been used for radar ship detection, especially with the modern Transformer models.

The Results: Small Brain, Big Brain Power

The researchers tested this on two famous radar datasets (SSDD and HRSID).

The Winner: The method worked best for the "Two-Stage" detectors.
The Magic: The small student model (using a tiny ResNet-18 brain) became so smart that it actually beat the performance of the giant teacher model (using a massive ResNet-101 brain).
The Trade-off: The student model used 50% fewer parameters (half the size) but achieved higher accuracy.
Speed: Because the "teaching" only happens while the computer is learning (training), the final small model runs just as fast as a normal small model. It doesn't slow down the real-time detection.

Summary

The paper introduces SURGE, a teaching method that helps small, fast AI models learn to spot ships in radar images by understanding how objects relate to each other, rather than just copying pixel-by-pixel answers. This allows small models to perform as well as (or better than) giant, slow models, making real-time ship detection on drones and satellites much more feasible.

Technical Summary: Lightweight SAR Ship Detection via Contrastive Distillation (SURGE)

Problem Statement

Deep convolutional and transformer-based detectors (e.g., Faster R-CNN, RetinaNet, DETR) achieve high accuracy in Synthetic Aperture Radar (SAR) ship detection but are often computationally prohibitive for real-time or onboard deployment due to memory and latency constraints. While lightweight models offer improved efficiency, they frequently struggle to capture the complex structural relationships and geometric cues inherent in SAR backscatter, leading to a loss of discriminative power against clutter.

Existing knowledge distillation (KD) approaches for SAR primarily rely on feature-map matching or logit alignment. These methods enforce localized activation similarity but fail to explicitly model the relational geometry that governs how a teacher detector organizes object representations in feature space. Consequently, student models learn local responses rather than the structural reasoning critical for distinguishing targets in SAR imagery. Furthermore, there is a lack of unified frameworks that can handle heterogeneous detector architectures (two-stage, one-stage, and transformer-based) without modifying their core designs.

Methodology: SURGE Framework

The authors propose SURGE (Structured Unified Relational knowledGE distillation), a framework designed to transfer relational geometry from a powerful teacher to a compact student using a contrastive InfoNCE objective within a shared projection embedding space.

Core Mechanisms

Architecture-Agnostic Region Generation:
The framework converts predictions from heterogeneous detector families into aligned candidate regions:
- Two-stage detectors: Regions are derived from the Region Proposal Network (RPN).
- One-stage detectors: High-confidence dense predictions are decoded into bounding boxes and treated as pseudo-proposals.
- Transformers (DETR): High-confidence bounding boxes predicted by the decoder are used.
  These regions are spatially aligned with the student's image space to account for preprocessing differences.
Cross-Detector Feature Alignment:
Teacher-derived regions are mapped back to the ground-truth coordinate system and transformed into the student's image space. Region-level features are extracted via RoIAlign from both the teacher and student networks.
Relation-Aware Distillation Objective:
Features are globally pooled and projected into a shared embedding space ( $z^T_i$ and $z^S_i$ ) and L2-normalized. The distillation is formulated as a student-anchored InfoNCE (Information Noise-Contrastive Estimation) objective:
- Positive Pairs: Defined by regions sharing the same semantic label and high IoU ( $\ge \tau_{pos}$ ) with the ground truth.
- Negative Pairs: Defined by regions with low IoU ( $\le \tau_{neg}$ ).
- Loss Function: The objective encourages the student embedding to preserve the relative neighborhood structure induced by the teacher's semantic and localization cues, rather than performing direct one-to-one feature regression.
Unified Training Strategy:
- For convolutional detectors (Faster R-CNN, RetinaNet), the total loss combines the standard detection loss, vanilla output-level distillation (logits/boxes), and the proposed relational loss.
- For transformer-based detectors (DETR), output-level distillation is omitted because unordered set-based predictions lead to unstable teacher-student correspondence. Instead, DETR relies exclusively on the region-level relational distillation.

Key Contributions

The paper outlines four primary contributions:

SURGE Framework: A unified, relation-aware KD framework for SAR ship detection. Its novelty lies in converting heterogeneous predictions into aligned candidate regions to transfer relational geometry among object-level embeddings, rather than introducing a new general-purpose loss function.
Unified Interface: A region-level distillation interface applicable to two-stage, one-stage, and transformer-based detectors without modifying their core architectures.
First Transformer-Based SAR KD: To the best of the authors' knowledge, this is the first framework to demonstrate effective relational transfer for transformer-based SAR ship detectors without relying on query-level supervision.
Performance Gains: Experiments on SSDD and HRSID benchmarks show substantial improvements for lightweight two-stage detectors (up to 6.2 mAP and 8.0 AP75 gains) while achieving over 50% parameter reduction.

Experimental Results

The framework was evaluated on the SSDD and HRSID benchmarks using Faster R-CNN (two-stage), RetinaNet (one-stage), and DETR (transformer).

Two-Stage Detectors (Faster R-CNN): The method yielded the most significant improvements. On SSDD, the ResNet-18 student improved from 62.87 mAP to 68.03 mAP (+5.16 mAP over the non-distilled student). Notably, the distilled student surpassed the ResNet-101 teacher by +1.33 mAP while using ~51% fewer parameters. On HRSID, gains were +6.03 mAP and +7.71 AP75.
One-Stage Detectors (RetinaNet): Results were mixed. On SSDD, the proposed method showed performance within the margin of statistical variability compared to the baseline student and vanilla KD. However, on the more complex HRSID dataset, it provided consistent gains (+0.65 mAP, +1.10 AP75).
Transformer Detectors (DETR): The method provided consistent, albeit smaller, gains (+0.33 mAP on SSDD). The authors note that a conventional output-level distillation baseline was not reported for DETR as it degraded performance due to unstable correspondences.
Ablation Studies: Combining relational distillation with vanilla KD yielded the best results, indicating that relational supervision complements rather than replaces conventional distillation.
Efficiency: Since the framework is training-only, deployment latency and memory usage remain identical to the baseline student models.

Significance and Claims

The paper claims that SURGE establishes relation-aware distillation as a principled and scalable strategy for efficient SAR ship detection under resource constraints.

Structural Reasoning: The authors emphasize that the method's success stems from transferring the "structural organization" of the teacher's representation space, which is critical for SAR imagery where contextual and geometric cues are vital.
Architecture Agnosticism: The framework successfully bridges the gap between different detector paradigms (two-stage, one-stage, transformer) using a unified region-level interface.
Teacher-Surpassing Performance: In the case of Faster R-CNN, the distilled student not only outperformed the non-distilled student but also exceeded the performance of the larger teacher model, demonstrating that relational geometry transfer can compensate for reduced model capacity.
Modesty on Scope: The authors acknowledge that gains for one-stage and transformer-based detectors are more modest compared to two-stage detectors, attributing this to the absence of explicit region proposals in those architectures. They position the work as a step toward efficient detection, with future work planned for multi-class detection and cross-modal distillation.