GeoSAM-Lite: A Lightweight Foundation Model for Onboard Remote Sensing Segmentation

Technical Summary of GeoSAM-Lite: A Lightweight Foundation Model for Onboard Remote Sensing Segmentation

Problem Statement
The deployment of large-scale foundation models, such as the Segment Anything Model (SAM), on resource-constrained Earth observation platforms (e.g., satellites and UAVs) is currently hindered by two primary challenges. First, the computational cost of heavyweight image encoders (e.g., ViT-H with over 600M parameters) is prohibitive for onboard processing. Second, existing lightweight adaptations of SAM suffer from a significant "domain shift" because they distill knowledge from teachers trained on natural images, lacking essential geospatial priors. Furthermore, aggressive parameter reduction in generic lightweight models often induces a "low-pass filtering" effect, leading to the attenuation of high-frequency details and blurred boundaries—a critical failure mode for fine-grained remote sensing tasks like cloud detection, which involve extreme scale variations and complex edges.

Methodology
The authors propose GeoSAM-Lite, a prompt-free, lightweight segmentation framework designed to reconcile segmentation fidelity with onboard efficiency. The architecture employs a decoupled two-stage paradigm:

1. Stage I: Geospatial-Domain Initialization (Geo-Init):
   Instead of distilling from a general-purpose vision model, GeoSAM-Lite utilizes a Domain-Expert Teacher (RSAM-Seg with a ViT-L backbone) pre-trained on large-scale remote sensing datasets. The student model (a lightweight ViT-S) undergoes a pre-training strategy involving:

   • Masked Image Modeling (MIM): The student learns to reconstruct holistic context from masked image patches (75% masking ratio), forcing feature abstraction for geospatial scenes.
   • Teacher-Guided Knowledge Distillation: The student aligns its partial-view representations with the full-view feature space of the frozen expert teacher, ensuring the transfer of domain-specific geospatial semantics.

2. Stage II: Task-Specific Fine-Tuning with Feature Fusion Layers (FFL):
   To compensate for the capacity bottlenecks of the lightweight backbone, the pre-trained encoder is augmented with two specialized modules before being fine-tuned on specific downstream tasks with a lightweight convolutional decoder:

   • FFL-S (Spatial Recalibration): Embedded within ViT blocks, this module uses a residual bottleneck structure to dynamically recalibrate features based on global significance, handling targets with wide spatial scales.
   • FFL-F (High-Frequency Injection): This module explicitly addresses the "low-pass filtering" effect. It converts input features to the frequency domain via Fast Fourier Transform (FFT), applies a high-pass filter to isolate high-frequency components (HFC) rich in boundary information, and reconstructs them via Inverse FFT. These cues are injected back into the encoder to restore crisp segmentation edges.

Key Contributions

1. GeoSAM-Lite Framework: A prompt-free lightweight framework that reduces parameters by 92.8% compared to the heavyweight teacher model (RSAM-Seg) while maintaining competitive segmentation accuracy.
2. Geo-Init Strategy: A domain-aware pre-training approach that distills remote sensing-specific knowledge from an expert teacher via masked feature alignment, effectively bridging the domain gap between natural and geospatial imagery.
3. Feature Fusion Layers (FFL): A dual-module design integrating spatial recalibration (FFL-S) and frequency-domain reconstruction (FFL-F) to explicitly compensate for spectral bias and boundary blurring inherent in compact networks.

Experimental Results
The framework was evaluated on representative remote sensing datasets, with a primary focus on cloud detection (38-Cloud, CloudSEN12, SPARCS) to test performance under extreme scale variations and complex boundaries, and a secondary evaluation on farmland segmentation (Sentinel-2) to assess generalization.

• Performance: On the CloudSEN12 dataset, GeoSAM-Lite improved the Jaccard index by approximately 15% over other lightweight baselines (EfficientSAM, MobileSAM). While it showed a slight performance drop compared to the heavyweight RSAM-Seg, it maintained high Recall and Precision, validating its practicality.
• Efficiency: Compared to RSAM-Seg (ViT-L), GeoSAM-Lite (ViT-S) achieved a 92.8% reduction in parameters and a 92.5% reduction in FLOPs. Hardware resource consumption was significantly lowered, with a 58.3% reduction in GPU memory usage and a 50% reduction in system RAM requirements.
• Ablation Studies: Removing the Geo-Init pre-training caused a significant drop in performance across key metrics, confirming the necessity of domain-specific initialization. Similarly, removing FFL-F led to degradation in boundary fidelity, highlighting its critical role in preserving high-frequency details.
• Generalization: The model demonstrated robust generalization to farmland scenarios without architectural modifications, effectively delineating agricultural parcel boundaries.

Significance
The paper claims that GeoSAM-Lite establishes a superior Pareto frontier between efficiency and fidelity, offering a practical solution for real-time segmentation on edge devices. By decoupling representation learning into domain-aware initialization and frequency-spatial adaptation, the model successfully inherits geospatial semantics while recovering lost high-frequency details. This approach addresses the dual challenges of domain shift and feature degradation, enabling the deployment of foundation model capabilities in resource-constrained onboard remote sensing environments where previous lightweight adaptations failed to preserve critical boundary details.