MOMO: Mars Orbital Model Foundation Model for Mars Orbital Applications

1. Problem Statement

While Foundation Models (FMs) have revolutionized Earth Observation (EO) by leveraging large-scale data for diverse downstream tasks, no dedicated foundation model exists for Mars remote sensing. Current approaches for Mars data rely on:

• ImageNet pre-training: Models trained on natural images lack the specific spectral, textural, and geometric features of Martian surfaces.
• Sensor-specific training: Training separate models for individual sensors (e.g., HiRISE, CTX, THEMIS) is inefficient and fails to leverage cross-sensor knowledge.
• Data Combination limitations: Directly merging heterogeneous data from different sensors (varying resolutions from 0.25m to 100m/pixel) into a single pre-training dataset is challenging due to coverage disparities and the need for complete re-training when new sensors are added.

The core challenge is building a unified model that effectively handles multi-sensor and multi-resolution data from Mars without requiring massive, monolithic re-training efforts.

2. Methodology

The authors propose MOMO, a multi-sensor foundation model constructed via a model merging framework rather than direct data concatenation. The methodology consists of three key pillars:

A. Independent Sensor Pre-training

Instead of combining raw data, MOMO pre-trains independent Masked Autoencoder (MAE) models for each of the three key Martian sensors:

1. HiRISE: 0.25 m/pixel (Optical).
2. CTX: 5 m/pixel (Optical).
3. THEMIS: 100 m/pixel (Infrared).

Each model learns the unique distribution and feature characteristics of its specific sensor independently.

B. Novel Cost Function

To improve reconstruction quality beyond simple pixel matching, the authors introduce a composite loss function during pre-training:
$$L_{total} = \lambda_1 L_{MSE} + \lambda_2 L_{SSIM} + \lambda_3 L_{LPIPS} + \lambda_4 L_{grad}$$

• $L_{MSE}$: Standard pixel-level reconstruction.
• $L_{SSIM}$ & $L_{LPIPS}$: Perceptual and structural similarity losses to preserve texture and high-level features.
• $L_{grad}$: A gradient penalty ($\ell_1$ norm) to enforce spatial smoothness and structural consistency (preserving crater rims, landslide boundaries, etc.).

C. Equal Validation Loss (EVL) Strategy

The core innovation is the EVL strategy for merging the independently trained models.

• Problem: Merging models at their final checkpoints often leads to instability because models trained on different data distributions converge at different rates and may be overfitted or underfitted relative to each other.
• Solution: The authors align checkpoints based on validation loss similarity.
  1. They identify "loss-aligned" tuples of epochs $(e_1, e_2, ..., e_n)$ where the validation losses across all sensors are mutually close ($\Delta_{ij} \leq \epsilon$).
  2. From these aligned tuples, they select the one closest to the early-stopping epoch of each individual sensor to ensure optimal generalization.
  3. The selected checkpoints are merged using Task Arithmetic (specifically, weight addition), creating a unified model that integrates representations from all sensors at compatible convergence stages.

3. Key Contributions

1. First Mars Foundation Model: MOMO is the first systematically developed, analyzed, and evaluated foundation model specifically for Mars orbital applications.
2. Multi-Sensor/Multi-Resolution Handling: It successfully integrates data spanning a 400x resolution gap (0.25m to 100m) without requiring complex heterogeneous data stacking.
3. EVL Checkpoint Selection: Introduces a novel, systematic strategy for model merging that ensures stability and generalization by aligning models based on validation loss rather than arbitrary epoch counts.
4. Scalability: The framework allows for the easy integration of new sensors in the future by pre-training a new model and merging it via EVL, avoiding the need for full re-training of the entire system.

4. Experimental Results

MOMO was evaluated on 9 downstream tasks from Mars-Bench, covering both classification and segmentation.

• Dataset: Pre-trained on a curated corpus of ~12 million samples (4M per sensor) derived from HiRISE, CTX, and THEMIS.
• Baselines: Compared against ImageNet pre-training, Earth Observation FMs (SatMAE, CROMA, Prithvi, TerraFM, DINOv3), sensor-specific pre-training, and joint data pre-training (Data Merge).
• Performance:
  • Overall: MOMO outperformed all baselines, achieving the best average rank (1.67) across all tasks.
  • Segmentation: MOMO showed significant improvements, particularly on the Boulder task (~4% mIoU improvement over ImageNet) and ConeQuest. It demonstrated superior ability to localize fine-grained features compared to Earth-based models.
  • Classification: MOMO achieved comparable or slightly better performance than Earth FMs and ImageNet, with notable gains in the AtmosDust task.
  • Checkpoint Strategy: The EVL strategy outperformed "Last Epoch" and "Early Stopping" merging strategies, providing an average ~2.5% mIoU improvement on segmentation tasks.
  • Comparison to Data Merge (DM): MOMO outperformed the joint data pre-training approach by 15.25% in classification and 5.4% in segmentation, while being more computationally efficient and modular.

5. Significance and Impact

• Planetary Science Advancement: MOMO provides planetary scientists with a powerful tool to train models for custom science tasks at a lower cost than fully supervised methods, enabling better analysis of Martian geology, climate, and surface changes.
• Generalization: The model demonstrates strong generalization to out-of-distribution regions, successfully generating global maps of features (e.g., volcanic cones) in unseen areas of Mars.
• Real-World Application: MOMO outperformed the model currently deployed in NASA's Planetary Data System (PDS) for content-based image search, improving macro-average F1-scores by 4% and significantly enhancing the detection of complex features like slope streaks and impact ejecta.
• Methodological Shift: The paper establishes that model merging with optimal checkpoint selection is a superior and more scalable approach for building foundation models for multi-resolution, multi-sensor remote sensing data compared to traditional monolithic pre-training.

Resources: The authors have open-sourced the model weights, pre-training code, data, and evaluation code at github.com/kerner-lab/MOMO.