MoonAnything: A Vision Benchmark with Large-Scale Lunar Supervised Data

1. Problem Statement

Accurate perception of lunar surfaces is critical for modern exploration missions (e.g., NASA's Artemis, China's Chang'E) for tasks such as Terrain Relative Navigation (TRN), Hazard Detection and Avoidance (HDA), and 3D surface reconstruction. However, developing robust learning-based perception systems is hindered by the lack of comprehensive datasets that simultaneously provide:

• Geometric Ground Truth: Dense depth maps and camera poses for 3D reconstruction.
• Photometric Realism: Accurate surface reflectance modeling (BRDF) to handle the Moon's unique optical properties (e.g., opposition surge, anisotropic scattering).
• Illumination Diversity: Data covering diverse lighting conditions, from harsh polar shadows to varying solar angles.
• Scale: Large-scale coverage to prevent overfitting and ensure generalization.

Existing datasets fail to meet these criteria:

• Real Mission Data (LRO, Chang'E): Provide authentic appearance but lack dense geometric ground truth and calibrated poses.
• Synthetic Datasets: Often lack real lunar topography, use simplified lighting/reflectance models, or lack multi-illumination variation.
• Laboratory Mockups: Offer precise ground truth but are limited in scale and diversity.

2. Methodology: The MoonAnything Pipeline

The authors introduce MoonAnything, a unified benchmark built on real lunar topography using physically-based rendering (PBR). The pipeline integrates real Digital Elevation Models (DEMs) with advanced rendering tools to generate two complementary sub-datasets.

A. Common Infrastructure

• Terrain Sources: High-resolution DEMs from NASA's LRO mission:
  • South Pole: 5 m/px resolution (LOLA data), featuring extreme low-angle lighting conditions.
  • Tycho Crater: 1 m/px resolution, featuring complex geomorphology (central peak, terraced walls).
• Rendering Framework: Built on SurRender, a physically-based renderer.
• Reflectance Modeling: Uses the Hapke BRDF (physically grounded for airless bodies). For Tycho, it employs a learned Spatially-Varying BRDF (SVBRDF) to capture local photometric variations, improving realism over constant-parameter models.

B. Sub-dataset 1: LunarGeo (Geometric Perception)

• Goal: 3D reconstruction and pose estimation.
• Content: Stereo image pairs with dense depth maps.
• Configuration:
  • Regions: South Pole and Tycho Crater.
  • Trajectories: Three types simulating descent phases: Nadir (vertical), Oblique (tilted), and Dynamic (varying altitude/roll).
  • Resolution: 512×512 px.
  • Ground Truth: Includes intrinsics, extrinsics (6-DOF poses), dense depth maps, and stereo geometry.
• Scale: ~58,000 stereo pairs (38k South Pole, 20k Tycho).

C. Sub-dataset 2: LunarPhoto (Photometric Perception)

• Goal: Appearance modeling, reflectance estimation, and illumination-robust perception.
• Content: Pairs of geometry and reflectance data with multi-illumination renderings.
• Configuration:
  • Region: Tycho Crater only.
  • Base Data: 128×128 px DEM crops aligned with orthorectified real LRO NAC images.
  • Multi-Lighting: For each sample, 9 distinct solar configurations are simulated using the SPICE toolkit to generate varied shadow patterns and shading.
  • Ground Truth: Includes DEM, real ortho-images, depth maps, surface normals, per-pixel SVBRDF parameters, and illumination metadata.
• Scale: 84,000 samples, generating ~750,000 rendered images across different BRDF models.

3. Key Contributions

1. First Unified Benchmark: MoonAnything is the first dataset to provide comprehensive geometric and photometric supervision under systematic illumination variation at a large scale (130K+ samples).
2. Realistic Physics: It combines real lunar topography with physically-based rendering (Hapke and SVBRDF), addressing the "domain gap" where Earth-trained models fail on lunar imagery.
3. Dual-Task Capability:
   • LunarGeo enables training for stereo matching and 3D reconstruction.
   • LunarPhoto enables training for reflectance estimation, photometric stereo, and intrinsic image decomposition.
4. Open Source: The authors release the complete dataset, generation tools, and baselines to support community extension.

4. Experimental Results

The authors established baselines using State-of-the-Art (SOTA) 3D reconstruction models (MASt3R and VGGT) to validate the dataset's utility.

• Fine-Tuning Effectiveness: Models pre-trained on Earth data performed poorly on lunar data. However, fine-tuning on a limited subset of MoonAnything (South Pole) led to substantial improvements.
  • Example: On the South Pole "Seen" split, VGGT fine-tuned (FT) reduced Chamfer distance from ~226m to ~43m.
• Cross-Region Generalization: Models fine-tuned on the South Pole data successfully generalized to the unseen Tycho Crater region, despite significant differences in terrain morphology.
  • Example: MASt3R FT achieved the lowest errors on Tycho under oblique and dynamic trajectories, proving the dataset's ability to teach transferable, geometry-aware representations.
• Robustness: The dataset successfully challenged models with extreme shadowing and low-texture conditions, demonstrating its value for training robust navigation algorithms.

5. Significance and Impact

• Mission Critical: MoonAnything directly supports the development of autonomous systems for lunar landing and rover navigation, which are essential for the Artemis program and future commercial ventures.
• Beyond the Moon: The dataset serves as a challenging testbed for computer vision algorithms in low-textured, high-contrast environments. The methodologies and data can generalize to other airless celestial bodies (e.g., asteroids, Mercury).
• Research Standard: By providing the first large-scale, physically accurate, and multi-modal lunar benchmark, MoonAnything sets a new standard for data-driven planetary perception research, moving the field away from reliance on sparse real-world data or unrealistic synthetic simulations.