VAD4Space: Visual Anomaly Detection for Planetary Surface Imagery

1. Problem Statement

Space exploration missions (e.g., Lunar Reconnaissance Orbiter, Mars Rovers) generate massive volumes of high-resolution imagery that far exceed human capacity for manual review. Current automated solutions rely heavily on supervised deep learning, which faces two critical limitations in the space domain:

• Data Scarcity: Rare geological phenomena (e.g., fresh craters, rockfalls, meteorites) are difficult to label due to the high cost of expert annotation and the sheer rarity of these events.
• Closed-World Assumption: Supervised models can only detect classes they were explicitly trained on. They fail to identify truly novel or unanticipated phenomena, which are often the primary scientific targets of exploration.

The paper proposes Visual Anomaly Detection (VAD) as an "open-world" alternative. VAD models are trained exclusively on "normal" data and identify deviations without requiring labeled examples of anomalies, making them ideal for discovering the unexpected.

2. Methodology

The authors conducted the first empirical evaluation of state-of-the-art feature-based VAD methods on real planetary data.

A. Benchmarks Introduced

Two new datasets were curated to reflect operational space scenarios:

1. Lunar Benchmark: Derived from Lunar Reconnaissance Orbiter Camera (LROC) Narrow Angle imagery.
   • Normal Class: Degraded terrain ("None").
   • Anomaly Classes: Fresh craters (with ejecta) and old/degraded craters.
   • Goal: Simulate orbital surveying for landing site selection and hazard detection.
2. Mars Benchmark: Derived from the Curiosity Rover's Mastcam (M-100) multispectral imagery.
   • Normal Class: Typical Martian geology (dust, rocks).
   • Anomaly Classes: 8 categories of geological novelty (e.g., meteorites, drill holes, veins, exposed bedrock).
   • Note: Although the source data is 6-band multispectral, the experiments used only the RGB channels to test compatibility with standard pretrained backbones.

B. Evaluation Protocols

The study introduced three settings to test robustness:

1. Balanced Test Set: Mirrors original dataset papers (roughly equal normal/anomaly).
2. 5% Positives: Standard VAD setting where anomalies are rare (5% of test set).
3. 5% Positives with Contamination: A realistic scenario where the training set contains 5% anomalies. This tests model robustness when a "perfectly clean" normal dataset cannot be guaranteed.

C. Models and Architecture

The study focused on feature-based VAD methods (avoiding computationally expensive reconstruction) suitable for edge deployment.

• Backbone: Replaced standard WideResNet50 with MobileNetV2 to reduce memory and compute requirements for onboard satellites/rovers.
• Evaluated Methods:
  • Memory Bank: PatchCore, PaDiM, CFA.
  • Teacher-Student: STFPM, RD4AD.
  • Synthetic Anomalies: SimpleNet.
  • Normalizing Flow: FastFlow.

3. Key Contributions

1. First Empirical Evaluation: Systematic benchmarking of seven feature-based VAD methods on real orbital (Lunar) and surface (Mars) planetary imagery.
2. New Benchmarks: Introduction of the Lunar and Mars datasets with standardized evaluation protocols for planetary anomaly detection.
3. Realistic Training Contamination: Introduction of a "contaminated training set" setting (Setting 3) to evaluate model robustness under imperfect data curation conditions, revealing significant performance differences between models.
4. Edge-Oriented Analysis: Demonstration that high-performance VAD is achievable on resource-constrained hardware using lightweight backbones (MobileNetV2), making onboard deployment feasible.

4. Results

Lunar Benchmark (Orbital)

• Performance: PaDiM achieved the best results, reaching 73% accuracy in the balanced setting, within 10% of a fully supervised VGG-11 baseline (82%) despite having zero anomaly labels during training.
• Robustness: In the Contamination Setting (Setting 3), PatchCore proved most resilient, showing minimal performance degradation. Conversely, STFPM and RD4AD suffered significant drops.
• Efficiency: PaDiM required only 10.7 MB of storage, compared to 507 MB for the supervised VGG-11 baseline.

Mars Benchmark (Surface)

• Performance: PatchCore outperformed all other models, achieving an IMG-ROC of 0.81 (vs. 0.705 for the previous best reconstruction-based baseline), despite using only 3 RGB channels instead of the original 6 spectral bands.
• Robustness: Unlike the Lunar results, PatchCore collapsed in the Mars Contamination Setting (dropping from 0.80 to 0.40 IMG-ROC). The authors attribute this to the "memory bank" mechanism: Martian anomalies are globally distinct, so their inclusion in training corrupts the memory bank. Other models (PaDiM, RD4AD) remained stable.
• Efficiency: VAD models were significantly lighter (10–17 MB) than previous baselines (~34 MB) and supported higher resolutions (256×256 vs. 64×64).

Qualitative Analysis

• Localization: VAD models successfully generated pixel-level anomaly maps without ground truth.
  • On the Moon, models accurately localized compact crater structures.
  • On Mars, localization was less precise due to anomalies (e.g., veins, drill holes) often spanning the entire image, but the maps still provided valuable guidance for prioritizing data.

5. Significance and Impact

• Operational Viability: The study proves that VAD is not just theoretically sound but practically viable for space missions. The low memory footprint (e.g., <20 MB) and fast inference times (18–41 ms) allow for onboard processing on satellites and rovers.
• Scientific Discovery: By operating in an "open-world" manner, VAD can detect unanticipated geological processes (e.g., new rockfalls or mineral veins) that supervised models would miss.
• Mission Efficiency: VAD enables bandwidth-aware data prioritization. Instead of downlinking terabytes of routine terrain data, spacecraft can automatically flag and transmit only the anomalous, scientifically valuable images, optimizing limited communication windows.
• Future Applications: The framework extends beyond surface imagery to other mission-critical tasks, such as monitoring solar panel degradation, rocket nozzle erosion, and propellant tank integrity.

In conclusion, VAD4Space establishes that feature-based anomaly detection is a robust, efficient, and essential tool for the next generation of autonomous planetary exploration.