⚡ electrical engineering

Training Data Selection for Weakly-Supervised Lunar Traversability Estimation: A Cross-Domain Study

This paper demonstrates that for weakly-supervised lunar traversability estimation, training data selection should prioritize visual and morphological similarity to the target lunar environment over dataset realism, as lunar analog facilities consistently outperform other Earth-based sources in cross-domain generalization.

Original authors: Bartosz Ptak

Published 2026-10-26

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Bartosz Ptak

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot rover standing on the surface of the Moon, surrounded by a landscape of gray dust, jagged rocks, and deep shadows. To move safely, the machine must constantly answer a simple but critical question: can I drive over this patch of ground, or will it get stuck? For humans, this is an instinctive judgment based on sight. For a robot, it is a complex calculation that usually requires a computer to be trained on thousands of examples of what "safe" and "dangerous" look like. The problem is that we do not have enough pictures of the Moon to teach a robot everything it needs to know. We have plenty of photos from Earth, but the Moon looks different; the light is harsher, the dust behaves differently, and the rocks have a unique texture. If a robot learns to drive only on Earth, it might fail when it arrives on the Moon.

Scientists have been trying to solve this by teaching robots to learn from just a few examples rather than millions. Instead of labeling every single pixel in a photo as "safe" or "unsafe," researchers now use a method where humans simply point to two spots on a picture and say which one looks easier to drive over. This is a much faster way to teach a computer, but it raises a new question: if we train a robot on Earth data using this quick method, will it actually work when transferred to the Moon? A new study from the Poznań University of Technology in Poland investigates exactly this. The researchers tested seven different sources of training data, ranging from computer simulations and volcanic landscapes on Earth to indoor facilities that mimic the Moon. They wanted to find out which training environment produces a robot that can truly understand the lunar surface, and whether the usual way of testing these robots gives a false sense of security.

The team set up a large-scale experiment to see how well a robot trained on one type of terrain could handle another. They gathered images from seven distinct sources: two sets of real photos taken by Chinese rovers on the Moon, two indoor facilities built to look like the Moon, two datasets from volcanic mountains in Italy that serve as Earth-based stand-ins for the Moon, and two different computer-generated simulations of lunar landscapes. Using a weakly supervised learning approach, they trained a model on each of these seven sources. In this method, the model learns by comparing pairs of points on an image, learning that one spot is better to drive on than another, without needing a perfect, detailed map of the entire scene. Once trained, they tested each model on the real lunar photos from the Chang'e-3 and Chang'e-4 missions to see how often the robot's judgment disagreed with what a human would have decided.

The results revealed a surprising truth about how robots learn to see. The most important factor was not whether the training data was real or fake, nor was it whether the data came from a volcano or a computer screen. The deciding factor was how closely the visual appearance and the shape of the ground in the training data matched the actual Moon. The models trained on the indoor lunar analog facilities performed the best when transferred to the real lunar images. These facilities, which use real soil and rocks arranged to look like the Moon under controlled lighting, provided the most reliable training ground. Surprisingly, the computer simulations, which are entirely synthetic, performed better than the real-world volcanic datasets from Italy. This suggests that the specific texture and lighting of the Italian volcanoes were too different from the Moon, confusing the robot, whereas the simulations managed to capture the essential visual cues needed for the task.

One of the most critical findings of the study challenges how scientists currently verify their systems. It is common practice to test a robot's performance on the same type of data it was trained on. The researchers found that a model could perform perfectly on its training data yet fail miserably when moved to the Moon. For instance, a model trained on the Italian volcanic data was excellent at identifying safe paths on those specific volcanoes, but it was one of the worst performers when tested on the lunar surface. This disconnect means that a robot passing all its internal tests could still be a disaster in space. The study showed that the visual and physical similarity between the training environment and the target environment is a much stronger predictor of success than the realism of the data or the category it belongs to.

The researchers also explored whether adding a small amount of real lunar data to the training set could fix the problem. They took the best-performing models and swapped out some of the training images with real photos from the Moon. They found that adding even a small number of real lunar images improved the robot's performance, but it did not completely close the gap. No matter how much real data was added, the models trained on the best lunar analog facilities still outperformed those trained on other sources. This indicates that while real data is helpful, the foundation built during the initial training phase matters most. If the robot is first taught to recognize the Moon in an environment that looks and feels like the Moon, it will be far more capable than one that is first taught on a very different landscape, even if that landscape is real.

Ultimately, this study provides a clear guide for future lunar missions. It suggests that before sending a robot to the Moon, engineers should prioritize training it in environments that closely replicate the Moon's specific look and feel, such as indoor facilities with lunar-like soil, rather than relying on generic Earth landscapes or purely computer simulations. The research also warns that standard testing methods, which only check if a robot works on its training data, are insufficient for space exploration. To ensure a rover can navigate the alien terrain of the Moon, its training and testing must happen across different domains, ensuring that the lessons learned in the lab hold true when the robot finally steps onto the lunar surface.

Technical Summary: Training Data Selection for Weakly-Supervised Lunar Traversability Estimation

Problem Statement
Autonomous lunar rovers require reliable traversability assessment to navigate unfamiliar terrain. However, learning-based perception systems face a fundamental data scarcity challenge: annotated lunar datasets are extremely limited, necessitating that models be trained on Earth-based data (simulations, terrestrial analogs, or lunar analog facilities) and transferred to the lunar environment. Existing fully-supervised approaches rely on dense per-pixel labeling, which is costly and imposes rigid semantic taxonomies ill-suited to the continuous nature of terrain. While weakly-supervised methods using sparse pairwise annotations offer a more efficient alternative, their cross-domain generalization behavior remains largely unexplored. Practitioners currently lack a quantitative basis for selecting the most effective training source among heterogeneous options (synthetic, terrestrial, or analog) when target-domain data is unavailable.

Methodology
The study adopts a weakly-supervised learning framework based on the W-RIZZ and CHUNGUS paradigms.

Model Architecture: The system utilizes a frozen DINOv2 ViT-S backbone for feature extraction, followed by a coordinate-based cross-attention upsampler (LoftUp) to generate per-pixel features, and a lightweight MLP head to output traversability scores. Only the MLP head is trained.
Training Signal: Instead of dense labels, the model is trained on sparse pairwise annotations indicating relative traversability (more, less, or equal) between point pairs within and across images.
Datasets: The study evaluates cross-domain transfer across seven datasets categorized into four groups:
1. Real Lunar Mission Imagery: Chang'E-3 and Chang'E-4 (CE-3/4) panoramic camera data.
2. Lunar Analog Facilities: LunaPolaris (indoor, controlled lighting) and In-House (OTRK Kakolewo, diverse rock formations).
3. Terrestrial Planetary Analogs: ETNA LRNT and S3LI (collected on Mt. Etna, Italy).
4. Synthetic Simulations: LunarSim (Unity) and LuSNAR (Unreal Engine).
Evaluation Metric: The Human Disagreement Rate (HDR) is used to measure the fraction of annotated point pairs where the model's predicted ordinal relation disagrees with human labels, evaluated across a range of score-difference thresholds ( $t \in [0.10, 0.50]$ ).
Experimental Design: A systematic cross-domain matrix was constructed involving 49 train-evaluation combinations. Experiments were run 15 times with different random seeds to ensure statistical robustness. A specific experiment also investigated the effect of substituting a portion of source data with target-domain (CE-3/4) data to measure domain gap closure.

Key Contributions

Systematic Cross-Domain Evaluation: This work presents the first comprehensive evaluation of weakly-supervised traversability models across a spectrum of training sources, from synthetic simulations to real lunar imagery.
Empirical Ranking of Training Sources: The study establishes a consistent hierarchy of training data effectiveness for lunar transfer, demonstrating that visual and morphological similarity to the target domain is a stronger predictor of performance than dataset realism or domain category.
Identification of Evaluation Limitations: The paper highlights that strong in-domain performance does not predict cross-domain generalization, exposing a critical flaw in standard verification protocols that rely solely on in-domain metrics.
Validation of Lunar Analog Facilities: The research validates the use of controlled indoor lunar analog facilities (specifically the newly introduced In-House facility at OTRK Kakolewo) as highly effective training sources, outperforming both terrestrial analogs and synthetic simulations.

Results

Source Ranking: When transferring to real lunar imagery (CE-3/4), lunar analog facilities (In-House and LunaPolaris) achieved the lowest HDR (best performance), followed by terrestrial analogs (ETNA LRNT) and synthetic datasets (LuSNAR, LunarSim). The S3LI terrestrial analog dataset performed the worst, despite achieving the best in-domain performance.
Visual vs. Realism: Synthetic datasets (LunarSim, LuSNAR) outperformed the S3LI terrestrial analog, indicating that visual and morphological alignment with the lunar surface is more critical than the photorealism or origin of the data.
Failure Modes: Models trained on S3LI tended to under-predict hazards on uneven terrain, while ETNA LRNT models over-penalized flat surfaces, likely due to the rough volcanic texture of the training environment.
Threshold Stability: The ranking of training sources remained stable at the extremes (best and worst) across all HDR thresholds ( $t \in [0.10, 0.50]$ ), though intermediate sources showed significant divergence at higher thresholds.
Domain Gap Reduction: Introducing limited target-domain data (substituting source pairs with CE-3/4 pairs) monotonically improved performance for all sources. However, even with 50% target data substitution, no source fully closed the gap to a model trained entirely on CE-3/4, and the initial source ranking was largely preserved.

Significance and Claims
The paper argues that for lunar robotics, where target-domain data is inaccessible during training, the selection of training data is a critical design decision. The findings suggest that:

In-domain metrics are insufficient: A model's high performance on its training distribution does not guarantee success in the lunar environment; cross-domain evaluation must be a standard component of the verification and validation pipeline.
Morphological alignment is paramount: Training environments that replicate lunar surface morphology (such as indoor analog facilities with regolith-like terrain) are superior to both terrestrial volcanic analogs and synthetic simulations for transfer tasks.
Data selection has lasting impact: While adding limited target-domain data improves performance, it cannot fully compensate for a poor initial choice of training source. Therefore, early investment in morphologically appropriate training environments yields benefits that post-hoc data augmentation cannot fully recover.

The study concludes by calling for a shift in perception system development toward prioritizing training data collection in environments that closely mimic lunar surface morphology, rather than relying solely on high-fidelity simulations or terrestrial analogs.

Technical Summary: Training Data Selection for Weakly-Supervised Lunar Traversability Estimation

More like this