Addressing Camera Sensors Faults in Vision-Based Navigation: Simulation and Dataset Development

This paper addresses the lack of representative faulty data for training AI-based fault detection in Vision-Based Navigation by systematically characterizing camera sensor faults and introducing a simulation framework to generate a comprehensive dataset of fault-injected images for interplanetary exploration missions.

The Gist

Imagine you are sending a robot explorer to a distant, rocky asteroid. This robot doesn't have a human pilot holding the controls; it has to navigate entirely on its own using its "eyes"—cameras. This is called Vision-Based Navigation.

The problem is, space is a harsh place. Just like your smartphone camera can get a smudge of dust on the lens, a broken pixel, or a glare from the sun, a space camera can suffer from similar "sicknesses." If the robot's eyes get blurry or blinded, it might crash, get lost, or fail its mission.

This paper is about building a training school for Artificial Intelligence (AI) so it can learn to spot these "sick eyes" before they cause a disaster.

Here is the breakdown of what the authors did, using some everyday analogies:

1. The Problem: The "Blind Spot" in AI Training

Usually, to teach an AI to recognize a disease (like a broken camera), you need to show it thousands of pictures of that disease. But in space, we don't have a giant library of photos of broken space cameras. We have plenty of photos of working cameras, but very few of broken ones.

Without these "sick" photos, the AI is like a medical student who has only ever seen healthy people. When a real patient (a broken camera) shows up, the student has no idea what's wrong.

2. The Solution: The "Cosplay" Simulator

Since the authors couldn't wait for a real camera to break in space (which might take years or never happen), they built a digital simulator. Think of this like a video game engine or a special photo-editing software.

They created a virtual world featuring:

• A comet (specifically 67P/Churyumov-Gerasimenko, the same one the Rosetta mission visited).
• A virtual spacecraft.
• A virtual camera.

Then, they used this simulator to artificially break the camera in thousands of different ways. They didn't just take a photo and blur it; they simulated the physics of how space actually breaks things.

3. The "Diseases" They Simulated

The authors identified five main ways a space camera can get sick and simulated them:

• Dust on the Lens (The "Smudge"): Imagine landing a helicopter on a dusty planet. The dust kicks up and sticks to the camera lens. In the simulator, they painted digital "dust grains" over the image, making parts of the view dark and shadowy.
• Broken Pixels (The "Dead Spots"): Think of an old TV screen with a few dead pixels that are always black or always white. In space, radiation from the sun can zap the camera sensor, turning specific pixels "hot" (too bright) or "dead" (too dark). They simulated these as tiny, annoying dots that ruin the picture.
• Straylight (The "Glare"): Have you ever taken a photo with the sun in the frame, and suddenly you see a rainbow flare or a hazy ghost across the image? That's straylight. In space, this is dangerous because it can hide the rocks the robot needs to avoid. They simulated this by adding digital "ghosts" and "rings" of light to the images.
• Vignetting (The "Dark Corners"): This is when the edges of a photo look darker than the center, like looking through a tunnel. It's a natural flaw in almost all cameras, but in space, it can make the robot think the edges are in shadow when they aren't.
• Optics Degradation (The "Foggy Glasses"): Over years in space, the lens gets worn down by radiation and dust, making the whole image look blurry, like looking through a foggy window. They simulated this by applying a "blur" filter to the images.

4. The Result: The "Sick Camera" Dataset

The authors used their simulator to generate 5,000 images.

• Some images are "healthy" (perfectly clear).
• Some have one "sickness" (just dust).
• Some have a mix (dust and a broken pixel and a sun glare).

Crucially, they also created masks (like stencils) for every image. If you look at a "sick" image, the mask tells the AI exactly where the sickness is. It's like a teacher giving a student a test paper with the answers already highlighted in red.

5. Why This Matters

Now, researchers can take this dataset and feed it to an AI. The AI learns to look at a picture and say:

• "Ah, this image has a sun glare in the top right corner."
• "This image has a broken pixel cluster in the middle."

Once the AI is trained on these fake "sick" photos, it can be put on a real spacecraft. If the real camera starts to get dusty or blinded by the sun, the AI will spot it immediately. It can then tell the spacecraft, "Hey, my eyes are blurry, I can't navigate safely right now," allowing the computer to switch to a backup plan or wait for the dust to settle.

The Big Picture

This paper is essentially a recipe book for creating fake space disasters. By creating a massive library of "what-if" scenarios, the authors are helping the space industry build smarter, safer robots that won't crash just because their camera got a little dirty or a little bright. It's about teaching AI to be a vigilant doctor for our space explorers.

Technical Summary

1. Problem Statement

Vision-Based Navigation (VBN) is becoming critical for ambitious space missions, such as interplanetary exploration, in-orbit servicing, and active debris removal. However, the reliability of VBN algorithms is threatened by sensor faults (e.g., dust, broken pixels, straylight) which can lead to inaccurate navigation outputs or complete processing failures.

While Artificial Intelligence (AI) offers a robust solution for Fault Detection, Isolation, and Recovery (FDIR) in image processing, its adoption is hindered by a lack of sufficient and representative datasets containing faulty image data. Real-world space data is scarce, and manually labeling faults in existing mission data is inefficient. Furthermore, traditional FDIR methods based on telemetry standards (PUS) struggle with the multi-dimensional nature of image data. There is a specific gap in simulating faults for Small Solar System Body (SSSB) exploration scenarios, such as the proposed Astrone KI mission (autonomous relocation on a comet).

2. Methodology

The authors developed a comprehensive framework to simulate camera sensor faults and generate a synthetic dataset tailored for AI training in VBN contexts.

A. Simulation Environment

• Platform: The study utilizes CamSim (developed by ASTOS Solutions GmbH), a high-fidelity image simulator integrated with Simulink.
• Scenario: The environment models Comet 67P/Churyumov-Gerasimenko (based on Rosetta mission data) and a spacecraft mock-up (Astrone KI).
• Reference Frames: The simulation employs multiple reference frames (International Celestial, Target-Body Fixed, Local-Level, Body-Fixed, and Camera-Fixed) to accurately define spacecraft attitude, position, and illumination conditions.

B. Fault Characterization and Injection

The authors identified five critical fault categories affecting optical sensors in space and developed specific simulation strategies for each:

1. Dust on Optics: Simulated by overlaying Gaussian bivariate distributions on the image intensity channel to mimic the shadowing effect of dust grains accumulating on the lens. Parameters include grain count, size, and darkening intensity.
2. Broken Pixels: Modeled based on radiation effects (Hot/Dead pixels) and readout electronics failures.
   • Single Pixels: Simulated as "Salt & Pepper" defects with randomized brightness (0–255).
   • Demosaicing Effects: Neighboring pixels are corrupted via linear interpolation to mimic color filter array artifacts.
   • Line Defects: Simulated full rows or columns of dead/saturated pixels, accounting for specific sensor architectures (CCD vs. CMOS).
3. Straylight: Simulated using pre-defined textures (flares and glares) overlaid on the image. The simulation ensures flares align with the light source (Sun) and vary in size and density based on distance from the source, mimicking physical lens behavior without requiring specific lens geometry.
4. Vignetting: Modeled using the Cosine-fourth law to create a radial brightness decay from the center to the edges of the image.
5. Optics Degradation: Simulated as a uniform image blurring using a Gaussian Filter with randomized kernel sizes.

C. Dataset Generation Strategy

• Stratified Sampling: To ensure a balanced dataset, the authors used a two-stage sampling process:
  1. Sun-facing: Images where the Sun is within the Field of View (FoV) or close to it (to capture Straylight).
  2. Sun-averted: Images where the Sun is outside the FoV.
• Fault Injection Matrix: A matrix controls the injection of faults for each generated image, ensuring a balanced distribution of fault classes.
• Labeling: The system generates binary segmentation masks for each fault type. For straylight, a post-processing step subtracts the nominal image from the faulty image to isolate the flare texture for the mask.

3. Key Contributions

1. Comprehensive Fault Analysis: A detailed Failure Modes and Effects Analysis (FMEA) specific to interplanetary VBN, covering dust, pixel defects, straylight, vignetting, and degradation.
2. Novel Simulation Framework: A flexible, physics-informed simulation pipeline (CamSim + Simulink) capable of injecting diverse faults into synthetic imagery with controlled parameters.
3. Specialized Dataset: The creation of a dataset comprising 5,000 synthetic images of Comet 67P with injected faults.
   • Includes diverse illumination conditions and spacecraft attitudes.
   • Provides pixel-level ground truth (segmentation masks) for all fault types.
4. Stratified Sampling Methodology: A technique to overcome the bias of random sampling regarding straylight, ensuring the dataset contains a statistically significant number of Sun-facing and Sun-averted scenarios.

4. Results

• Dataset Generation: Successfully generated 5,000 images with varying combinations of the five fault types.
• Visual Realism: The simulation produces images where faults (e.g., dust shadows, flare rings, dead pixel lines) are visually distinct and physically plausible within the context of the comet environment.
• Labeling Accuracy: The automated labeling process successfully produced binary masks for every fault class, enabling the dataset to be used for both image classification (detecting if a fault exists) and image segmentation (locating the fault pixels).
• Applicability: The paper demonstrates that the generated faults are relevant not only for the Astrone KI mission but also for other mission classes (Landing, Earth Observation, Interplanetary), as summarized in the authors' applicability matrix.

5. Significance and Future Work

• Enabling AI for Space FDIR: This work addresses the "data bottleneck" preventing the deployment of AI for onboard fault detection. By providing a labeled dataset of faulty space imagery, it allows researchers to train robust AI models to detect anomalies before they compromise navigation.
• Mission Safety: The ability to detect and mitigate sensor faults (e.g., distinguishing between a broken pixel and a real terrain feature) is crucial for the safety of autonomous missions in harsh, unshielded radiation environments.
• Future Directions:
  • Deployment: Using the dataset to train AI models for the Astrone KI mission's anomaly detection subsystem.
  • Generalization: Expanding the simulation to include more complex physical models (e.g., physically-based rendering for straylight) and other mission scenarios.
  • Open Access: The dataset is intended for public release (after an embargo) to foster further research in space AI and VBN reliability.

In conclusion, this paper provides a critical infrastructure for the next generation of autonomous space exploration by bridging the gap between theoretical fault analysis and practical AI training data.