Digital and Robotic Twinning for Validation of Proximity Operations and Formation Flying

This paper presents a unified, closed-loop digital and robotic twinning framework that integrates faster-than-real-time simulations with Stanford's robotic testbeds to validate and verify the performance of spacecraft guidance, navigation, and control systems for rendezvous and formation flying missions.

The Gist

Imagine you are trying to teach a robot to park a car in a tight spot, but instead of a garage, the car is a spaceship, the "parking spot" is another satellite floating in the vacuum of space, and you can't just drive out and try it because space is too expensive and dangerous to mess up.

This paper describes a brilliant solution to that problem: a "Digital and Robotic Twinning" framework. Think of it as a high-tech, multi-layered training simulator that bridges the gap between a video game and the real world.

Here is the breakdown in simple terms:

1. The Problem: The "Space is Hard" Dilemma

Spacecraft need to fly close to each other (for docking, repairs, or formation flying) without crashing. To do this, they need a "brain" (software) that can see where they are and steer them perfectly.

• The Issue: You can't just test this software on a real satellite in space yet. If the software has a bug, the satellite crashes, and millions of dollars are lost.
• The Old Way: Engineers used to run two separate tests:
  1. Pure Simulation (Software-in-the-Loop): Like playing a flight simulator on a PC. It's fast and cheap, but it's perfect. Real space isn't perfect.
  2. Physical Testing (Hardware-in-the-Loop): Putting the real computer chip in a box and testing it. But this is slow, expensive, and usually only tests one specific thing (like just the camera or just the radio).

2. The Solution: The "Hybrid Twin"

The authors built a system that mixes the best of both worlds. Imagine a video game where you can swap out the graphics engine for real physical objects whenever you want.

• The Digital Twin (The Video Game): This is a super-fast computer simulation of space. It calculates gravity, sunlight, and how the satellite moves. It can run a whole mission in minutes.
• The Robotic Twin (The Real World): This is where the magic happens. The system can "swap out" the fake camera or radio in the simulation with real hardware sitting in a lab at Stanford University.

3. The Three "Training Gyms" (Testbeds)

To make this work for different types of space missions, they built three specialized "gyms" where the satellite's brain can practice:

• The "Star Projector" (Optical Stimulator - OS):
  • Analogy: Imagine a movie projector shining images of stars and other spaceships onto a screen.
  • What it does: It tests the satellite's camera. The satellite thinks it's looking at space, but it's actually looking at a screen in a lab. This checks if the camera can handle real glare, noise, and lens distortion.
• The "Robot Dance Floor" (TRON):
  • Analogy: Two giant industrial robot arms holding a model spaceship and a camera, moving around a room.
  • What it does: This tests close-range docking. The robots move the "target" and the "chaser" (the satellite) to simulate them getting very close to each other. It checks if the camera can recognize the other ship when it's just a few meters away.
• The "Radio Wave Machine" (GRAND):
  • Analogy: A giant radio tower that broadcasts fake GPS signals.
  • What it does: This tests the radio navigation. Instead of using a camera, the satellite uses radio waves to find its partner. This machine creates realistic radio signals that include all the messy errors real space signals have.

4. How They Tested It (The "Driving Test")

The researchers took a piece of software (the satellite's brain) and put it through a full mission scenario:

1. The Long Haul: Starting 75 kilometers away (far away).
2. The Approach: Getting closer and closer.
3. The Docking: Ending up just 7 meters away.

They ran this mission twice for every scenario:

• Run A (Pure Digital): The software ran in the computer simulation.
• Run B (Hybrid): The software ran in the computer, but the "eyes" (camera) and "ears" (radio) were the real physical machines in the lab.

5. The Results: "It Works, But Reality is Messy"

The results were fascinating:

• Consistency: The software behaved almost exactly the same in the simulation and the real lab. This proves the simulation is accurate.
• The "Reality Gap": However, the real hardware tests revealed tiny glitches that the perfect simulation missed. For example, the real camera sometimes got confused by weird lighting or shadows that the computer simulation didn't predict.
• The Lesson: The simulation is great for planning, but the robotic twin is essential for finding the "gotchas" before you launch.

The Big Takeaway

This paper introduces a universal training ground for space robots. It allows engineers to:

1. Design software quickly in a fast computer simulation.
2. Swap in real hardware to stress-test it.
3. Catch bugs and errors before the satellite ever leaves Earth.

It's like teaching a self-driving car: you start in a video game, then you test it in a controlled parking lot with real sensors, and only then do you let it drive on the highway. This framework ensures that when our satellites finally meet in space, they won't crash into each other.

Technical Summary

1. Problem Statement

Autonomous Guidance, Navigation, and Control (GNC) systems are critical for Spacecraft Rendezvous, Proximity Operations (RPO), and Formation Flying (FF). However, verifying these systems is exceptionally difficult due to:

• Inaccessibility: The space environment cannot be fully replicated on Earth.
• Complexity: GNC algorithms must handle diverse sensing modalities (vision-based and radio-frequency) across varying inter-satellite distances (ISD), from far-range (unresolved targets) to close-range (resolved features).
• Limitations of Current V&V:
  • Software-in-the-Loop (SIL): While fast and flexible, digital twins often fail to capture hardware imperfections, sensor noise, and timing constraints.
  • Hardware-in-the-Loop (HIL): Existing testbeds are often specialized (e.g., only for GNSS or only for optical) and lack the modularity to test full mission profiles or multi-modal GNC stacks in a unified framework.

There is a need for a unified, modular framework that bridges the gap between rapid software prototyping and rigorous hardware validation to ensure robustness before flight.

2. Methodology: Hybrid Digital and Robotic Twinning Framework

The authors propose a hybrid, closed-loop, end-to-end framework that seamlessly integrates digital simulation with physical robotic testbeds.

A. Architecture Overview

The framework operates on a common "ground truth" dynamics engine. It allows for the seamless substitution of software sensor models with physical hardware components via a composable interface.

• Digital Twin Core: A high-fidelity, event-driven C++ simulation environment (developed at Stanford's Space Rendezvous Laboratory, SLAB). It includes:
  • Astrodynamics propagation (S3 library) with perturbations (gravity, drag, SRP).
  • SIL Sensor Models: An OpenGL-based renderer for vision and a GNSS Software Emulator for RF navigation.
  • GNC Stack: The autonomous flight software under test.
• Robotic Twin Components (Hardware): Three complementary testbeds replace specific digital sensor models:
  1. Optical Stimulator (OS): A variable-magnification optical bench using an OLED microdisplay and lenses to simulate far-, mid-, and close-range vision scenarios for star trackers and monocular cameras.
  2. Testbed for Rendezvous and Optical Navigation (TRON): A robotic system using two KUKA industrial robots to physically move a target model and a camera, simulating close-range kinematic motion and illumination (using albedo lightboxes).
  3. GRAND (GNSS and Radiofrequency Autonomous Navigation Testbed): A system combining an IFEN NOVA+ multi-GNSS signal generator with flight-heritage NovAtel receivers and ZYNQ7000 flight computers to validate RF-based navigation (CDGNSS).

B. The Test Article

The framework validates a multi-modal GNC software stack developed at SLAB, featuring:

• Management: Fault detection and mode transitions.
• Navigation: Fuses optical (Angles-Only Tracking, Spacecraft Pose Network) and RF (CDGNSS with Integer Ambiguity Resolution) data using Sequential Estimation and Data Fusion (SEDF).
• Safety & Control: Collision risk assessment and trajectory planning using various solvers (Chernick-D'Amico, Koenig-D'Amico, Least Squares).

C. Experimental Design

Three closed-loop experiments were conducted to cover a full-range RPO mission (75 km to 7 m) in Low Earth Orbit (LEO):

1. Exp 1 (Far-Range, Uncooperative): Angles-Only Tracking (AOT) + Closed-Form Control. Tested using SIL vs. OS.
2. Exp 2 (Close-Range, Uncooperative): Spacecraft Pose Network (SPN) + Koenig-D'Amico Control. Tested using SIL vs. TRON.
3. Exp 3 (Close-Range, Cooperative): CDGNSS with IAR + Koenig-D'Amico Control. Tested using SIL vs. GRAND.

3. Key Contributions

• Unified Hybrid Framework: A novel architecture that allows "plug-and-play" switching between SIL and HIL modes without modifying the GNC code, enabling consistent comparison.
• Multi-Modal Coverage: The first framework to integrate vision-based (far/mid/close range) and RF-based navigation testbeds into a single validation pipeline.
• Calibration and Characterization: Detailed calibration of the robotic testbeds (e.g., TRON's Robot-World-Hand-Eye calibration achieving ~2.9mm translation error) to quantify the "sim-to-world" gap.
• Incremental V&V Workflow: A structured methodology moving from unit tests to full-system HIL, isolating hardware-induced failures from algorithmic flaws.

4. Results

The study compared SIL and HIL performance across navigation error, control error, and cumulative $\Delta V$.

• Experiment 1 (AOT/Far-Range):
  • Result: SIL and HIL showed highly consistent navigation errors.
  • Insight: Within the calibrated region of interest, hardware optics did not significantly degrade the AOT algorithm. Navigation error increased relative to separation distance due to reduced observability, a trend consistent in both modes.
• Experiment 2 (SPN/Close-Range):
  • Result: HIL showed higher navigation error (approx. 2.7x mean error) compared to SIL.
  • Insight: The "domain gap" between synthetic training data and real hardware images (lighting artifacts, texture differences) caused the SPN (a deep learning model) to struggle with keypoint detection. This led to a temporary spike in state covariance and a 20% increase in $\Delta V$ expenditure due to corrective maneuvers.
• Experiment 3 (CDGNSS/Cooperative):
  • Result: SIL and HIL achieved similar steady-state accuracy (mm-level position error).
  • Insight: HIL convergence was slower due to non-stationary hardware dynamics (tracking-loop transients) not fully captured in SIL. This resulted in a 25% increase in initial $\Delta V$ for HIL as the filter reacted to transient noise, though long-term performance aligned.

General Findings:

• The framework successfully reproduced comparable scenarios between SIL and HIL.
• HIL testing revealed specific failure modes (e.g., domain gap in AI, hardware latency) that SIL alone missed.
• The modular approach allowed for clear explainability of performance deviations.

5. Significance

This work represents a significant advancement in the Verification and Validation (V&V) of space autonomy:

• Risk Reduction: By identifying hardware-induced performance degradation early, the framework reduces the risk of mission failure in complex RPO and Formation Flying missions.
• Scalability: The modular design supports future missions with different sensors or control strategies without rebuilding the entire test infrastructure.
• Bridging the Gap: It provides a quantitative method to assess the "sim-to-world" gap, allowing engineers to tune digital twins to better reflect reality.
• Future Application: The framework is positioned to support high-precision missions like NASA's STARI (Starlight Acquisition and Reflection toward Interferometry), where tight formation flying and rigorous validation are paramount.

In conclusion, the paper demonstrates that a hybrid digital and robotic twinning approach is a reliable, necessary evolution for validating the next generation of autonomous spacecraft systems.