xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing

The paper proposes xTED, a flexible cross-domain adaptation framework that utilizes a diffusion model to edit and transform source domain trajectories into target domain distributions at the data level, thereby bridging domain gaps and enhancing policy learning performance without requiring complex domain-specific modeling.

The Gist

Imagine you are trying to teach a robot how to cook a specific dish, like a perfect omelet.

The Problem:
You have a recipe book (data) from a famous chef in Paris (the Target Domain). But you only have a few pages of it. You also have a massive library of recipes from a chef in Tokyo (the Source Domain). The Tokyo recipes are great, but they use different ingredients, different pans, and the chef holds the whisk differently.

If you just dump the Tokyo recipes into your Parisian cookbook, the robot gets confused. It tries to use a Japanese wok in a French kitchen and ends up burning the eggs. This is the "Domain Gap."

The Old Way (The "Translator" Approach):
Previous methods tried to build a complex, expensive translator. They would say, "Okay, when the Tokyo chef says 'whisk fast,' the Parisian robot needs to 'whisk slow.' When Tokyo uses 'soy sauce,' Paris needs 'salt'."
This requires building a massive, custom machine for every single pair of chefs. It's rigid, hard to maintain, and if you add a third chef from New York, you have to rebuild the whole machine.

The New Way (xTED): The "Style Transfer" Magic
The paper introduces xTED (Cross-Domain Trajectory Editing). Instead of building a translator for the robot, xTED acts like a magic photo editor for the robot's memories.

Think of the robot's learning data as a series of video clips showing the chef's hand movements.

1. The Concept: In image editing, you can take a sketch of a cat and use AI to turn it into a photorealistic cat without changing the pose or the action. xTED does this for robot movements.
2. The Process:
   • Step 1: The system learns what a "perfect Parisian omelet" looks like by studying the few clips it has (the Target Data). It builds a mental model of the "Parisian style."
   • Step 2: It takes the Tokyo clips (Source Data) and adds "noise" to them—like blurring them slightly or shaking the camera. This removes the specific "Tokyo style" (the weird pan size, the different grip) but keeps the core action (the flipping motion).
   • Step 3: It asks the "Parisian model" to clean up the blur. The model reconstructs the movement, but this time, it fills in the details using the Parisian style.
   • Result: You now have a video of the Tokyo chef's exact same flipping motion, but it looks like it was filmed in a Parisian kitchen with a Parisian pan.

Why is this special?

• It's Universal: You don't need a new translator for every new robot. Once the "Parisian model" is trained, you can edit data from any other robot (Tokyo, New York, Mars) to fit Paris.
• It Keeps the Soul: It doesn't change what the robot is doing (the task), only how it looks and feels (the dynamics). It preserves the "primitive task information."
• It Works with Anything: Once the data is "edited" to look like the target, you can feed it to any standard learning algorithm. You don't need special cross-domain code anymore.

The Real-World Test:
The researchers tested this on real robots.

• Scenario: They had a robot arm (WidowX) that needed to learn to pick up a cup. They had data from a different robot arm (Airbot) that looked and moved very differently.
• Without xTED: If they just mixed the data, the robot failed miserably (0% success rate). It was confused by the conflicting instructions.
• With xTED: They "edited" the Airbot's data to look like WidowX data. The robot's success rate skyrocketed from 43% to 97%.

The Analogy Summary:
Imagine you are learning to drive a Ferrari (Target), but you only have a few hours of practice. You have thousands of hours of driving data from a Toyota (Source).

• Old Method: You try to build a complex adapter to make the Toyota's steering wheel feel like a Ferrari's while you drive. It's clunky and confusing.
• xTED Method: You take the Toyota driving footage, run it through a filter that changes the car's physics, the road texture, and the steering feel to match a Ferrari, but keeps the turns, stops, and accelerations exactly the same. Now you have a Ferrari driving video that you can use to learn perfectly.

In a nutshell: xTED is a tool that "translates" robot experiences from one world to another at the data level, making it easy to reuse old data for new robots without needing complex, custom-built solutions.

Technical Summary

Here is a detailed technical summary of the paper "xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing".

1. Problem Statement

Reinforcement Learning (RL) and Imitation Learning (IL) often face severe data scarcity in real-world target domains. A common solution is to leverage pre-collected data from source domains (e.g., simulations or different robot embodiments). However, direct transfer is hindered by domain gaps, including:

• Appearance/Viewpoint Gaps: Differences in camera angles or visual inputs.
• Dynamics Gaps: Differences in physics (gravity, friction, mass).
• Morphology Gaps: Differences in robot body structures (e.g., arm length, joint types).

Limitations of Existing Methods:
Current cross-domain policy transfer methods typically attempt to bridge these gaps during the policy learning process. They rely on:

• Learning domain-specific discriminators or correspondences.
• Designing complex, task-specific architectures (e.g., special encoders).
• Applying domain-specific regularizations.
  These approaches often result in heavy models, lack flexibility for multiple source domains, and fail to address the root cause: the domain gaps existing within the data itself.

2. Methodology: xTED Framework

The authors propose xTED (Cross-Domain Trajectory EDiting), a paradigm that reframes cross-domain adaptation as a data pre-processing problem. Instead of adapting the policy, xTED edits the source trajectories to align with the target domain's distribution while preserving the original task semantics.

Core Philosophy

Inspired by diffusion-based image editing (e.g., changing an image's style while keeping its content), xTED treats trajectory editing as a process of removing domain-specific "noise" (biases) while retaining task-relevant "content" (primitive skills).

Key Technical Components

A. Specialized Diffusion Model Architecture
Unlike standard diffusion models that treat data as homogeneous pixels, xTED addresses the heterogeneity of decision-making data (States, Actions, Rewards).

1. Separate Encoding/Decoding: States ($s$), Actions ($a$), and Rewards ($r$) are encoded and decoded separately using distinct sub-networks. This preserves their distinct physical meanings and prevents the model from learning spurious correlations.
2. Dependency Structure Modeling: The architecture explicitly models the causal and temporal dependencies between elements:
   • State-Action Cross-Attention: Captures mutual dependencies between states and actions.
   • Reward Dependency: Rewards are queried using concatenated state-action embeddings (reflecting the causal direction: $s, a \to r$), but not vice versa.
3. Temporal Modeling: Self-attention blocks are used within each modality to capture long-horizon temporal consistency.

B. The Editing Process (SDEdit-inspired)
The editing process involves two stages:

1. Forward Process (Perturbation): Source trajectories ($\tilde{\tau}_0$) are perturbed with Gaussian noise to an intermediate step $k$ (controlled by a ratio $\kappa = k/K$). This removes fine-grained domain-specific details (e.g., specific dynamics) while retaining mesoscopic task primitives.
2. Reverse Process (Denoising): The noisy source trajectory is denoised using a diffusion model pre-trained exclusively on target domain data. This forces the trajectory to conform to the target domain's dynamics and observation patterns.
3. Constraint: The initial state-action-reward triple is kept unchanged during editing to ensure the task goal remains consistent.

C. Integration
The edited source data is simply concatenated with the target data. Any downstream policy learning algorithm (RL or IL, single-domain or cross-domain) can then be applied without modification.

3. Key Contributions

1. Data-Level Adaptation Paradigm: Shifts the focus from complex policy-level adaptation to data-level editing, offering a generic, flexible solution compatible with any downstream algorithm.
2. Novel Architecture: Introduces a diffusion model architecture specifically designed for sequential decision-making, handling the heterogeneous nature of states, actions, and rewards and their intricate dependencies.
3. Task and Domain Agnosticism: The method does not require retraining when new source domains are introduced; the same pre-trained target diffusion model can edit data from multiple source domains.
4. Dual Utility: Functions effectively as both a cross-domain adapter and a single-domain data augmentation tool (generating high-quality synthetic trajectories from small datasets).

4. Experimental Results

The authors evaluated xTED on both real-robot manipulation tasks and MuJoCo simulation environments.

Real-Robot Experiments

• Setup: Source data collected from an Airbot robot; Target data from a WidowX robot. Tasks included picking cups, ducks, and pots with distractors.
• Results:
  • Target + Edited Source: Achieved success rates of 97% (Cup), 83% (Duck w/ distractors), and 80% (Pot w/ distractors).
  • Target + Raw Source: Often resulted in performance degradation (e.g., dropping to 0% success in the Pot task) due to domain gaps.
  • Improvement: Edited data improved success rates by >100% in some cases compared to training on target data alone.

Simulation Experiments (MuJoCo)

• Setup: Source domains created by introducing dynamics gaps (2x gravity, 0.25x friction, 2x thigh size) on HalfCheetah and Walker2d.
• Results:
  • Performance Gain: xTED consistently improved performance across 18/18 tasks.
  • Comparison: Directly adding unprocessed source data degraded performance in 5/18 tasks. xTED provided an average 16.4% improvement over target-only training.
  • Dynamics Error: Edited source data showed dynamics errors (MAE) comparable to target data, significantly lower than raw source data.
  • Compatibility: When combined with other cross-domain methods (e.g., DARA), xTED further enhanced their performance.

Ablation Studies

• Architecture: The proposed separate encoding/decoding and dependency modeling significantly outperformed baselines that concatenated data (Feature/Transition concatenation) or used inverse dynamics models.
• Editing Ratio ($\kappa$): $\kappa=0.5$ was found to be the optimal balance, preserving task semantics while removing domain bias.
• Single-Domain Augmentation: xTED outperformed standard augmentation methods (S4RL, SER) in small-data regimes.

5. Significance

• Simplicity and Flexibility: xTED decouples data adaptation from policy learning, allowing researchers to use the best policy learning algorithm for their specific task without worrying about cross-domain architectural complexities.
• Scalability: It handles multiple source domains simultaneously without retraining the diffusion model, making it highly scalable for robotic learning where data comes from diverse simulators or robots.
• Robustness: By correcting the underlying data distribution, it prevents the negative transfer often seen when naively mixing source and target data, particularly in high-stakes real-world robotics.
• Generalizability: The frequency-domain analysis suggests that domain gaps primarily exist in high-frequency components (dynamics), while low-frequency components (task semantics) remain consistent. xTED effectively targets these high-frequency discrepancies, validating the diffusion-based editing approach for sequential decision-making.

In conclusion, xTED provides a powerful, plug-and-play solution for cross-domain robotic learning, demonstrating that editing the data is often more effective and flexible than adapting the policy.