Technical Summary: BEACON

Problem Statement

Robot learning via behavior cloning (BC) has advanced significantly with sequence models like Diffusion Policies, yet a critical bottleneck remains: policies trained in one domain (source) often degrade when deployed in another (target), even when task semantics are identical. These domain gaps arise from visual discrepancies (texture, lighting, camera pose) and physical mismatches (friction, contact dynamics, actuation delays). While collecting large-scale target-domain datasets is expensive and difficult to scale, source data (e.g., from simulation or human demonstrations) is abundant but often mismatched.

Existing approaches to this problem fall into two categories:

1. Domain Randomization: Increases source diversity to improve robustness but does not explicitly align distributions.
2. Domain Adaptation: Explicitly aligns source and target representations (e.g., via feature matching or adversarial training). However, these methods often rely on heuristic objectives, may miss non-visual shifts, and struggle with multi-source settings where sources differ from each other as well as the target.

The central question addressed by this paper is: How can we leverage abundant but mismatched source demonstrations to improve target-domain generalization while minimizing the need for labeled target data, without relying on heuristic alignment losses?

Methodology: BEACON

The authors propose BEACON (Best-Effort Adaptation for Cross-Domain Co-Training), a theory-driven framework that recasts cross-domain co-training as a discrepancy-aware importance-reweighting problem.

Theoretical Foundation

BEACON is grounded in the Best-Effort Adaptation (BEA) framework [23]. Instead of naively mixing source and target data or enforcing feature alignment, BEACON jointly learns a policy $\pi_\theta$ and a set of per-sample weights $q$ to minimize a theoretically derived generalization bound for the target domain.

The objective function (Eq. 6) minimizes:
$$\sum_{i=1}^{m+n} q_i (\ell_{BC}(\theta; x_i, y_i) + \lambda_d d_i) + \gamma \|q\|_\infty R(\theta) + \lambda_1 \|q - p_0\|_1 + \lambda_2 \|q\|_2^2$$

Where:

• $\ell_{BC}$ is the diffusion behavior cloning loss.
• $d_i$ is a discrepancy term penalizing source samples that are mismatched with the target.
• $p_0$ is a reference weighting (target-only empirical distribution).
• The regularization terms ($\lambda_1, \lambda_2$) ensure the learned weights remain anchored to the target distribution and prevent degenerate solutions.

Practical Adaptations for Robotics

Applying the original BEA framework to high-dimensional diffusion policies required several key adaptations:

1. Instance-Level Discrepancy Estimation: Instead of a single domain-level penalty, BEACON computes a per-sample discrepancy score $d_i$. The default estimator uses k-Nearest Neighbors (k-NN) in the policy's vision-encoder embedding space. It measures the distance of a source sample to its nearest target neighbors, normalized by the within-target spread. This allows the system to identify and down-weight specific "outlier" source trajectories that are visually or behaviorally incompatible with the target, even within a single simulator.
2. Stochastic Alternating Optimization: Due to the computational cost of diffusion models, BEACON employs stochastic alternating updates rather than full-batch minimization. It alternates between:
   • Weight Update ($q$): A projected subgradient step based on the per-example loss and discrepancy.
   • Policy Update ($\theta$): An AdamW step on the weighted minibatch loss.
   • Complexity Control: The original norm-based capacity control is replaced by optimizer-compatible weight decay ($\gamma \|q\|_\infty R(\theta)$), linking the maximum sample weight to the regularization strength.
3. Multi-Source Extension (MS-BEACON): For scenarios with heterogeneous source domains (e.g., multiple simulators), the method introduces a two-level weighting scheme: a domain-level weight $w$ to select useful source domains, and an instance-level weight $\tilde{q}$ to filter mismatched trajectories within each domain.

Key Contributions

1. Theory-Driven Co-Training: BEACON is the first method to adapt the BEA framework to generative robot policies, directly minimizing a discrepancy-based generalization bound rather than relying on heuristic feature-alignment losses.
2. Scalable Instance-Level Reweighting: The introduction of scalable, instance-level discrepancy estimators (k-NN in embedding space) allows the method to handle high-dimensional sequence policies and heterogeneous source data effectively.
3. Implicit Feature Alignment: The paper demonstrates that feature alignment emerges naturally as a byproduct of the discrepancy-aware reweighting objective, without requiring an explicit alignment loss term.
4. Multi-Source Robustness: The MS-BEACON extension effectively balances heterogeneous source domains, mitigating negative transfer when sources differ significantly from each other and the target.

Experimental Results

BEACON was evaluated across sim-to-sim, sim-to-real, and multi-source manipulation tasks (block stacking, mug cleanup, threading) on a Franka Emika Panda robot.

• Superior Generalization: BEACON consistently outperformed baselines, including Target-Only, fixed-ratio Co-Training, and feature-alignment methods (MMD, UOT).
  • In sim-to-sim (texture and viewpoint gaps), BEACON achieved an average success rate of 0.64 (texture gap) and 0.45 (texture + viewpoint), surpassing the next best baseline (MMD) by a significant margin.
  • In sim-to-real, where Target-Only policies nearly failed (0.10 average success), BEACON achieved 0.77 average success, demonstrating robustness to both visual and physical gaps.
• Out-of-Distribution (OOD) Robustness: BEACON maintained performance when evaluated on task instances outside the distribution of target demonstrations, whereas feature-alignment baselines (like MMD) suffered significant degradation.
• Implicit Alignment: UMAP visualizations confirmed that BEACON brings source and target latent features closer together as a result of the reweighting dynamics, despite lacking an explicit alignment loss.
• Data Efficiency: Scaling up target demonstrations further improved BEACON's performance, indicating that source reweighting amplifies the value of target data rather than replacing it.

Significance and Claims

The paper claims that BEACON offers a complementary paradigm to existing domain adaptation methods. Rather than focusing on forcing latent spaces to match (which can collapse task-relevant geometry), BEACON focuses on selecting which source trajectories should influence target learning.

The authors assert that:

• Importance reweighting is a more fundamental design lever for cross-domain robot learning than the choice of where to align representations in a network.
• Grounding this lever in generalization theory yields concrete gains in robustness and data efficiency.
• The method is particularly effective in low-target-data regimes, where collecting large target datasets is infeasible.

The paper acknowledges limitations, noting that experiments were restricted to quasi-static interactions and that the assumption of scarce real data might not hold in all future scenarios where large, heterogeneous real-world datasets might be available. However, the results suggest that the proposed reweighting mechanism remains a robust strategy for handling domain shifts.