Technical Summary: MirrorDuo

Problem Statement

Image-based behavior cloning (BC) offers a scalable path for robotic skill acquisition but faces significant data efficiency constraints, particularly when generalizing across spatial variations in workspace layouts or asymmetric scene configurations. While methods leveraging 3D inputs (e.g., point clouds) can exploit SE(3) rigid transformation equivariance to synthesize transformed demonstrations, applying such transformations to 2D RGB images often yields inconsistent results. Existing attempts to approximate symmetry in 2D, such as SO(2) rotation of third-person images, are limited to in-domain settings and fail to support broader generalization. Furthermore, reflection symmetry—a natural property of many manipulation tasks (e.g., left-right pick-and-place variants)—remains underexplored in image-based policies due to challenges in maintaining semantic consistency between mirrored images and 6-DoF end-effector actions, especially when visual artifacts (e.g., robot wrist asymmetry, background textures) are present.

Methodology: MirrorDuo

The authors propose MirrorDuo, a general formulation for incorporating reflection symmetry into visuomotor learning. The core innovation is a reflection-based transformation that operates jointly on RGB observations (both eye-in-hand and third-person), proprioceptive states, and full 6-DoF end-effector actions to generate a semantically and physically consistent mirrored counterpart for every original demonstration.

Key Technical Components

1. Pose Space Mirroring:

   • To ensure consistency with image mirroring (horizontal flip), the authors define a mirroring operation in pose space. For an end-effector pose $X_H$ in the camera frame, the mirrored pose $X^*_H$ is derived via $X^*_H = X_C E X^{-1}_C X_H E$, where $E = \text{diag}([-1, 1, 1, 1])$.
   • Camera Extrinsics Independence: To avoid dependency on often-unavailable camera extrinsics ($X_C$), the method re-parameterizes poses into local coordinate systems (delta pose $\Delta X_H$ or relative pose $\delta X_H$). This allows the mirroring operation to be expressed as $\Delta X^*_H = E \Delta X_H E$, making the approach dataset-agnostic.
   • SO(3) Discontinuity Handling: Reflection in SO(3) can cause abrupt changes in rotation. To ensure the robot starts the mirrored skill from a near-identical configuration, the method enforces a constant alignment rotation $Q$ that maps the mean tool axis of the initial poses to the world X-axis, mitigating discontinuities.

2. Unified Mirroring Operator ($M$):

   • The method defines a unified operator $M$ that applies horizontal flipping to images ($M_I$) and element-wise multiplication by a sign pattern vector $\rho$ to vectorized poses and actions. This creates a paired dataset of original and mirrored tuples $(o_t, a_t)$ and $(M(o_t), M(a_t))$.

3. Two Complementary Applications:

   • MirrorAug (Data Augmentation): A strategy where, during batch sampling, a portion of the training data is replaced by its mirrored counterpart. This expands the training distribution to include mirrored configurations without modifying the network architecture.
   • MirrorDiffusion (Structural Prior): A reflection-equivariant diffusion policy where the mirror transformation is embedded as an inductive bias within the latent diffusion process. The policy network utilizes E(n)-equivariant Steerable CNNs to enforce that the noise prediction function commutes with reflection transformations ($\pi(M(o)) = M(\pi(o))$).

4. Handling Visual Asymmetry:

   • The authors acknowledge that mirroring introduces visual out-of-distribution (OOD) artifacts (e.g., a right-handed robot appearing left-handed in the mirror, or asymmetric table textures).
   • To mitigate this, the approach combines MirrorDuo with Random Overlay (augmenting backgrounds) and pretrained visual backbones (ResNet-18 on ImageNet) to enhance robustness to task-irrelevant variations.

Key Contributions

• Reflection-Consistent Formulation: A novel method to generate mirrored demonstration pairs that are consistent across 2D images, 3D proprioception, and 6-DoF actions, effectively achieving "collect one, get one for free."
• Dual Utility: The framework serves as both a data augmentation strategy (MirrorAug) and a structural prior for equivariant policy learning (MirrorDiffusion), adaptable to standard pipelines like Behavior Cloning (BC) and Diffusion Policies.
• Generalization to Unseen Mirrors: The method enables efficient skill transfer to mirrored workspaces with as few as zero or five demonstrations in the target arrangement, significantly outperforming standard baselines in low-data regimes.
• Robustness to Asymmetry: The paper demonstrates that reflection symmetry can be leveraged even in the presence of significant visual asymmetries (robot design, background textures) when combined with lightweight visual generalization techniques.

Experimental Results

The authors evaluated MirrorDuo on the MimicGen dataset (simulation) and a real-world Franka Emika setup with two tasks: a plush toy pick-and-place (one-side) and block stacking (two-side).

• One-Side Transfer (Simulation & Real):

  • In settings with demonstrations confined to one side of the workspace, standard policies trained on original data failed to generalize to the mirrored side (success rates near 0% for some baselines).
  • MirrorAug combined with visual generalization (Random Overlay + Pretrained weights) enabled policies to achieve near in-domain performance on the mirrored side with as few as 5 additional opposite-side demonstrations.
  • MirrorDiffusion (equivariant) failed at zero-shot transfer due to noise injection breaking global symmetry during the reverse diffusion process but recovered performance with minimal opposite-side data.

• Two-Side Data Efficiency:

  • When demonstrations were distributed across both sides, MirrorDuo significantly improved data efficiency.
  • In low- to medium-data regimes (50–200 demos), MirrorAug and MirrorDiffusion yielded consistent performance gains (e.g., +31% on Square D0) over baselines.
  • In high-data regimes, performance gains plateaued, and in some complex long-horizon tasks, a slight performance drop was observed, attributed to fragmented decision boundaries caused by conflicting observation distributions from near-identical tasks.

• Real-World Validation:

  • On the real-world plush toy task, Diffusion Policy (M, O, P) (MirrorAug + Overlay + Pretrained) achieved 83.3% success on the mirrored setup with only 5 mirrored demos, matching its in-domain performance.
  • Standard Diffusion Policy trained on one-side data achieved only 3.3% on the mirrored setup.
  • The authors note that while MirrorDiffusion is theoretically elegant, the standard Diffusion Policy with MirrorAug and visual generalization techniques proved more practical, offering superior performance under challenging visual conditions and faster training convergence.

Significance and Claims

The paper claims that MirrorDuo establishes reflection symmetry as a powerful and underexplored inductive bias for scalable, generalizable robot learning. By leveraging the overlap between original and mirrored domains, the approach allows robots to generalize to unseen workspace configurations with minimal additional data.

The authors emphasize that while projecting 3D reflections into 2D visual observations introduces complexity (visual asymmetries), these challenges can be effectively managed. The work demonstrates that reflection-consistent learning is highly compatible with image-based pipelines, offering a practical solution to the high cost of collecting diverse robotic demonstrations. The paper concludes that combining reflection symmetry with lightweight visual generalization techniques enables policies to tolerate visual asymmetries from both the robot and the environment, facilitating robust deployment in varied real-world settings.