Visual-Tactile Peg-in-Hole Assembly Learning from Peg-out-of-Hole Disassembly

1. Problem Statement

Peg-in-Hole (PiH) assembly is a fundamental yet challenging robotic manipulation task requiring precise alignment, compliant force control, and the ability to recover from jams.

• Challenges: Traditional Reinforcement Learning (RL) approaches require extensive exploration and trial-and-error, leading to high data costs and hardware wear. Supervised Learning (SL) relies on high-quality expert demonstrations which are difficult to collect for PiH without risking damage.
• The Gap: While PiH is difficult, its inverse task, Peg-out-of-Hole (PooH) disassembly, is inherently easier. PooH primarily requires overcoming friction via pulling actions and does not demand the same level of precise initial alignment or complex jamming recovery.
• Core Question: Can the robot learn the difficult PiH assembly task efficiently by leveraging data from the easier PooH disassembly task?

2. Methodology

The authors propose a Visual-Tactile Skill Learning Framework that transfers knowledge from PooH to PiH within a unified Partially Observable Markov Decision Process (POMDP) setting.

A. Unified Environment & Observation Space

Both tasks share the same environment, robot configuration, and observation space, defined as a tuple $(K, V, C)$:

• Kinematics ($K$): Cartesian pose of the end-effector.
• Vision ($V$): Global context images (96x96 pixels) for approach and coarse localization.
• Tactile ($C$): High-resolution contact information derived from vision-based tactile sensors (processed via PCA into 15D vectors) for detecting misalignment and contact forces.

B. The Three-Stage Pipeline

1. PooH Policy Learning:

   • A policy ($\pi_{out}$) is trained using Soft Actor-Critic (SAC) in simulation.
   • The agent learns to extract the peg using multi-modal observations (visual for approach, tactile for contact).
   • This phase is efficient because PooH requires less precise alignment and fewer contact adjustments.

2. PiH Demonstration Generation (The Core Innovation):

   • Temporal Reversal: Successful PooH trajectories are reversed in time to create approximate PiH trajectories.
   • Handling Asymmetry: Simply reversing PooH trajectories fails to capture the complex contact dynamics (jamming, alignment) of PiH because extraction is often contact-free once the peg is released.
   • Action Randomization: To bridge this "tactile gap," the framework introduces action randomization during the reversal process. When the peg is near the hole, random lateral displacements ($\Delta x, \Delta y$) are injected. This forces the robot to simulate diverse contact patterns and jamming scenarios, regenerating realistic tactile feedback for the insertion phase.

3. PiH Policy Learning:

   • The generated PiH demonstrations (reversed PooH + randomization) are used to train the PiH policy ($\pi_{in}$).
   • Hybrid Replay Buffer: The training combines standard RL exploration data with the "expert" reversed trajectories.
   • Behavior Cloning (BC) Loss: An auxiliary loss function guides the policy toward expert actions early in training, which is annealed out as the policy learns to explore autonomously.

3. Key Contributions

1. Inverse Task Learning Framework: A novel approach that formulates both assembly and disassembly as POMDPs, allowing robots to acquire difficult insertion skills from the easier inverse task.
2. Action Randomization for Tactile Symmetry: A specific mechanism to address the asymmetry between insertion and extraction. By randomizing actions during trajectory reversal, the system generates diverse contact configurations that mimic the complex dynamics of PiH, which simple kinematic reversal cannot achieve.
3. Multi-Modal Integration: Demonstrates that combining Vision (for long-range approach) and Tactile sensing (for local alignment and force regulation) significantly outperforms single-modality approaches.
4. Sim-to-Real Transfer: The framework successfully transfers policies from simulation to a real-world ABB YuMi robot equipped with GelSight-like tactile sensors and an Intel RealSense camera.

4. Experimental Results

Experiments were conducted on diverse peg-hole geometries (cubes, cylinders, hexagons, etc.) with varying clearances (0.5mm, 1mm, 2mm).

• Success Rates:
  • Seen Objects: 87.5% success rate.
  • Unseen Objects: 77.1% success rate.
  • Comparison: The proposed method outperforms direct RL (training PiH from scratch) by 18.1% in success rate. It also outperforms pure Supervised Learning and Residual Policy baselines.
• Contact Forces:
  • The visual-tactile policy achieved 6.4% lower contact forces compared to single-modality (vision-only or tactile-only) counterparts.
  • Vision reduced impact forces at first contact; tactile feedback regulated forces during insertion.
• Ablation Studies:
  • Removing Vision caused a drop to ~51% success (due to approach misalignment).
  • Removing Tactile caused a drop to ~42% success (due to inability to correct jamming).
  • Removing Action Randomization reduced success by 14.8%, proving its necessity for generating robust contact data.
  • Removing Hybrid Buffers/BC Loss degraded performance to near Direct RL levels.

5. Significance

This work addresses a critical bottleneck in robotic assembly: the high cost of data collection for complex contact-rich tasks. By leveraging the physical symmetry between assembly and disassembly, the authors demonstrate a highly sample-efficient learning strategy.

• Efficiency: It reduces the need for risky, time-consuming exploration in the difficult PiH domain.
• Robustness: The method generalizes well to unseen objects and geometries, suggesting that the learned "contact intuition" is transferable.
• Practicality: The successful Sim-to-Real transfer on a dual-arm robot with vision-based tactile sensors validates the framework's applicability to real-world industrial and domestic assembly tasks.

In summary, the paper establishes that learning the inverse (disassembly) and intelligently reversing it with action randomization is a powerful paradigm for mastering complex assembly tasks, significantly outperforming traditional "learn from scratch" approaches.