Technical Summary: TACTFUL

Problem Statement

Robotic manipulation has traditionally relied on vision for object localization and identification, often treating tactile sensing as a secondary or corrective modality. However, in visually occluded or confined environments (e.g., warehouse bins, cluttered bags), vision is unreliable or unavailable. While humans can effortlessly locate and identify objects using touch and geometric priors, robotic systems struggle with autonomous tactile exploration. Existing tactile approaches often rely on simplified contact signals, simulation-based training, or hand-crafted heuristics, with limited demonstration of high-fidelity performance on real-world hardware for the combined task of discovering objects in an unknown workspace and reconstructing their geometry for identification.

Methodology

The paper presents TACTFUL, a vision-free framework enabling a multi-fingered robot to autonomously explore confined workspaces, discover objects via contact, and identify them through tactile reconstruction. The system operates entirely on real hardware without simulation pre-training.

1. Hardware and Setup

The system utilizes a five-fingered Inspire RH56DFTP Series Dexterous hand mounted on a UR10e robotic arm. The hand is equipped with 1,062 resistive taxels measuring normal contact forces. The workspace is a bin with known bounds containing 3D-printed objects placed at arbitrary locations. No cameras are used.

2. Learning Framework (POMDP)

The task is formulated as a Partially Observable Markov Decision Process (POMDP). The agent must infer object locations and shapes solely from accumulated tactile feedback and proprioception.

• Observation Space ($O$): Includes proprioceptive data (end-effector pose, joint angles, hand configuration) and high-resolution tactile readings. Crucially, it incorporates a Tactile Point Cloud embedding ($c_t$) generated by Sonata, a self-supervised transformer-based encoder that compresses the history of tactile contacts into a representation of spatial coverage and geometric structure.
• Action Space ($A$): Defined as incremental changes to the end-effector pose ($\Delta p_t$) and finger joint angles ($\Delta h_t$).
• State Representation: To handle temporal dependencies, the state consists of a sequence of the last $k=10$ observations, processed by a Transformer architecture with self-attention mechanisms.

3. Training Strategy: BC Initialization + PPO

Learning tactile policies directly via Reinforcement Learning (RL) in the real world is inefficient and risky. TACTFUL employs a two-stage training pipeline:

1. Behavior Cloning (BC) Initialization: A policy is pre-trained offline on 5,311 tactile-only teleoperated demonstrations. The BC model learns to regress expert actions (delta pose and configuration) from the observation state, capturing basic contact strategies and local exploration patterns.
2. Online PPO Training: The BC policy initializes the actor in a Proximal Policy Optimization (PPO) framework. The policy is refined online on real hardware.
   • Critic Network: Receives "privileged information" (approximate 6-DoF pose of the target object) to stabilize value estimation, which is not available to the actor during deployment.
   • Actor Network: Uses the same Transformer architecture as the critic but relies only on observable tactile and proprioceptive data.

4. Dynamic Reward Schedule

A composite reward function guides the agent through distinct phases of exploration and refinement, with weights evolving over time ($t$) within an episode horizon ($T$):

• Contact Reward ($r_{contact}$): Incentivizes novel, spatially distinct contact points (new taxel activations > 5mm from previous contacts) to encourage high-resolution surface coverage.
• Exploration Reward ($r_{explore}$): Encourages the agent to visit new regions of the workspace (tracked via a 1cm$^3$ voxel grid) to prevent local minima and ensure global discovery.
• Reconstruction Reward ($r_{recon}$): Penalizes inaccurate shape representations by measuring the Chamfer distance between the current dense reconstruction and the ground truth.
• Schedule: Early in the episode, $r_{explore}$ is weighted heavily to facilitate object discovery. As the episode progresses, $r_{contact}$ and $r_{recon}$ weights increase to prioritize surface refinement and geometric accuracy.

5. Shape Completion

Sparse tactile point clouds are passed to a pretrained point cloud diffusion model to generate dense, complete object geometries. This model acts as a learned prior over the known object library, bridging the gap between sparse tactile data and full 3D shapes for identification.

Key Contributions

1. Learned Tactile Exploration Policy: Development of a single policy enabling a multi-fingered robot to autonomously explore confined workspaces and localize objects using tactile sensing alone, without vision.
2. Dynamic Reward Schedule: Introduction of a time-varying reward mechanism that balances global workspace exploration with local contact refinement, leveraging object-level geometric priors to guide 3D reconstruction.
3. Real-World Validation: Demonstration of reliable object identification based solely on reconstructed tactile geometry, trained entirely on real hardware without simulation.

Experimental Results

The framework was evaluated on three geometrically diverse real-world objects (a cube, a deformed cylinder, and a deformed cup) placed in a bin. The robot was tasked with locating a specific target among distractors.

• Performance: The TACTFUL method achieved a 77% success rate (averaged across objects) with an average Chamfer-L2 reconstruction error of 0.015 m.
• Comparisons:
  • Outperformed a Heuristic baseline (0.416–0.5 success rate) which lacked learning from contact feedback.
  • Outperformed BC-only (0.416–0.583 success rate), demonstrating the value of online RL adaptation.
  • Outperformed RL without BC initialization (0.583–0.75 success rate), confirming that BC initialization provides a crucial structured prior for safe and efficient real-world learning.
• Ablation Studies:
  • Removing any single reward component ($r_{contact}$, $r_{explore}$, or $r_{recon}$) significantly degraded performance, confirming the necessity of balancing exploration, contact quality, and reconstruction accuracy.
  • Removing the shape completion model reduced success rates to 34%, highlighting its role in resolving geometric ambiguities inherent in sparse tactile data.

Significance and Claims

The paper claims that TACTFUL demonstrates tactile sensing can serve as an effective primary modality for object-level reasoning in unstructured, occluded, or confined environments, rather than merely a reactive fallback for vision. By combining structured learning (BC), dynamic reward shaping, and shape completion priors, the system achieves robust object localization and identification without visual input. The authors note that while the current work is limited to a known library of three objects and static scenes, the results validate the potential for autonomous tactile reasoning in real-world settings where vision is denied.