Technical Summary: Enhancing Physical Human-Robot Interaction via Intrinsic Robot Tactile Sensing

Problem Statement

Physical Human-Robot Interaction (pHRI) aims to facilitate natural and safe collaboration between humans and robots. However, current approaches often rely on external tactile sensors (e.g., electronic skin or force/torque sensors mounted on the flange) to detect and classify touch. These external additions increase system complexity, cost, and mechanical bulk, while often lacking the durability and flexibility required for seamless integration. Furthermore, existing collaborative robots already possess intrinsic sensing capabilities—specifically joint torque sensors—that are typically underutilized for tasks beyond simple contact detection or collision avoidance. This study addresses the question of whether these intrinsic sensors are sufficient to enable rich, intuitive interactions, such as recognizing handwritten digits drawn by a user, without the need for specialized external hardware.

Methodology

System Architecture and Data Collection

The proposed system utilizes a seven-degree-of-freedom collaborative robot (KUKA LBR iiwa 14 R820) equipped with torque sensors at the link side of each joint. Users interact with an uninstrumented touchpad (a plexiglass square mounted on the robot's flange) by sliding their fingers to draw digits (0–9).

• Sensors: The system captures intrinsic joint torque signals ($\tau_{ext}$) and calculates End-Effector Forces ($F_{ext}$) and Moments ($M_{ext}$) via the Jacobian.
• Preprocessing: To address signal offsets caused by varying robot poses and touch pressures, a sliding window technique is employed for real-time baseline correction.
• Segmentation: Touch events are isolated using a threshold-based approach on joint 2 torque or z-axis force. Segmented signals are resampled to a fixed length of 100 samples to ensure consistency despite variations in drawing speed and communication latency.

Data Augmentation Strategy

To enhance robustness against variations in user behavior (reversed writing direction and rotated orientations), a data augmentation technique was developed:

• Reversed Digits: The geometric path is reversed using a time-reversal operator ($T$), and the signs of specific force and moment components are inverted to account for the change in finger pressure distribution and direction.
• Rotated Digits: Forces and moments are transformed using a rotation matrix $R_z(\theta)$ corresponding to the rotation angle of the digit on the touchpad.

Classification Model

A Bidirectional Long Short-Term Memory (Bi-LSTM) network is employed to classify the time-series data. The bidirectional architecture allows the network to capture temporal dependencies from both preceding and subsequent movements, which is critical for recognizing the shape of a digit based on the full trajectory of the touch.

Experimental Design

Experiments were conducted to evaluate:

1. Baseline Performance: Comparing single-user vs. multi-user training datasets.
2. Generalizability: Testing the model on users who did not participate in the training phase.
3. Pose Robustness: Evaluating performance across 10 randomly selected robot poses.
4. Augmentation Efficacy: Assessing the system's ability to classify reversed and rotated digits.
5. Real-world Application: Implementing a fruit delivery task where users select items by drawing corresponding digits.

Key Results

Classification Accuracy

• Standard Digits: The Bi-LSTM network achieved an offline accuracy near 100% and an online accuracy of 94.1% (multi-user training) when tested on users not involved in training, even with the robot in variable poses.
• Generalizability: A single-user trained model showed significantly lower accuracy (45.2%) on unseen users compared to the multi-user model (94.1%), highlighting the necessity of diverse training data.
• Augmented Inputs:
  • Reversed Digits: Achieved 92% accuracy. The primary error source was misclassifying '8' as '3'.
  • Rotated Digits: Achieved approximately 80–86% accuracy depending on the rotation angle (+90° or -90°). Common errors occurred with digits '5', '1', and '7'.
  • The system demonstrated that even a model trained only on standard digits could recognize slightly rotated inputs (<45°), though performance degraded with larger rotations.

Confusion Analysis

The confusion matrix indicated that most classification errors occurred between visually or structurally similar digits, specifically '0' vs. '6', '1' vs. '7', and '2' vs. '3'.

Real-World Demonstration

The methodology was validated in a fruit delivery scenario. The system utilized a Hierarchical Finite State Machine (HFSM) to manage task execution and safety. The robot provided spoken feedback (e.g., "I see that you want an orange") and required touch confirmation before executing the task. Safety protocols included stopping motion upon detecting contact during task execution and resuming via a double-tap gesture.

Significance and Contributions

The paper claims the following contributions to the field of pHRI:

1. Intrinsic Sensing Viability: It demonstrates that intrinsic joint torque sensing in commercial collaborative robots is sufficient for high-accuracy digit classification, eliminating the need for external tactile sensors.
2. Public Dataset (pHRI-DIGI-TACT): The release of a dataset containing 10 classes of tactile handwritten digits collected from multiple participants, including 7-DOF joint torques and EEF forces/moments.
3. Robustness to Variations: The development of a data augmentation method that enables the recognition of reversed and rotated digits without requiring additional data collection, thereby increasing system flexibility.
4. Practical Application: A successful demonstration of an intuitive interface for everyday tasks (fruit delivery), showing potential for assisting inexperienced users through natural physical interaction and voice feedback.

The authors conclude that this approach offers a promising alternative to traditional pHRI methods, reducing system complexity and cost while maintaining high interaction quality. Future work is identified as focusing on expanding the system to recognize multi-digit numbers and digits drawn with pauses or different geometric paths.