Human-Guided Co-Manipulation of Carbon Fiber Plies

Technical Summary: Human-Guided Co-Manipulation of Carbon Fiber Plies

Problem Statement
The manipulation of flexible, deformable materials, such as carbon fiber plies, presents significant challenges for full automation due to the dynamic distortion and changing states of these objects. While fully manual processes avoid these robotic limitations, they are often ergonomically demanding, tedious, and inefficient, particularly in industries like aviation and automotive manufacturing where large composite sheets must be placed on molds. Human-Robot Collaboration (HRC) and cooperative manipulation (co-manipulation) offer a promising middle ground, allowing human involvement in decision-making while leveraging robotic efficiency. However, effective co-manipulation requires intuitive interaction modalities that do not hinder the operator's hands, which are needed for handling the material. Traditional interfaces like teach pendants are often infeasible, and pre-defined trajectories lack the flexibility required for dynamic environments. Consequently, there is a need to evaluate and compare different control modalities—specifically speech, vision, and force—to determine the most effective strategies for guiding a robot during the transportation and placement of flexible plies.

Methodology
The authors developed a system architecture integrating perception, control, and a Franka Research 3 (FR3) collaborative robot to test various control strategies in a simplified HRC workcell. The setup involved transporting a prepreg carbon fiber ply (with backing film) over an aluminum mold, with experiments conducted both with and without an obstacle to simulate complex trajectories.

The system utilized three primary modalities:

1. Perception: An Intel RealSense D435 camera with a YOLO11n-pose model detected the human operator's wrist to provide 3D coordinates. A Creoir EdgeVUI SDK handled Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) for verbal commands and robot feedback.
2. Control Architecture: A central controller node managed the switching between different control modes and communicated with MoveIt and the robot controllers.
3. Control Methods Tested: Five distinct strategies were evaluated:
   • Compliant Control: Using a Cartesian impedance controller with zero stiffness, allowing the human to pull the ply and the robot to follow.
   • Predefined Trajectory: The robot followed a fixed path while the human adapted.
   • Stepwise Voice Control: The operator used discrete voice commands (e.g., "move forward," "open gripper") to guide the robot in 10cm increments.
   • Wrist Tracking: The robot's impedance reference pose was updated in real-time based on the detected wrist position, allowing the robot to follow the human at a fixed offset.
   • Hybrid Approach: A combination of wrist tracking for gross movement, voice commands for adjustments, and compliant control for final placement.

Key Results
The experiments measured task completion time, path length, and the magnitude of force applied to the robot's end-effector.

• Predefined Trajectory: This was the fastest method (approx. 10–11 seconds) and required the least human effort, but it lacked flexibility and required reprogramming for any environmental changes.
• Compliant Control: This method offered the most intuitive, human-led interaction and was the second fastest (approx. 25–31 seconds). However, it required the operator to exert force to pull the material, which could strain the operator or damage the ply, and made backward movement difficult.
• Stepwise Voice Control: While intuitive and adaptable, this method resulted in the longest execution times (approx. 60–65 seconds) and the longest path lengths due to stop-start, bumpy movements. Accuracy was limited by the operator's ability to estimate distances.
• Wrist Tracking: This method eliminated the need for pulling force and allowed for smoother 3D movement. However, it required a learning curve for the operator and presented challenges in precise placement near the surface.
• Hybrid Approach: By combining wrist tracking, voice, and compliance, this approach achieved the shortest end-effector path (approx. 0.99–1.04 m) and balanced speed with control. It required less force than pure compliance and was faster than voice-only control, though the time spent switching modes slightly increased total duration compared to pure wrist tracking.

Key Contributions
The paper's primary contribution is a comparative analysis of distinct control modalities for the specific task of co-manipulating flexible carbon fiber plies. Unlike previous works that often focus on single modalities or model-based approaches, this study:

1. Evaluates Modalities in Context: It provides empirical data comparing compliance, voice, vision-based tracking, and hybrid methods in a controlled setting with obstacles.
2. Proposes a Multimodal Solution: It argues that a hybrid system combining speech, vision (wrist tracking), and force (compliance) offers the most robust solution. This approach leverages the strengths of each modality: vision for non-contact guidance, speech for discrete state changes, and force for intuitive fine-tuning.
3. Avoids Material Modeling: The proposed methods do not require complex modeling of the carbon fiber ply itself, avoiding the computational overhead and reduced transferability associated with model-based or learning-based methods.

Significance and Claims
The authors modestly claim that this work presents preliminary developments focused specifically on the transportation phase of the layup process. They posit that while no single method is perfect, a multimodal combination allows for a system that is both safe and efficient. The significance lies in demonstrating that vision-based wrist tracking can effectively replace force-based pulling for 3D movement, improving ergonomics and material safety, while speech and compliance provide necessary flexibility and precision. The paper concludes that future work will aim to integrate these isolated methods into a complete industrial scenario, suggesting that the hybrid approach is a viable path toward solving the challenges of flexible material handling in Industry 5.0.