Stochastic Decision-Making Framework for Human-Robot Collaboration in Industrial Applications

Technical Summary: Stochastic Decision-Making Framework for Human-Robot Collaboration in Industrial Applications

Problem Statement
The integration of collaborative robots (cobots) into industrial environments has shifted operations from solitary, safety-constrained settings to cooperative ones where robots and humans share workspaces. While existing research effectively utilizes sensors (vision, proximity, EMG) to detect human intentions, significant challenges remain in handling the uncertainty of human factors, specifically emotional states (motivation and aggression) and dynamic behavioral cues. Current safety mechanisms often rely on static musculoskeletal models or simple distance thresholds, which may be insufficient in open, unconstrained environments where human behavior is stochastic. The core problem addressed is the need for a decision-making framework that can anticipate human actions and emotions under uncertainty to ensure both safety and efficiency in Human-Robot Collaboration (HRC).

Methodology
The paper proposes a stochastic decision-making framework based on a Partially Observable Markov Decision Process (POMDP). The approach moves beyond assuming a fixed human model; instead, it learns human models from observed data to handle hidden states such as human emotion and motivation.

1. State Space Formulation: The model defines a comprehensive state space ($S$) comprising five variable types:

   • Task Assignment ($\tau$): Includes priority, duration, nature (robot-only, human-only, or collaborative), and commitment status.
   • Distance ($d$): Modeled using a novel "tri-circle" safety approach. The circles represent: (1) the human's working space, (2) the human with fully stretched arms (aggressive act), and (3) a human falling over (accident). The robot dynamically selects the appropriate safety circle based on context.
   • Human Emotion ($e$): Includes motivation level and aggression level, which are hidden states inferred from observations.
   • Human Activity ($h$) and Robot Activity ($b$): Representing current priority and progress of tasks for both agents.

2. Observation and Inference: Since emotional states are not directly observable, the system utilizes an observation space ($O$) consisting of human body position, joint angles, and angular velocities (e.g., from visual sensors). The POMDP framework uses these observations to update a belief state (probability distribution over hidden states like motivation and aggression).

3. Decision Variables: The robot's action set ($D$) includes:

   • Normal work mode ($nw$).
   • Requesting the human to work alone ($rh1$) or collaboratively ($rh2$).
   • Motivating the human ($mh$).
   • Committing to a task ($ct$).
   • Adjusting distance ($dp$ to increase, $dm$ to decrease).
   • Do nothing/idle mode ($dn$).

4. Cost Function: The objective function ($J$) balances three goals: avoiding collision (maintaining safe distance based on aggression levels), completing tasks efficiently, and respecting human emotional states. The cost function penalizes unsafe distances, unassigned tasks, low motivation, and incorrect task assignments.

5. Solution Approach: The authors employ Value Iteration to solve the underlying MDP and derive an optimal policy. To handle partial observability, a heuristic action selection method is used where the robot aggregates weights from the belief state distribution to choose the action with the highest cumulative probability of being optimal.

Key Contributions

• Tri-Circle Safety Model: A novel safety decision-making approach that dynamically adjusts the robot's safety zone based on three distinct human scenarios (working, stretching, falling), rather than relying on a single static safety radius.
• Emotion-Aware HRC: The integration of human emotional states (motivation and aggression) into the decision-making loop, allowing the robot to adapt its behavior (e.g., increasing distance if aggression is high, motivating the human if engagement is low).
• Bilateral Intent Prediction: Unlike prior works focusing solely on detecting human intent, this framework utilizes a POMDP to reason about the bilateral interaction, treating human decisions as exogenous events while the robot optimizes its own policy to facilitate safe and smooth collaboration.
• Unified Framework: The combination of task management, safety constraints, and emotional reasoning within a single stochastic control framework.

Results
The proposed framework was validated through a numerical case study involving a simulated industrial scenario with a human and a robot. The simulation introduced three sequential tasks with varying priorities, durations, and collaboration requirements (collaborative, human-only, and robot-only).

• Task Execution: The system successfully identified optimal policies to execute tasks in the shortest time while maintaining safety.
• Dynamic Adaptation: The results demonstrated the robot's ability to adjust its behavior based on the task nature. For collaborative tasks, the robot maintained a close distance (small safety circle) and motivated the human. For human-only tasks, the robot increased the distance and motivated the human before assigning the task.
• Safety Management: The simulation showed the robot effectively managing the "tri-circle" safety zones, increasing distance when the human was not engaged in a collaborative task and reducing it when collaboration was required.
• Belief State Updates: The system successfully inferred hidden emotional states (motivation and aggression) based on observed human velocities, updating the belief state to guide decision-making.

Significance and Claims
The paper claims that its primary significance lies in addressing the gap in current HRC research, which often focuses on intent detection without adequately accounting for the stochastic nature of human emotions and the dynamic requirements of safety. By utilizing a POMDP, the framework offers a generalized approach to handle uncertainties arising from concealed human actions and imperfect sensor data. The authors assert that this approach enhances the safety and transparency of collaborative robots, enabling them to anticipate human actions and adapt behavior in real-time. The study concludes that incorporating both robot states and uncertain human emotional states into a unified hierarchical structure provides a comprehensive solution for dynamic human-robot interactions, moving beyond simple collision avoidance to proactive, emotion-aware collaboration.