Technical Summary: Adaptive Robotic Failure Allocation (ARFA)

Problem Statement

Despite advancements in autonomy, mobile manipulators and humanoid assistive robots frequently encounter failures in dynamic, unstructured environments due to hardware malfunctions, software errors, or interaction uncertainties. While some failures can be resolved autonomously, many require human intervention. Current human-robot collaboration systems often lack dynamic, real-time strategies for allocating these failures to human operators. Existing methods typically rely on static capability assessments or offline allocation, failing to account for the time-sensitive nature of failures, the specific skills of remote versus local operators, and the current workload distribution. Without effective allocation, robots may experience prolonged idle times, system performance degrades, and operator workloads become unbalanced, potentially disrupting primary tasks.

Methodology

The authors propose Adaptive Robotic Failure Allocation (ARFA), a framework designed to dynamically assign robotic failures to the most competent human operator (either local or remote) while balancing workload. The methodology consists of three core components:

1. Modeling Human Capabilities

The system models each operator $i$ across three capability dimensions:

• Physical ($phys$): Ability to perform physical tasks (e.g., moving cables).
• Cognitive ($cog$): Ability to process information and make decisions.
• Responsiveness ($resp$): Ability to respond quickly to urgent situations.

Capabilities are represented as a belief distribution with lower ($\ell$) and upper ($u$) bounds. For a given failure $f$ with requirements $r = [r_{phys}, r_{cog}, r_{urg}]$, a capability score $\lambda_{i,j}(r_j)$ is calculated. This score is 1 if the operator's capability exceeds the requirement, 0 if it falls below the lower bound, and linearly decreases in between. The overall performance index $\Lambda_i$ is a weighted sum of these scores.

2. Expected Reward and Failure Assignment

A reward function $R_i$ evaluates the assignment of a failure to an operator based on:

• Failure Urgency ($r_{urg}$): Higher urgency amplifies the impact of historical performance differences.
• Historical Performance ($\tau_i$): Derived from the ratio of resolution times against a task-dependent time threshold.
• Workload Cost ($C_i$): A penalty based on the proportion of total resolution time already assigned to the operator.

The total expected reward ($ER_i$) is calculated as:
$$ER_i = \Lambda_i \cdot (R_i(r_{urg}, \tau_i) - C_i)$$
The failure is assigned to the operator $i^*$ that maximizes this expected reward.

3. Capabilities Optimization (Online Learning)

The system continuously updates the belief bounds ($\ell, u$) for each operator's capabilities using an Adam optimizer. The objective is to minimize the mean squared error between the predicted success probability (based on current capability bounds) and the observed success probability (empirical data from resolved failures). This allows the system to adapt to an operator's actual performance in real-time.

Key Contributions

The paper outlines three primary contributions:

1. Framework Development: A system enabling human intervention for robotic failure recovery involving both local and remote operators.
2. Adaptive Strategy: An adaptation of task allocation strategies to the specific domain of failure recovery, introducing a reward function that minimizes operator workload and task disruption while considering urgency and historical performance.
3. Validation: Empirical validation through simulations and a user study involving real users interacting with a mobile nursing assistant robot (IONA).

Evaluation and Results

Simulation

The authors simulated 100 failures across 20 trials with one remote and one local operator.

• Success Rates: ARFA significantly outperformed random allocation. The remote operator's success rate improved from 55% (random) to 82% (ARFA). The team success rate increased from 78% to 95%.
• Convergence: The capability belief optimizer converged successfully in 98.33% of trials, typically within the first 20 failures.

User Study

A study with 20 participants (paired as local and remote operators) using the IONA robot and augmented reality interfaces (HoloLens 2 and screen-based GUI) compared ARFA against random allocation.

• System Performance (RQ1):
  • Resolution Time: ARFA significantly reduced the mean failure resolution time from 312.73s (random) to 257.6s.
  • Task Completion: Overall task completion time decreased from 632.98s to 571.85s.
  • Success Rate: Team success rate (resolving within time thresholds) increased from 69% to 85%. Remote operator success rates improved significantly (40.5% to 55%), while local operators maintained near-perfect performance in both conditions.
• Workload Distribution (RQ2):
  • ARFA significantly reduced the disparity in recovery time between operators, narrowing the difference from 132.30s (random) to 33.64s.
  • Primary task performance (Sudoku solving) remained consistent between both allocation modes, indicating that workload balancing did not compromise the operators' main duties.

Significance and Claims

The paper claims that ARFA bridges the gap in real-time robotic failure recovery by moving beyond static or random assignment. By dynamically modeling operator capabilities and continuously updating beliefs based on observed outcomes, the framework:

• Reduces Robot Idle Time: Faster resolution of failures leads to more efficient system operation.
• Optimizes Human-Robot Collaboration: Ensures failures are assigned to the operator best suited for the specific requirements (physical, cognitive, or urgency).
• Balances Workload: Prevents overburdening specific operators, particularly local ones who may face frequent interruptions, thereby maintaining a sustainable human-robot ecosystem.

The authors acknowledge limitations, noting that the framework relies on simplified models of human capabilities (not fully capturing fatigue or nonlinear cognitive loads) and assumes failures belong to a predefined set. Future work aims to address uncontrolled scenarios with more failure types and evaluate the framework with larger teams.