Impact of Different Failures on a Robot's Perceived Reliability

This study demonstrates that in human-robot interaction, different failure types impact perceived reliability differently—with mistakes being less damaging than slips or lapses—and that trust can be effectively recovered through subsequent successful executions without the need for explicit social repair actions.

The Gist

Imagine you hire a robot butler to help you clean your kitchen. You ask it to pick up a plastic water bottle and throw it in the trash. Sometimes, the robot does a perfect job. But sometimes, things go wrong.

This paper asks a simple but crucial question: Does it matter how the robot messes up? And if it messes up, can it just "try again" to win your trust back, or does it need to apologize?

To find out, the researchers set up a game. They showed people videos of a robot trying to throw away a bottle. After each video, the people had to bet $2 of their own money: "Will the robot succeed next time?" or "Let's just flip a coin."

If they bet on the robot and it failed, they lost money. If they bet on the coin and lost, they also lost money. This betting game was a clever way to measure Perceived Reliability (PR)—basically, how much people actually trusted the robot to do its job, rather than just saying "I trust it" on a survey.

Here is what they discovered, broken down into simple analogies:

1. Not All Mistakes Are Created Equal

The researchers found that people react very differently depending on the type of failure. They categorized the failures into three main groups:

• The "Clumsy Hand" (Slip): The robot knows exactly what to do (throw the bottle in the trash), but its mechanical hand slips, and it pushes the bottle across the table instead of picking it up.
  • Verdict: Huge Trust Drop. This is like a waiter dropping a full plate of food on your lap. It looks like a mechanical failure. People lost a lot of confidence in the robot here.
• The "Frozen Brain" (Lapse): The robot walks up to the bottle, gets ready, and then just... stops. It freezes for 15 seconds, doing nothing, before giving up.
  • Verdict: Big Trust Drop. This is like a driver staring at a red light for a minute without moving. It feels like the robot's "brain" crashed. People lost significant trust here too.
• The "Wrong Choice" (Mistake): The robot's hand works perfectly, and it moves with confidence. But it picks up the wrong object. Maybe it throws a thermos in the trash, or a stapler, or puts a marker in a coffee mug.
  • Verdict: Small Trust Drop (or None!). This is the most surprising finding. If the robot confidently picks up a stapler and throws it away, people were surprisingly forgiving. It's like a chef who confidently cooks a steak instead of the fish you ordered. You might be annoyed, but you still think, "Well, at least they know how to cook!" People often didn't even realize this was a failure; they thought the robot was just doing a different task.

2. The "Second Chance" Superpower

The most important discovery is about recovery.

In the study, some videos showed the robot failing first, but then immediately succeeding on the next try (or succeeding later in the same video).

• The Result: If the robot eventually succeeded, the people's trust bounced back almost instantly.
• The Analogy: Imagine a basketball player misses a shot (failure), but then immediately makes the next three shots (success). The crowd doesn't care about the miss anymore; they are cheering for the streak.
• The Takeaway: The robot did not need to apologize, explain itself, or say "I'm sorry." It just needed to do the job correctly one more time. The act of succeeding erased the memory of the failure.

3. The "Thermos" Confusion

One specific mistake was so funny that it confused the participants. In one video, the robot threw a reusable metal thermos in the trash instead of the plastic bottle.

• The Reaction: 42% of the people watching thought, "Hey, that's a success!" They didn't care that it was the wrong bottle, as long as something got thrown away.
• Why? It seems people care more about the action (throwing something away) than the specific object, as long as the robot is moving with purpose.

The Big Picture

This study teaches us three main lessons for the future of robots in our homes and workplaces:

1. Don't let the robot freeze or drop things. Mechanical failures (slips) and "brain freezes" (lapses) are the worst for trust. These need to be fixed immediately.
2. Confidence matters. If the robot makes a mistake but does it confidently (like picking up the wrong object), people are more forgiving than if it just stops working.
3. Action speaks louder than words. You don't need a robot to say "I'm sorry" to fix trust. You just need it to succeed next time. A single success can wipe out the bad feeling of a previous failure.

In short: If your robot is clumsy, you'll lose faith fast. But if it makes a silly mistake and then gets back on track, you'll likely forgive it and keep betting on it.

Technical Summary

Here is a detailed technical summary of the paper "Impact of Different Failures on a Robot's Perceived Reliability."

1. Problem Statement

As robots are increasingly deployed in human-centric environments (logistics, manufacturing, service), they inevitably encounter failures due to software, hardware, or environmental interaction limitations. While previous research has established that failures erode trust, there is a lack of granular understanding regarding how different types of failures affect a robot's Perceived Reliability (PR). Furthermore, it is unclear whether immediate physical recovery (success following failure) can restore PR without explicit social repair mechanisms (e.g., apologies or explanations).

The authors aim to answer two primary questions:

1. Do different failure taxonomies (slips, mistakes, lapses) impact PR differently?
2. How does a subsequent successful execution affect PR compared to a failure, and does the type of preceding failure matter?

2. Methodology

The study utilized a preregistered, controlled online video experiment involving 326 participants.

• Experimental Design:

  • Task: A pick-and-place task where a robotic arm attempts to move a disposable plastic water bottle into a trash can.
  • Conditions (11 Total): Participants were randomly assigned to view one of 11 video conditions, followed by two additional videos of successful executions. The conditions included:
    • Success: Baseline successful execution.
    • Failure Types (5 variations):
      • Slip: Mechanical execution failure (gripper fails to open, pushes bottle).
      • Lapse: Execution freeze (robot moves then freezes for 15 seconds).
      • Mistake (Thermos): Picks up a thermos and places it in the trash (wrong object, correct location).
      • Mistake (Stapler): Picks up a stapler and places it in the trash (wrong object, correct location).
      • Mistake (Marker): Picks up a marker and places it in a mug (wrong object, wrong location).
    • Failure + Success: Each of the above 5 failure modes immediately followed by a successful execution of the correct task within the same video.
  • Procedure:
    1. Baseline Bet: Before viewing the condition video, participants bet $2 of a $10 compensation pool on either the robot's success or a 50/50 coin flip.
    2. Condition Video: Participants viewed their assigned video.
    3. Post-Video Assessment: Participants answered survey questions regarding the robot's performance and placed a second bet.
    4. Recovery Phase: Participants viewed two additional successful execution videos and placed two more bets.
  • Dependent Variable (Perceived Reliability - PR):
    • Measured via "Signed Confidence."
    • Calculation: (Binary Decision: -1 for Coin, +1 for Robot) $\times$ (Confidence Score: 1–5).
    • Range: -5 (Very confident in coin) to +5 (Very confident in robot).
    • This metric serves as an objective proxy for trust, avoiding the need for anthropomorphism.

• Data Analysis:

  • Statistical tests included One-Way ANOVA to compare failure types and Welch's t-tests to compare failure vs. success conditions.
  • Qualitative analysis used affinity diagramming on open-ended responses to understand participant reasoning.
  • Rigorous filtering was applied to remove participants suspected of using Large Language Models (LLMs) based on writing style and anachronistic references.

3. Key Contributions

1. Taxonomy-Specific Impact: The study provides empirical evidence that not all robot failures are perceived equally. It distinguishes between Slips (mechanical), Lapses (temporal/freeze), and Mistakes (cognitive/planning).
2. Physical Repair vs. Social Repair: The paper demonstrates that physical success (completing the task after a failure) is a highly effective recovery strategy that restores PR to baseline levels, even in the absence of social repair actions (apologies, explanations).
3. Nuanced Perception of Mistakes: It reveals that "mistakes" are not monolithic; the semantic similarity of the error (e.g., wrong object vs. wrong location) significantly influences user perception.
4. Objective Measurement: The use of a monetary betting mechanism provides a rigorous, objective measure of reliability, contrasting with standard self-report Likert scales often used in Human-Robot Interaction (HRI).

4. Key Results

• Failure Severity Hierarchy:
  • Slips caused the largest drop in PR ($\Delta$PR = -3.40).
  • Lapses caused the second-largest drop ($\Delta$PR = -2.46).
  • Mistakes were significantly less damaging.
    • Thermos Mistake: Had almost no negative impact on PR ($\Delta$PR = +0.19); 42% of users even perceived this as a success.
    • Stapler Mistake: Negligible drop ($\Delta$PR = -0.67).
    • Marker Mistake: Moderate drop ($\Delta$PR = -1.68), likely due to the combination of wrong object and wrong location.
• Recovery via Success:
  • When a failure was immediately followed by a success (Failure+Success), the PR recovered to the same level as a direct Success condition.
  • Statistically, there was no difference in PR between "Failure+Success" and "Success" conditions after the recovery videos.
  • 95% of participants bet on the robot after seeing two subsequent successful executions, regardless of the initial failure type.
• User Interpretation:
  • Users often misclassified "cognitive" failures (lapses/mistakes) as "physical" slips, suggesting users conflate "what the robot knows" with "what the robot can do."
  • Users were more forgiving of errors where the robot successfully executed the action (placing an object) even if the object was wrong, provided the goal location was correct.

5. Significance and Implications

• Design of Failure Recovery: The findings suggest that for autonomous manipulation tasks, demonstrating competence through subsequent success is a more effective and efficient repair strategy than social repair (apologies). Robots do not necessarily need to "speak" to regain trust; they need to "act."
• Prioritization of Mitigation: Developers should prioritize fixing Slips and Lapses as these cause the most severe and immediate loss of trust. Mistakes, particularly those involving correct goal placement, may be more tolerable to users.
• Human Expectations: The study highlights a distinction in how humans judge robot ability vs. motivation. Users may interpret a wrong object pick as a planning error (motivation/semantics) rather than a mechanical inability, whereas a freeze (lapse) or slip is seen as a fundamental capability failure.
• Future HRI Strategies: Interaction strategies should focus on ensuring that after a failure, the robot quickly demonstrates a successful execution to reset user expectations, rather than dwelling on the error with verbal explanations.

Conclusion: The paper concludes that while failure types matter initially, successful execution is the ultimate equalizer. As long as the robot eventually completes the task, users are lenient regarding the path taken, provided the failure was not a mechanical inability (slip) or a total loss of agency (lapse).