Longitudinal Robot Learning from Demonstration with Care Providers in a Home Environment

Technical Summary: Longitudinal Robot Learning from Demonstration with Care Providers in a Home Environment

Problem Statement
Learning from Demonstration (LfD) allows non-expert users to teach robots novel skills without explicit programming. However, current usability evaluations of LfD with non-experts are predominantly conducted in controlled laboratory settings with the presence of a robotics experimenter. This creates a gap in understanding the barriers non-expert end users face when teaching robots in unstructured, dynamic home environments without live expert feedback. Furthermore, while prior work suggests demonstrators improve over multiple sessions, there is a lack of longitudinal data regarding how non-experts, specifically care providers, adapt to teaching assistive tasks over time in realistic domestic settings.

Methodology
The authors propose a human subjects experiment conducted over three visits in the Georgia Tech Aware Home, an authentic home environment. The study targets a population of care providers (both formal, such as nurses and therapists, and informal, such as family members) with no prior robotics or computer science experience.

• System Architecture:

  • Perception: The system processes RGB-D streams from an Intel RealSense camera using YOLOE for object detection and segmentation. Depth measurements are back-projected into 3D point clouds, and object poses are estimated using Iterative Closest Point (ICP) registration against predefined shapes.
  • Learning Algorithm: The robot utilizes Cartesian Probabilistic Motion Primitives (ProMPs) over end-effector and tracked object poses. ProMPs were selected for their ability to learn from limited demonstrations, rapid training times, and robustness to changing object poses compared to Behavior Cloning or Vision-Language Action models.
  • Interface: An interactive interface allows participants to decompose tasks into low-level skills, demonstrate them, and create skill "recipes."

• Experimental Conditions:
  The study employs a between-subjects design with two conditions:

  1. PT+AF Condition: Participants receive Pre-Training (PT) in Visit 1, where they attempt task decomposition and view videos of optimal decompositions and expert demonstrations. They then receive Adaptive Feedback (AF) in Visits 2 and 3.
  2. AF Condition: Participants skip Visit 1 and receive only AF in Visits 2 and 3.

• Feedback Mechanisms (AF):

  • Foundation Model (FM) Feedback: Provides guidance on task decomposition generalizability. An iterative clarification module helps novices define domains and tasks with sufficient detail.
  • Real Robot Replay (RRR): The robot executes the learned policy in the physical environment for user observation.
  • Augmented Reality (AR) Feedback: The robot executes the policy in AR, allowing users to visualize performance and generalizability without physical risk or time-consuming interactions.

• Procedure:

  • Visit 1: PT condition participants teach tasks in predefined domains (e.g., Tupperware stacking).
  • Visit 2: Participants teach additional tasks in new domains with all three AF types available.
  • Visit 3: Participants define custom domains and tasks with AF available.
  • Metrics: The study evaluates task completion percentage, alignment between predicted and actual robot performance, and user experience metrics (usability, acceptance, learned trust, and workload). Statistical analysis will use multiple linear regressions with mixed effects.

Key Contributions

1. Experimental Design: The paper proposes a human subjects experiment designed to characterize challenges and opportunities for deploying LfD systems with non-expert end users in a home environment, specifically utilizing pre-training and adaptive feedback.
2. Dataset Release: The authors propose to open-source a multi-visit dataset comprising care providers teaching a robot assistive tasks. This dataset includes natural language descriptions, point clouds, end-effector/joint trajectories, audio, and video footage (egocentric and exocentric). It aims to supplement existing datasets like MIME and RoboPro with realistic, multi-visit assistive tasks.

Results and Outcomes
As this paper outlines a proposed study and data collection framework, it does not report final statistical results or specific performance metrics from the completed experiment. Instead, the "Outcomes" section details the intended deliverables:

• The release of the longitudinal dataset to enable standardized benchmarking for interactive robot learning algorithms.
• The reporting of human-factors findings related to the three Research Questions (task performance differences, prediction alignment, and user experience).
• The proposal of design guidelines for researchers and practitioners deploying LfD systems in home environments for assistive tasks, specifically tailored to care providers and multi-visit data collection.

Significance
The paper claims significance by addressing the "ecological validity" of LfD research. By moving from laboratory settings to a realistic home environment and focusing on a longitudinal study with care providers, the work seeks to identify the specific barriers non-experts face when teaching robots without live expert feedback. The resulting dataset and design guidelines aim to facilitate the development of more robust, personalized, and user-friendly assistive robotics that can adapt to unstructured human environments.