Robo-Blocks: Generative Scaffolding in End-User Design and Programming of Social Robots

Technical Summary: Robo-Blocks – Generative Scaffolding in End-User Design and Programming of Social Robots

Problem Statement

Programming social robots is inherently challenging for novice end-users due to the requirement to integrate planning, interaction design, and programming expertise. While Large Language Models (LLMs) offer promise by generating code from natural language, they often obscure critical programming elements and can supplant designer intent, leading to over-reliance rather than skill development. Existing methods, such as block-based programming (e.g., Blockly) or narrative-based design (e.g., bodystorming), address parts of the challenge but fail to bridge the gap between early conceptual planning and executable robot behaviors. There is a lack of tools that effectively scaffold the translation of high-level social intents into programmable actions without automating the process entirely.

Methodology

The authors employed a Research through Design (RtD) approach to develop and evaluate Robo-Blocks, a block-based programming environment that leverages LLMs as "generative scaffolding." The system is built on the Misty II robot platform using React, the Misty II REST API, and OpenAI's GPT-4o.

The Robo-Blocks workflow follows a four-phase structure designed to make interaction flow and context explicit:

1. Narrative Creation: Users co-create a story with an LLM agent describing robot actions, emotions, and context. The interface uses "narrative milestones" (characters, location, time, actions, events, ending, emotions) to structure the input, guiding users to articulate intent before coding.
2. Goal Generation: The LLM translates the narrative into actionable programming goals. Each goal is accompanied by optional hints suggesting specific programming blocks and parameters, making the translation from intent to implementation visible.
3. Programming and Simulation: Users construct programs using a Blockly-based interface. The previously generated goals and hints remain visible to support reflection. A built-in simulator allows users to visualize robot motion, expressions, and speech before deployment.
4. Testing and Deployment: Users connect to a physical Misty robot to test the program. The system encourages iteration, allowing users to revisit earlier phases if the physical execution does not align with the simulation or narrative intent.

User Study:
A study was conducted with 14 novice participants (self-reported beginner-level programming proficiency). The procedure involved:

• Creating a paper storyboard of a social interaction.
• Using Robo-Blocks to co-author a narrative, generate goals, program the robot, and deploy the code.
• Data collection included screen recordings, chat logs, activity logs, final programs, System Usability Scale (SUS) and USE surveys, and semi-structured interviews.

Key Contributions

1. Novel Approach: A narrative-based generative scaffolding method that supports end-users in authoring social robot behaviors by grounding programming in intuitive social narratives.
2. System Design: The development of Robo-Blocks, an environment that instantiates a four-phase process (Narrative → Goals → Programming → Deployment) where LLMs act as cognitive aids rather than autonomous code generators.
3. Empirical Insights: An analysis of how generative scaffolding shapes end-user design and programming strategies, revealing distinct user personas and usage patterns.

Results

The study identified three primary user personas based on their reliance on scaffolding, and a fourth persona emerging from system mismatches:

• Independent Builders: Users who rarely requested help or used hints, maintaining full control over their narrative and code.
• Dependent Users: Users who relied heavily on the LLM for narrative generation and followed hints step-by-step to assemble programs.
• Collaborative Users: Users who engaged in a hybrid approach, modifying narratives based on suggestions and using hints selectively to guide specific coding tasks.
• The Frustrated User: A persona emerging when scaffolding suggestions (e.g., specific robot capabilities) conflicted with the actual available blocks in the programming environment. This mismatch caused users to search for non-existent features, leading to frustration and disengagement.

Quantitative Findings:

• SUS Score: Mean of 73.85 (SD = 14.78), indicating generally positive usability.
• USE Subscales: High scores for Ease of Learning (M = 5.85) and Ease of Use (M = 5.14).
• Usage Patterns: In the narrative phase, 5/14 participants requested help; in the programming phase, 11/14 used hints. However, 4/14 participants reported disagreements with the scaffolding, primarily due to hallucinated capabilities (e.g., suggesting a timer block that did not exist).

Qualitative Findings:
Participants appreciated the narrative phase for clarifying ideas and adding creative details. However, a tension existed between the creative freedom offered by the LLM and the constraints of the physical robot. Some users felt the scaffolding led them astray by suggesting capabilities the robot did not possess, highlighting a "double-edged sword" effect where scaffolding aids creativity but can introduce confusion when not aligned with system constraints.

Significance and Claims

The paper claims that generative scaffolding can effectively support novice robot programmers by bridging the gap between abstract social intents and executable behaviors. The authors argue that:

• Narrative as an Entry Point: Using narratives allows novices to reason about social context and character before tackling technical implementation.
• Visibility of Reasoning: By exposing intermediate representations (goals and hints), the system prevents the "black box" effect of LLMs, allowing users to inspect, revise, and understand the mapping between intent and code.
• Need for Calibration: The study highlights that scaffolding must dynamically calibrate user expectations with actual robot capabilities. Static prompts that hallucinate capabilities undermine trust and engagement.
• Design Implications: Effective scaffolding should not only lower barriers but also actively help users align their creative goals with system constraints, potentially through reflective nudges and adaptive support that shifts based on user needs and task context.

The authors conclude that while their approach successfully demonstrated the potential of narrative-based scaffolding, future work must address the limitations of scripted behaviors and explore more adaptive, non-linear scaffolding strategies for complex, reactive social robots.