Advancing Improvisation in Human-Robot Construction Collaboration: Taxonomy and Research Roadmap

Technical Summary: Advancing Improvisation in Human-Robot Construction Collaboration

Problem Statement
The construction industry faces persistent challenges regarding productivity stagnation, skilled labor shortages, and safety hazards, evidenced by high fatality rates and frequent project delays. While robotics and automation offer potential solutions, current construction robots struggle in unstructured, dynamic environments. Traditional preprogrammed automation lacks the flexibility to handle unexpected site conditions, often requiring extensive human supervision. Although Human-Robot Collaboration (HRC) has emerged as a paradigm to leverage complementary human and robotic strengths, a critical gap remains: the ability to engage in collaborative improvisation. Improvisation—the capacity to adapt to unexpected situations through creative problem-solving and real-time decision-making—remains predominantly a human trait. Current research lacks a systematic framework to classify the progression from basic human-controlled execution to true collaborative improvisation, leaving unclear the technical and conceptual pathways required to bridge this gap.

Methodology
The authors conducted a systematic review of 214 articles published between 2010 and 2025, sourced from databases including Google Scholar, IEEE Xplore, Scopus, and ScienceDirect, as well as key journals and conference proceedings (e.g., ISARC, CRC). The search strategy utilized Boolean combinations of keywords related to construction robotics, HRC, improvisation, adaptation, and learning.

The core methodological contribution is the development of a six-level taxonomy for classifying HRC in construction, adapted from Liang et al.'s (2021) framework and the Level of Robot Autonomy (LoRA). This taxonomy categorizes research based on the distribution of improvisation capabilities and responsibilities between humans and robots. The levels range from:

• Level 0: Manual Work
• Level 1: Human-Controlled Execution
• Level 2: Adaptive Manipulation
• Level 3: Imitation Learning (Learning from Demonstration)
• Level 4: Human-in-Loop BIM Workflow
• Level 5: Cloud-Based Knowledge Integration
• Level 6: True Collaborative Improvisation

To visualize the evolution of capabilities across these levels, the authors developed a five-dimensional radar framework assessing:

1. Planning/Organization: Task sequencing and resource scheduling.
2. Cognitive Role: Real-time decision-making and reasoning.
3. Physical Execution: Direct manipulation and force control.
4. Learning Capability: Skill acquisition and generalization.
5. Improvisation/Adaptability: Creative problem-solving for novel situations.

Representative research contributions were mapped onto "spoke-maps" for each level to illustrate the current state of the art and identify research frontiers.

Key Contributions and Results
The analysis reveals that current construction robotics research is heavily concentrated at the lower levels of the taxonomy (Levels 1–3), with limited progression toward true collaborative improvisation (Level 6).

• Level 1 (Human-Controlled Execution): Robots act as obedient executors with zero improvisation. Humans retain full planning and cognitive control, while robots provide high physical capability. Research focuses on teleoperation and preprogrammed execution.
• Level 2 (Adaptive Manipulation): Robots introduce sensor-driven micro-adjustments (e.g., force modulation, collision avoidance) within predefined parameters. Improvisation is limited to reactive adaptation within narrow envelopes; creative reasoning remains human.
• Level 3 (Imitation Learning): Robots learn policies from human demonstrations (kinesthetic or visual). They can generalize within known task families but lack the ability to generate novel strategies for unseen scenarios. Improvisation is retrospective and example-driven.
• Level 4 (Human-in-Loop BIM): Robots integrate Building Information Modeling (BIM) and Digital Twins to detect deviations between design and as-built conditions. They propose corrective actions based on semantic models, but humans retain authority for validation and high-level planning.
• Level 5 (Cloud-Based Knowledge): Robots access cloud repositories and Large Language Models (LLMs) to retrieve analogous precedents and generate candidate plans. This represents a shift to "consultative autonomy," where robots can handle a wider range of unexpected situations by recombining known elements, though they do not yet engage in co-creative ideation.
• Level 6 (True Collaborative Improvisation): Identified as the aspirational target state, this level envisions a partnership where humans and robots engage in continuous, co-creative problem framing and execution. It requires shared mental models, dialogic reasoning, and generative AI capable of proposing genuinely novel strategies grounded in physical constraints.

The radar diagram analysis demonstrates that while robots are increasingly surpassing humans in physical execution and task-level cognitive reasoning at higher levels, true collaborative improvisation requires intentional asymmetries. Humans must retain primacy in high-level planning, contextual learning, and ethical judgment, while robots augment these with computational power and generative synthesis.

Significance and Claims
The paper claims that its primary significance lies in establishing a systematic classification framework that moves beyond physical capabilities to focus on the critical, yet under-researched, dimension of improvisation. By defining the six levels and the five capability dimensions, the framework provides a clear roadmap for researchers and practitioners to position current work and identify specific gaps.

The authors identify three fundamental barriers preventing the transition to Level 6:

1. Technical Limitations: Insufficient grounding of LLM-generated plans in real-time perceptual affordances and a reliance on sequential query-response interactions rather than sustained dialogic co-reasoning.
2. Conceptual Gaps: A disconnect between human improvisation research (focusing on dyadic collaboration and tacit knowledge) and robotics research (focusing on individual robot capabilities).
3. Methodological Challenges: The need for integrative breakthroughs in shared situational awareness interfaces, explainable architectures, and trust calibration mechanisms.

The paper concludes that future research must prioritize the development of AR/VR interfaces for shared mental models, the integration of LLMs with robust grounding in construction constraints, and cloud-based systems that support dialogic co-reasoning. The ultimate goal is to evolve robots from high-performance assistants into creative partners capable of addressing construction unpredictability through co-generated innovation, while preserving essential human roles in ethical and strategic decision-making.