🤖 AI

Hybrid LLM-based Intelligent Framework for Robot Task Scheduling

This paper proposes a hybrid framework utilizing dual Large Language Model agents (a generator and a supervisor) to optimize construction robot task scheduling by balancing time and resources, adapting to real-time site conditions via natural language interfaces, and demonstrating efficacy through experimental evaluation.

Original authors: Swayamjit Saha, Subhabrata Das, Haonan Duan, Xiao-Yang Liu

Published 2026-05-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Swayamjit Saha, Subhabrata Das, Haonan Duan, Xiao-Yang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a construction site as a busy kitchen, but instead of chefs, you have a team of robots. These robots need to build walls, move bricks, and scan areas, but they can't just guess what to do next. They need a master plan.

This paper introduces a new "brain" for these robots that uses Large Language Models (LLMs)—the same kind of smart AI that powers chatbots—to create and manage these robot schedules.

Here is how their system works, broken down into simple concepts:

1. The Problem: Robots Need a "Manager"

Construction sites are messy and unpredictable. Traditional software is like a rigid recipe book; if you run out of an ingredient or the oven breaks, the book doesn't know how to adapt. The authors wanted a system that could understand natural language (like a human saying, "Build this wall and then check the bricks") and figure out the best way for robots to do it, even if things go wrong.

2. The Solution: A "Writer" and an "Editor"

The authors didn't just use one AI. They built a two-person team to act as the robot's brain:

The Generator (The Writer): This is a powerful AI (GPT-4) that acts like a creative writer. It listens to the human's request and drafts a step-by-step plan for the robots. It's fast and creative, but sometimes it makes mistakes, like forgetting to charge the robot's battery or telling it to walk through a wall.
The Supervisor (The Editor): This is a second AI (using models like Gemma, Llama, or Mistral) that acts like a strict editor. It takes the Writer's draft and checks it against the rules.
- Does the robot have enough battery?
- Can this robot actually lift that brick?
- Did it skip a spot that needs scanning?

If the Editor finds a mistake, it doesn't just say "No." It fixes the plan with the minimum amount of editing possible. It's like a teacher circling a typo and fixing it, rather than rewriting the whole essay.

3. The Analogy: The "Draft and Polish" Process

Think of it like writing a story:

Drafting: The Writer AI quickly scribbles down a story about robots building a wall. It gets the main idea right but might forget that the robot needs to stop at a charging station.
Editing: The Supervisor AI reads the story. It sees the robot is about to run out of power. Instead of throwing the story away, it simply inserts a sentence: "The robot stops at the charger for 5 minutes."
Final Product: The result is a perfect, executable plan that the robots can follow immediately.

4. How They Tested It

The researchers tested this "Writer-Editor" team in two specific scenarios:

The Wall Builder: A robot had to move bricks to build a wall. The Writer often forgot to tell the robot to recharge its battery. The Editor fixed this by inserting a "charge" step, ensuring the robot never ran out of power.
The Scanner: A robot had to scan a grid area. The Writer sometimes missed a corner. The Editor noticed the missing spot and added a single move to scan that specific corner.

5. The Results

The paper found that this two-AI team was much better than using just one AI or old-fashioned rule-based software.

Reliability: The "Editor" successfully fixed almost every mistake the "Writer" made, ensuring the robots could actually finish the job without crashing or running out of battery.
Efficiency: The Editor was very careful. It didn't rewrite the whole plan; it only made small, precise changes (like swapping two steps or adding one charging stop). This kept the plan close to the original idea while making it safe to execute.

In Summary

This paper proposes a smart system where one AI brain dreams up a plan for construction robots, and a second AI brain double-checks and fixes that plan to make sure it's safe and possible. By working together, they create a schedule that is both creative enough to handle complex tasks and strict enough to follow the laws of physics and battery life.

Technical Summary: Hybrid LLM-based Intelligent Framework for Robot Task Scheduling

Problem Statement
The construction industry faces significant challenges in task scheduling and resource allocation due to variable workforce skill levels, dynamic site conditions, and the limitations of traditional software tools. Conventional scheduling methods rely on predefined task descriptions and specialized tools that lack the flexibility to adapt to unprecedented changes or to interpret natural language instructions from construction professionals. While Large Language Models (LLMs) offer potential for natural language interaction and logical reasoning, they often struggle with mathematical precision, constraint satisfaction, and predicting future states based on logical constraints (e.g., battery depletion, precedence rules). Furthermore, existing research has not sufficiently evaluated a hybrid pipeline that combines a generative LLM with a supervisory LLM to benchmark accuracy using standardized metrics in multi-robot construction scenarios.

Methodology
The authors propose a Hybrid LLM-based Framework that utilizes a two-agent architecture to generate and validate robot task schedules. The system operates through a four-stage pipeline:

System Overview & Context Grounding: The system accepts English instructions and site context (maps, robot rosters, API schemas). It canonicalizes entities (locations, robot IDs) to reduce ambiguity.
Stage 1: Generator (Plan Synthesis): A Generator agent (GPT-4) receives a structured, code-like prompt containing the scenario context, few-shot examples, and explicit schema invariants. It produces a preliminary schedule ( $\tilde{\pi}$ ) as a sequence of API-level commands.
Stage 2: Supervisor (Constraint Checking): A Supervisor agent (Gemma 3, LLaMA 4, or Mistral 7B) independently validates the preliminary plan against five classes of constraints:
- Precedence: Ensuring tasks follow the Directed Acyclic Graph (DAG) order.
- Capability/Coalition: Verifying robots possess required skills or forming minimal coalitions for complex tasks.
- Capacity/Resources: Checking payload limits and material counts.
- Battery/Kinematics: Ensuring battery levels remain non-negative and movements are physically realizable.
- Schema/Uniqueness: Confirming command format and preventing duplicates.
Stage 3: Supervised Repair Loop: If violations are detected, the Supervisor performs a "minimal-edit projection," proposing a corrected plan ( $\hat{\pi}$ ) that minimizes the edit distance (reordering, substitution, coalition reassignment) while satisfying all constraints. This loop iterates up to a bounded number of times ( $T=3$ ).
Stage 4: Finalization: The validated schedule is output in an API-ready format for execution by robot middleware.

The framework models the problem as finding a feasible command sequence $\hat{\pi} \in \mathcal{P}$ that satisfies a set of constraints $\Phi$ , where the Supervisor acts as a projection operator mapping an infeasible draft to the nearest feasible plan.

Key Contributions

Hybrid Generator-Supervisor Architecture: The paper introduces a novel pipeline combining GPT-4 as a generator with smaller, specialized LLMs (Gemma 3, LLaMA 4, Mistral 7B) as supervisors to correct hallucinations and logical errors.
Constraint-Aware Scheduling: Unlike traditional rule-based schedulers (e.g., FCFS) or single-pass LLMs, this framework explicitly enforces complex constraints including battery life, coalition formation, and spatial safety.
Standardized Evaluation: The study provides a benchmark for LLM-based task scheduling using both text-form metrics (BLEU, ROUGE, METEOR) and plan-form metrics (Feasibility Rate, Edit Distance, Makespan).
Minimal-Edit Correction Strategy: The research demonstrates that high feasibility can be achieved through localized, minimal edits rather than complete plan regeneration, preserving the original intent of the generator.

Experimental Results
The framework was evaluated on two simulated construction tasks:

Battery-Constrained Wall Assembly: A single robot must build a wall while managing battery life.
- Results: The Generator-only approach frequently violated battery constraints. All hybrid configurations (GPT-4 + Gemma/LLaMA/Mistral) achieved a Feasibility Rate (FR) of 1.0.
- Correction Style: Gemma 3 and LLaMA 4 performed "surgical" corrections (e.g., inserting a single charge step or reordering navigation), resulting in high text similarity (BLEU > 0.87) and minimal makespan overhead. Mistral 7B adopted a more conservative approach, adding redundant charging steps, which lowered BLEU scores but maintained feasibility.
Scan Coverage and Path Feasibility: A robot must scan a grid and return to a start point.
- Results: The Generator often missed coverage of specific cells. All hybrid supervisors achieved 100% Feasible Plan Rate (FPR).
- Correction Style: LLaMA 4 provided the most efficient fix (a single SCAN insertion), achieving the highest BLEU and ROUGE scores. Gemma 3 required a short detour, while Mistral 7B added redundant scans for robustness.

Significance and Claims
The paper claims that the implementation of LLMs is crucial for enhancing the efficiency and flexibility of construction operational tasks involving robots. The primary significance lies in demonstrating that a hybrid, multi-agent approach effectively bridges the gap between the natural language understanding of LLMs and the rigorous constraints required for robotic execution.

The authors assert that their framework:

Overcomes "Myopic" Planning: By separating generation and verification, the system avoids the logical inconsistencies common in single-pass LLM planning.
Ensures Safety and Feasibility: The supervisor agent acts as a reliable guardrail, ensuring that generated plans are executable and safe (e.g., no battery underflow, full coverage).
Maintains Traceability: Because corrections are minimal and local, the final schedules remain close to the generator's original intent, facilitating human inspection and reliable handoff to robot middleware.

The study concludes that the GPT-4 + Gemma-3 pairing currently offers the strongest default configuration, balancing high text-form scores with reliable plan validation. Future work is identified as role-tuning agents and scaling to heterogeneous multi-robot settings with specialized transport and assembly capabilities.