💻 computer science

HECTOR: Human-centric Hierarchical Coordination and Supervision of Robotic Fleets under Continual Temporal Tasks

This paper proposes HECTOR, a human-centric hierarchical framework that enables efficient supervision and dynamic coordination of large-scale robotic fleets through a three-layer architecture designed to handle continual, uncertain temporal tasks via flexible human-fleet interaction.

Original authors: Shen Wang, Yinhang Luo, Jie Li, Meng Guo

Published 2026-05-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shen Wang, Yinhang Luo, Jie Li, Meng Guo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the conductor of a massive orchestra, but instead of violins and flutes, you are directing 80 different robots. Some robots are like delivery trucks, some are like drones, and others are like robotic arms. They need to work together to perform complex tasks like delivering packages, patrolling an area, or rescuing people.

The problem is that if you tried to tell every single robot exactly what to do, you would be overwhelmed. If you tried to program them all at once without any human help, they might get confused when the situation changes (like a new emergency popping up).

This paper introduces HECTOR, a smart system designed to be the "middle manager" between you (the human operator) and the robot fleet. Think of HECTOR as a super-organized team captain that translates your big-picture orders into specific instructions for the robots, while also letting you step in and change the plan whenever you need to.

Here is how HECTOR works, broken down into three simple layers:

1. The "Two-Way Radio" (Human Interaction)

In many old systems, you either programmed the robots before they started (and couldn't change anything), or you had to control them one by one. HECTOR is different. It uses a special "protocol" (a set of rules for talking) that lets you speak to the whole fleet at once.

What you can do: You can say, "Start a new mission," "Cancel that one," "Make this task more urgent," or even "Take these specific robots and give them to that team."
The Interface: Imagine a dashboard with a map. You can click, type, or even use your voice to give orders. The system instantly shows you where the robots are, what they are doing, and when they will finish. It's like a live video game where you can pause, change the rules, and watch the characters adapt instantly.

2. The "Team Builder" (Assigning Tasks)

Once you give an order, HECTOR doesn't just pick robots randomly. It acts like a smart coach forming teams for a sports tournament.

The Challenge: Some tasks need a heavy-lifter robot, while others need a fast drone. Some tasks need 3 robots working together, while others need 10.
The Solution: HECTOR looks at all the available robots and the new tasks. It figures out the best way to group them into "squads." It asks: "Who has the right skills? Who is closest? Who isn't busy?"
The "Rolling Horizon": Instead of planning the entire day for every robot at once (which is too slow and rigid), HECTOR plans the next few steps, lets the robots do them, and then immediately plans the next batch. It's like driving a car: you don't plan the route for the whole trip while sitting in your driveway; you look a few miles ahead, drive, and then look a few miles further. This keeps the system fast and ready for changes.

3. The "Local Coach" (Robots Working Together)

Once a squad is formed, the robots have to work together on the ground. HECTOR gives them different strategies depending on the situation:

Scenario A (Known Map): If the robots know exactly where the targets are (like delivering to 10 known houses), they calculate the fastest route to visit them all without crashing into each other.
Scenario B (Search and Rescue): If the robots are looking for something they don't know the location of (like finding survivors in a rubble field), they split up to cover the area. As soon as they find a "subtask" (a survivor), they instantly assign the nearest robot to help, while the others keep searching.
Scenario C (Moving Targets): If the targets are moving (like chasing a moving vehicle), the robots constantly talk to each other to switch who is chasing whom, ensuring they never lose the target.

Why is this a big deal?

The paper tested this system with 80 robots and 30 complex missions involving hundreds of tiny sub-tasks.

Speed: It was much faster than other methods. While other systems took minutes to re-plan when things changed, HECTOR did it in less than a second.
Reliability: It successfully completed 100% of the missions, even when the researchers simulated robots breaking down or getting lost.
Human Control: It proved that a human can effectively manage a huge fleet by giving high-level orders (like "Prioritize this!") without getting bogged down in the details of every robot's movement.

In summary: HECTOR is a system that lets a human boss a large, diverse team of robots. It handles the boring math of "who does what," forms the best teams on the fly, and lets the human step in to change the game plan instantly, making it perfect for chaotic situations like disaster relief or large-scale deliveries.

Technical Summary: HECTOR – Human-centric Hierarchical Coordination and Supervision of Robotic Fleets

1. Problem Statement

The paper addresses the challenge of coordinating large-scale, heterogeneous robotic fleets (e.g., UAVs and UGVs) in dynamic, partially unknown environments under continual temporal tasks. While robotic fleets offer efficiency through collaboration, direct human control of individual units is impractical. Existing approaches often fail to address three critical gaps simultaneously:

Dynamic Online Interaction: Most methods assume static, pre-defined task sets, neglecting the need for operators to add, cancel, or reprioritize missions in real-time.
Scalability under Uncertainty: Traditional centralized planning (e.g., MILP) suffers from combinatorial explosion, while decentralized methods often lack guarantees for complex temporal constraints or fail to handle uncertain subtask counts and locations.
Human-in-the-Loop Integration: Current frameworks often treat human interaction as an afterthought or restrict it to small teams, lacking protocols for high-level supervision and intervention in large-scale operations.

The core objective is to synthesize collective plans that minimize average mission response time while satisfying Linear Temporal Logic (LTL) specifications, capacity constraints, and dynamic operator requests (new missions, cancellations, priority changes, and manual reassignments).

2. Methodology: The HECTOR Framework

The proposed solution, HECTOR, employs a three-layer hierarchical architecture to decouple global coordination from local execution, enabling scalability and robust human interaction.

Layer I: Human-Fleet Interaction Protocol

This layer establishes a bidirectional, multimodal interface (text, voice, templates) allowing operators to issue four types of online requests:

$\kappa_1$ (New Mission): Releases new sc-LTL (syntactically co-safe LTL) specifications with uncertain subtask counts and locations.
$\kappa_2$ (Cancellation): Removes incomplete missions without disrupting ongoing tasks.
$\kappa_3$ (Modification): Updates deadlines and priorities of existing missions.
$\kappa_4$ (Reassignment): Manually assigns specific robots to specific missions.
The protocol maps these requests to a task decomposition module and a visualization interface, ensuring closed-loop feedback on fleet status and plan feasibility.

Layer II: Receding-Horizon Task Assignment and Team Formation

This layer handles the global allocation of missions to teams using an automata-guided search tree rather than a full synchronous product of robot models and task automata (which is computationally prohibitive).

Simultaneous Decomposition and Assignment: The algorithm searches a tree where nodes represent partial team plans and the current reachable states of mission automata. It simultaneously determines the optimal number of teams ( $K$ ), their composition, and the sequence of tasks.
Value Function & Bounding: A heuristic value function $\chi(\nu)$ balances makespan, cost, and mission progress. Nodes are pruned using a dominance relation based on a performance profile $\zeta(\nu)$ , ensuring the search focuses on non-dominated frontiers.
Receding Horizon: To handle dynamic updates, the planner operates with a finite horizon $H$ . It pauses when $H$ tasks are assigned and resumes upon replanning triggers (e.g., task completion, new requests, or infeasibility).
Redundancy-Aware Team Formation: Once tasks are assigned to abstract teams, a Mixed-Integer Linear Programming (MILP) formulation assigns specific robots. This step incorporates a redundancy margin ( $\alpha$ ) to account for subtask uncertainty and potential robot failures, ensuring disjoint team membership while satisfying capacity constraints.

Layer III: Local Coordination within Teams

Given a team's assigned tasks, this layer coordinates subtask execution and trajectory generation based on the nature of the subtasks:

Static & Known: Solves a multi-vehicle routing problem (MVRP). For non-holonomic robots, it uses motion primitives (e.g., Dubins curves) and hybrid-A* search to optimize trajectories and task sequencing jointly.
Static & Unknown: Employs a Simultaneous Exploration and Coordination (SEC) strategy. Robots collaboratively cover the region to reveal subtasks, dynamically inserting discovered subtasks into local plans to minimize makespan disruption.
Dynamic & Known: Uses a Distributed Dynamic Coalition Formation (DCF) algorithm. Robots update their coalition memberships based on local cost estimates and peer communication to track moving targets, ensuring disjoint coalitions and balanced workloads.

3. Key Contributions

The paper claims two primary contributions:

A Flexible, Scalable Framework: HECTOR enables operators to interact with and coordinate large-scale heterogeneous fleets in uncertain settings. It explicitly handles the interplay between high-level temporal logic missions and low-level motion feasibility.
Explicit Treatment of Practical Aspects: The framework integrates heterogeneous capabilities, parameterized collaboration utilities, temporal constraints, and variability in subtask distribution. It introduces a specific protocol for online human intervention that avoids the need for full system reconstruction upon mission changes.

4. Experimental Results

Extensive human-in-the-loop simulations were conducted with 80 heterogeneous robots performing 30 missions (450 subtasks) in a dynamic environment.

Performance vs. Baselines: HECTOR achieved a total mission response time of 184.4s, significantly outperforming centralized MILP (327.8s) and sampling-based methods (286.0s). It maintained a 100% success rate, whereas methods ignoring uncertainty (e.g., SAMP-Subtask, Flow) dropped to 86–96%.
Computational Efficiency: The average planning time was 0.46s (max 0.76s), demonstrating the efficiency of the hierarchical approach compared to the infinite-horizon baseline (Inf-H), which suffered from planning times up to 415.8s.
Scalability & Robustness: Tests with up to 170 robots and 100 tasks showed that planning times remained under 2.0s even with robot failure rates of 10%. The system successfully recovered from failures by reassigning resources without losing mission success.
High-Fidelity Validation: A ROS-based simulation with 12 UAVs in a disaster relief scenario confirmed the framework's ability to handle real-world dynamics, curvature constraints, and real-time operator interventions (cancellations, reassignments, deadline changes).

5. Significance and Claims

The paper positions HECTOR as a necessary evolution in multi-robot systems, bridging the gap between theoretical temporal logic planning and practical, large-scale deployment.

Practical Relevance: It addresses the "practical need" for operators to manage fleets in disaster relief, surveillance, and delivery, where tasks evolve and uncertainties are inherent.
Efficiency through Hierarchy: By decoupling team formation, task assignment, and local coordination, the method avoids the double-exponential complexity of global synthesis while retaining correctness guarantees.
Human-Centric Design: Unlike fully autonomous systems, HECTOR explicitly supports "lean communication" and high-level oversight, allowing operators to intervene effectively without being overwhelmed by low-level details.
Limitations: The authors modestly note that the current work does not address "dynamic and unknown" tasks (where subtask locations and counts are both unknown and changing simultaneously), as this would require a combination of exploration and dynamic coalition strategies not fully explored here. Additionally, communication constraints (latency, packet loss) are acknowledged as areas for future refinement.