Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport

This paper introduces Cognition-to-Control (C2C), a three-layer hierarchical framework that bridges high-level deliberation and low-level execution for human-humanoid collaborative transport by integrating a VLM-based grounding layer, a decentralized multi-agent reinforcement learning coordination layer, and a whole-body control layer to achieve robust, stable, and adaptive joint manipulation.

The Gist

Imagine you and a friend are trying to carry a very long, heavy table through a crowded house. You need to walk in perfect sync, turn corners without hitting the walls, and adjust your grip if your friend suddenly stops or changes direction. If one of you is too rigid or doesn't "read the room," the table crashes, or you both trip.

This paper presents a new "brain" for robots that helps them do exactly this with humans. The authors call their system Cognition-to-Control (C2C).

Here is how it works, broken down into three simple layers using a Human Brain Analogy:

1. The "Cerebral Cortex" (The Strategic Planner)

What it does: This is the high-level thinking part. It looks at the room, understands the goal ("Get the table to the kitchen"), and spots obstacles ("There's a narrow door").
The Analogy: Imagine this is the Captain of a ship. The Captain doesn't steer the wheel every second; instead, they look at the map and say, "Okay, we need to turn left in 10 seconds to avoid that iceberg."

• In the paper: This layer uses a Vision-Language Model (VLM). It looks at what the robot and human see, understands the scene in plain English, and generates a list of "waypoints" (like GPS dots) for where the object should go next. It doesn't worry about how to move the muscles; it just sets the destination.

2. The "Cerebral Lobes" (The Tactical Team)

What it does: This is the part that figures out how to move together to hit those waypoints. It's where the robot and human "dance" together.
The Analogy: Imagine this is the Dance Floor. The Captain says "Turn left," but the dancers (the robot and the human) have to figure out who leads, who follows, and how to step without stepping on each other's toes.

• In the paper: This uses Multi-Agent Reinforcement Learning (MARL). Instead of the robot being told "You are the leader, the human is the follower," they learn to adapt instantly.
  • If the human speeds up, the robot speeds up.
  • If the human slows down, the robot slows down.
  • They treat the task as a shared goal (a "Potential Game"). They don't need to guess what the human is thinking; they just react to the shared goal of moving the table safely. This allows them to switch roles naturally (sometimes the robot leads, sometimes the human does) without breaking the system.

3. The "Cerebellum" (The Muscle Memory)

What it does: This is the super-fast, physical execution layer. It takes the "dance steps" from the Tactical Team and actually moves the robot's joints.
The Analogy: This is your Reflexes. When you are walking on a slippery floor, your brain doesn't stop to think about physics; your body just adjusts your balance instantly so you don't fall.

• In the paper: This is the Whole-Body Control (WBC) layer. It runs at a very high speed (hundreds of times a second). It ensures the robot doesn't tip over, that its feet don't slip, and that the table stays level. It takes the high-level plan and makes sure the physics actually work.

---

Why is this a big deal?

The Old Way (The "Scripted" Robot):
Imagine a robot that follows a strict script: "Step forward, wait 1 second, turn left." If the human partner stops suddenly, the robot keeps walking and bumps into them. It's like a rigid puppet. It works in a perfect world but fails in a messy, real one.

The New Way (C2C):
This system is like a skilled partner.

1. It understands the big picture: It knows where to go (Cortex).
2. It learns to dance: It figures out how to move with you without needing a script (Lobes).
3. It has great reflexes: It keeps you from falling (Cerebellum).

The Results

The researchers tested this with a real humanoid robot (Unitree G1) and a human carrying heavy objects through tricky scenarios:

• Narrow Gates: Squeezing through tight doors.
• Long Objects: Carrying a long pole that is hard to balance.
• Turning Corners: Navigating tight turns.

The Outcome:

• The new system was 45% better than the old "scripted" robots.
• It was much more stable (the object didn't tilt or drop).
• It worked even when the human did something unexpected.

The Bottom Line

This paper solves the "gap" between thinking (planning a route) and doing (moving muscles). By separating these tasks into three specialized layers, they created a robot that doesn't just follow orders, but actually collaborates with humans like a skilled teammate, adapting in real-time to keep the job done safely.

Technical Summary

Here is a detailed technical summary of the paper "Cognition to Control – Multi-Agent Learning for Human-Humanoid Collaborative Transport."

1. Problem Statement

The paper addresses the critical challenges in Human-Robot Physical Collaboration (HRC), specifically for heavy-duty transport tasks involving humanoid robots and human partners. Existing approaches suffer from three main limitations:

• The Cognitive-to-Physical Gap: High-level reasoning (e.g., navigating a corridor) often relies on Vision-Language Models (VLMs) that output low-frequency, discrete tokens. These cannot directly drive the high-frequency, continuous kinodynamic control required for stable physical coupling.
• Brittleness of Heuristics: Traditional methods rely on explicit role assignment (leader-follower) or scripted coordination. These fail in unstructured environments where human behavior is unpredictable, leading to oscillatory or catastrophic failures.
• Non-Stationarity in Learning: Treating the human as a passive environmental disturbance in Single-Agent Reinforcement Learning (SARL) ignores reciprocal adaptation. As the robot learns, the human adapts, creating a shifting optimization target that destabilizes training.

The core problem is how to create a unified architecture that bridges long-horizon strategic planning with millisecond-level physical execution while enabling emergent, role-free mutual adaptation between a human and a robot.

2. Methodology: Cognition-to-Control (C2C)

The authors propose C2C, a three-layer hierarchical framework that explicitly decouples semantic reasoning, tactical coordination, and physical execution. The system is modeled as a Task-Centric Markov Potential Game.

Layer 1: Collaborative Cognitive Layer (The "Cerebral Cortex")

• Function: High-level strategic grounding and intent inference.
• Mechanism: Uses decentralized Vision-Language Models (VLMs). Each agent (human and robot) observes the environment from an egocentric view, generates a 2D overhead representation, and proposes candidate waypoints (anchors) for the object's Center of Mass (CoM).
• Consensus: Agents exchange compact summaries of their proposals to synthesize a collective intent and a consensus anchor sequence ($T = \{w_k\}$). This provides a shared, semantic reference path without requiring explicit role assignment.

Layer 2: Skill Policy Layer (The "Cerebral Lobes")

• Function: Tactical coordination and role-free adaptation.
• Mechanism: Implements Multi-Agent Reinforcement Learning (MARL).
  • Formulation: The task is defined as a Markov Potential Game where agents share a potential function $\Phi$ (negative distance to the task manifold). This ensures that individual policy updates align with global task progress.
  • Observation: Agents receive a 210-dimensional vector including strategic waypoints, self/partner kinematic states, object geometry, and LiDAR-like environmental data.
  • Action: The policy outputs residual commands ($u_{res}$) relative to a nominal base controller. This allows the MARL layer to focus on fine-grained adjustments (e.g., vertical synchronization, compliance) while the base handles gross motion.
  • Training: Uses Centralized Training with Decentralized Execution (CTDE) with a joint-action critic to mitigate non-stationarity caused by evolving partner policies.

Layer 3: Whole-Body Control Layer (The "Cerebellum")

• Function: High-frequency physical execution and stability.
• Mechanism: A high-rate controller ($f_{high}$) that maps the residual task-space commands from the MARL layer into joint-level torques.
• Constraints: Enforces kinematic/dynamic feasibility and contact stability, ensuring the humanoid does not fall and the object remains level.

3. Key Contributions

1. Hierarchical Architecture: A novel decoupling of semantic reasoning (VLM) from tactical coordination (MARL) and physical execution (WBC), effectively bridging the frequency and granularity gap between high-level planning and low-level control.
2. Emergent Role-Free Coordination: By formulating HRC as a Markov Potential Game, the system eliminates the need for explicit leader-follower roles or intent inference modules. Leader-follower behaviors emerge naturally as stable equilibria based on the shared task potential.
3. Robustness to Heterogeneity: The framework supports heterogeneous agents (human and robot) with independent policies, internalizing partner dynamics rather than estimating them, which significantly reduces out-of-distribution (OOD) risks.
4. Real-World Validation: Successful deployment on a Unitree G1 humanoid robot collaborating with a human partner in physically constrained, heavy-duty transport tasks.

4. Experimental Results

The framework was evaluated in a 9-scenario matrix covering Orientation-Sensitive Pushing (OSP), Spatially-Confined Transport (SCT), and Super-Long Object Handling (SLH).

• Performance Gains:
  • The proposed architecture achieved an overall success rate of ~83% across all scenarios, compared to 56.5% for a robot-script baseline.
  • This represents a 45.6% relative improvement (Architecture Synergy Index).
  • In specific difficult tasks (e.g., pivoting a long object), the improvement reached 55.9%.
• Real-World Metrics (Unitree G1):
  • Success Rate: 100% in OSP, 100% in SCT, and 80% in SLH (vs. 40% for Single-Agent baselines).
  • Efficiency: Task completion time was reduced by ~20% (e.g., 81.5s vs. 101.6s for SCT).
  • Stability: The mean object tilt rate was significantly lower (2.4°/s vs. 3.2°/s), indicating superior balance and contact stability.
• Ablation Study: Removing either the VLM cognitive layer or the MARL skill layer resulted in total task failure, confirming that all three layers are essential for success.

5. Significance

This work represents a significant step forward in embodied AI and human-robot collaboration.

• Paradigm Shift: It moves away from rigid, scripted interactions toward fluid, emergent collaboration where roles are dynamic and determined by the task context rather than pre-programmed rules.
• Scalability: The "Cognition-to-Control" hierarchy provides a blueprint for integrating large language/vision models with real-time control systems, solving the "frequency gap" that has hindered the deployment of VLA systems in physical robotics.
• Safety and Resilience: By internalizing mutual adaptation, the system is robust to unpredictable human behaviors, making it viable for real-world industrial and assistive applications where safety and stability are paramount.