Technical Summary: COCORELI

Problem Statement

Autonomous agents operating in collaborative settings (e.g., robotics, manufacturing, tool-use) frequently encounter human instructions that are incomplete or underspecified. Humans often rely on shared context or incremental communication, leaving essential task parameters unspecified. While recent approaches have improved the detection of missing information, detection alone is insufficient. Agents often proceed to execution even after recognizing underspecification, inferring missing details rather than acquiring them through interaction. This leads to "plausible but incorrect" actions, hallucinations, and unsafe outcomes.

The authors identify this failure as a structural decoupling between detecting missing information and preventing execution. Existing paradigms, including Chain-of-Thought (CoT) and agentic frameworks, typically assume instructions are executable as given or that missing details can be inferred from background knowledge. Increasing model scale does not resolve this; larger models tend to generate more plausible guesses rather than reliably detecting and resolving uncertainty. The paper posits that reliable collaborative execution requires an architectural enforcement where missing information acts as a hard precondition for action.

Methodology: COCORELI Architecture

To address this, the authors propose COCORELI (Cooperative, Reconstitution & Execution of Language Instructions), a modular agentic architecture designed to enforce execution preconditions. Unlike monolithic prompting, COCORELI decomposes the task into six specialized components, all utilizing the same underlying LLM (LLaMA-3.1 8B) but with specialized prompts and structured inputs:

1. Instruction Parser: Identifies object types and extracts preliminary attributes, filtering relevant schemas to avoid prompt bloat.
2. Builder: Resolves object-specific attributes (e.g., orientation, part configuration).
3. Locator: Interprets spatial descriptions and maps relational expressions to candidate coordinates, applying object-specific placement rules.
4. Discourse Module: Generates targeted clarification questions (CQs) when required fields remain unresolved.
5. External Memory: Stores previously constructed structures as relational graphs (nodes = parts, edges = spatial relations). It supports abstraction by separating structural patterns from instance-specific parameters (e.g., color, absolute position).
6. Executor: Converts completed representations into concrete JSON actions and validates them against environmental constraints.

Core Mechanism:
The system represents objects and actions as typed JSON structures where fields are initialized to null. As dialogue proceeds, fields are filled with extracted information. Crucially, execution is blocked whenever required fields remain unresolved. The system does not infer missing values; instead, it triggers the Discourse Module to request clarification. This creates a structural coupling: detecting a missing parameter simultaneously blocks execution, preventing hallucinated actions.

Experimental Setup

The authors evaluate COCORELI in ENVIRONMENT, a controlled collaborative construction benchmark designed to isolate underspecification, sequential state changes, and structural reuse.

• Environment: A 16×16×16 discrete 3D grid with heterogeneous part types (screws, bridges, etc.) and strict physical constraints (gravity, support dependencies). Incorrect placements are irreversible.
• Task Types:
  1. Single fully-specified instructions.
  2. Instruction sequences.
  3. Complex structure construction (multi-part shapes).
  4. Underspecified instructions: Parameters omitted, requiring detection and clarification.
  5. In-context abstraction and reuse: Reproducing previously built structures in new contexts without restating original instructions.
• Baselines:
  • Single-LLM Prompting (CoT): Monolithic prompting using GPT-4.1 and Claude-3.5 Sonnet.
  • Agentic LLM Decomposition: Modular decomposition where subtasks are solved end-to-end by an LLM (LLaMA-3-70B), including generating Python code for abstraction.
  • COCORELI: Structured parsing with explicit task representations and deterministic execution control.

The evaluation also includes a transfer test on ToolBench API tasks to assess the abstraction mechanism's generalizability beyond spatial construction.

Key Results

1. Handling Underspecification (Task iv)

• Detection & Clarification: COCORELI and the Agentic baseline achieved 100% detection of missing parameters and successfully issued clarification questions. In contrast, single-LLM CoT baselines often failed to identify missing parameters (e.g., 27.2% detection for GPT-4.1 on single-part tasks).
• Execution Safety: COCORELI achieved 0% hallucinations on parts, colors, and locations for underspecified instructions. Conversely, CoT baselines frequently inferred missing attributes, leading to high rates of hallucination (e.g., 100% location hallucination for GPT-4.1 on two-part tasks).
• Conclusion: Detection alone is insufficient; without architectural enforcement, baselines proceed with inferred (incorrect) values.

2. Complex Structure Construction (Task iii)

• COCORELI achieved the highest step-level accuracy (78.57%), outperforming CoT baselines (54.67%–69.04%) and the Agentic baseline (69.04%).
• COCORELI was the only system capable of partially reconstructing the most complex shape ("Moroccan bridge"), which required abstraction and subsequent reference.

3. Abstraction and Reuse (Task v)

• COCORELI successfully reconstructed all evaluated structures, including complex ones (e.g., "Skull" with 62 parts), achieving 100% accuracy on complex shapes.
• CoT baselines failed on the most complex structures (0% success), and the Agentic baseline failed on medium-sized shapes.
• The paper attributes COCORELI's success to treating abstraction as a structured parsing problem (extracting relational graphs) rather than requiring the model to infer reusable procedures or generate code from scratch.

4. ToolBench Transfer

• On 100 ToolBench workflows, COCORELI achieved 100% precision, recall, and F1, significantly outperforming the CoT baseline (30.9% F1).
• This demonstrates that the structured abstraction mechanism generalizes to non-spatial, API-driven tasks.

Significance and Claims

The paper claims that reliable collaborative execution under underspecification depends not merely on model capability but on architectural enforcement.

• Structural Coupling: The primary contribution is the structural coupling of detection and prevention. In COCORELI, incomplete task representations cannot be executed. This eliminates hallucinated actions by construction, a property absent in prompting-based paradigms where detection and execution are decoupled.
• Minimal Instantiation: The authors do not claim COCORELI as a definitive solution for all collaborative dialogue but as a minimal instantiation proving that prompting-based paradigms systematically lack the mechanism to block execution under uncertainty.
• Limitations: The authors acknowledge that COCORELI assumes tasks can be represented via structured schemas (JSON), abstracts away from multimodal perception, models only a narrow form of clarification dialogue (excluding negotiation or complex reasoning), and lacks an explicit high-level planning module.

In summary, the paper argues that for agents to operate reliably in collaborative settings with incomplete instructions, the system architecture must explicitly enforce that missing information is resolved before any action is taken, rather than relying on the model's implicit ability to guess or infer.