CLASP: Closed-loop Asynchronous Spatial Perception for Open-vocabulary Desktop Object Grasping

1. Problem Statement

The paper addresses the critical challenges in deploying Vision-Language Models (VLMs) for low-level robotic grasping in open-vocabulary, unstructured desktop environments. While VLMs offer strong semantic understanding, their direct application to physical manipulation faces three primary bottlenecks:

• Data Scarcity: A lack of high-quality, multimodal demonstration data for robotic grasping.
• Spatial Hallucinations: VLMs often suffer from weak geometric grounding, leading to "spatial hallucinations" where the model predicts grasp points that are physically impossible or unstable.
• Open-Loop Fragility: Traditional open-loop execution is highly vulnerable in dynamic environments; if a grasp fails, the system lacks a mechanism to diagnose the error and correct it without human intervention.

2. Methodology: The CLASP Framework

The authors propose CLASP (Closed-loop Asynchronous Spatial Perception), a framework that integrates multimodal perception, logical reasoning, and state-reflective feedback to bridge the gap between semantic intent and physical execution.

A. Dual-Pathway Hierarchical Perception

To mitigate spatial hallucinations, the system decouples high-level semantic intent from low-level geometric grounding:

• Semantic Pathway: Extracts global semantic embeddings to understand the task and object category.
• Geometric Pathway: Performs open-vocabulary object detection, segmentation, and explicit pose/centroid estimation.
• Output: Instead of raw text, the system outputs structured action tuples (2D contact points, angles, and object categories) in a deterministic JSON format, ensuring the reasoning model adheres to physical constraints.

B. Asynchronous Closed-Loop Evaluator

This module introduces a robust error-correction mechanism:

• Judger Module: Compares the system state before and after execution.
• Diagnostic Feedback: If a grasp fails, the Judger generates text-based diagnostic feedback (e.g., "object not moved," "grasp point slipped").
• Heuristic Adjustment: Based on feedback, a heuristic algorithm (Failure $\to$ Shift $\to$ Execute $\to$ Check) adjusts grasp parameters (e.g., shifting the grasp point toward the object's center) and re-routes the information to the reasoning module for a retry.
• Asynchronous Execution: The Reasoner and Judger operate on independent background threads, decoupling perception/reasoning from execution to minimize blocking overhead.

C. Scalable Multi-modal Data Engine

To solve the data scarcity issue, the authors developed an automated pipeline that synthesizes high-quality spatial annotations and reasoning templates from both real and synthetic scenes. This engine generates visual prompts, masks, and grasp poses without human teleoperation, creating a massive dataset for training and fine-tuning.

3. Key Contributions

1. Dual-Pathway Perception: A novel architecture that separates semantic reasoning from geometric grounding, outputting structured action tuples to eliminate spatial hallucinations and enable deterministic control.
2. Asynchronous Closed-Loop Feedback: An evaluator that establishes a robust error-correction loop using diagnostic text feedback and heuristic adjustments, significantly improving success rates in dynamic environments compared to open-loop baselines.
3. Automated Data Synthesis: A scalable engine that generates high-quality multimodal demonstration data without manual teleoperation, bridging the sim-to-real gap.
4. State-of-the-Art Performance: The framework achieves an 87.0% overall success rate in pick-and-place tasks, outperforming existing VLM and VLA baselines, particularly in geometrically challenging categories like tools.

4. Experimental Results

The authors evaluated CLASP in the ManiSkill simulation environment and on a WidowX 250S 6-DoF robotic manipulator in real-world settings.

• Baselines: Compared against Qwen3-VL, Doubao-Seed, AffordanceNet, and AnyGrasp.
• Performance Metrics:
  • Overall Success Rate: CLASP achieved 87.0%, significantly outperforming the best baseline (AffordanceNet at 75.33% and Qwen3-VL at 72.27%).
  • Category Breakdown:
    • Toys: 100% (CLASP) vs. 92% (Qwen3-VL).
    • Blocks: 84% (CLASP) vs. 78.4% (Qwen3-VL).
    • Tools: 77% (CLASP) vs. 46.4% (Qwen3-VL). The improvement on tools is particularly notable due to their complex geometry.
• Latency Benchmark: The proposed Asynchronous Policy reduced total execution time by 39.8% and average task completion time by 39.9% compared to a standard streaming (sequential) policy.
• Real-World Validation: In physical experiments, CLASP successfully identified optimal grasp points on complex tools (e.g., wrenches) where baselines failed (e.g., selecting edge points or failing to segment).

5. Significance

This work represents a significant step forward in making VLMs viable for practical robotic manipulation. By addressing the "spatial hallucination" problem through hierarchical perception and the "open-loop fragility" problem through asynchronous closed-loop feedback, CLASP enables robots to:

• Handle open-vocabulary objects without extensive retraining.
• Operate robustly in cluttered, dynamic environments.
• Self-correct errors autonomously, reducing the need for human intervention.
• Successfully manipulate geometrically complex objects (like tools) that typically defeat standard VLM-based grasping approaches.

The framework effectively bridges the semantic-geometric gap, providing a scalable and robust solution for intelligent manufacturing, logistics, and service robotics.