BiPreManip: Learning Affordance-Based Bimanual Preparatory Manipulation through Anticipatory Collaboration

1. Problem Definition

The paper addresses a specific class of bimanual robotic manipulation tasks termed Collaborative Preparatory Manipulation. Unlike standard pick-and-place or symmetric dual-arm tasks, these scenarios involve objects that cannot be directly grasped or functionally manipulated by a single arm due to their initial state or geometry.

• Core Challenge: The robot must perform a sequence of asymmetric, long-horizon actions where one arm (the assistant) performs preparatory manipulation to reconfigure the object (e.g., lifting, rotating, or pushing), thereby creating favorable physical conditions for the other arm (the primary) to execute the goal-directed task.
• Examples:
  • Lifting a capped bottle to allow the other hand to unscrew the cap without colliding with the table.
  • Pushing a flat tablet to the edge of a table to enable a grasp.
  • Rotating a pen lying on its side so the cap faces the primary arm.
• Requirements: The system requires deep understanding of object semantics, geometry, spatial relationships, and the ability to anticipate future states to coordinate two arms without interference.

2. Methodology: BiPreManip Framework

The authors propose BiPreManip, a visual affordance-based framework that utilizes anticipatory reasoning. Instead of reacting to the current state, the system envisions the final goal and works backward to plan the necessary preparatory steps.

The pipeline consists of the following key modules:

A. Goal Affordance Network (Anticipatory Stage)

• Input: Initial object point cloud ($O$) and a language instruction ($l$).
• Function: Predicts an anticipatory goal affordance map. It envisions where and how the primary arm should ideally interact with the object after preparation is complete.
• Mechanism: Uses PointNet++ for point cloud encoding and CLIP for text embedding. A Conditional Variational Autoencoder (cVAE) predicts the 6D gripper pose ($p_{goal}, d_{goal}$) for the primary arm's future interaction.

B. Pre-Affordance Network

• Function: Guided by the anticipated goal, this network infers the collaborative preparatory affordance for the assistant arm.
• Mechanism: It determines where the assistant arm should grasp and how it should manipulate the object (e.g., lift or rotate) to expose the target region for the primary arm, ensuring the assistant's actions do not block the primary arm's future path.

C. Anticipatory Object Pose Predictor & Reorient Actor

• Pose Predictor: Estimates the desired object transformation ($T_{obj}$) that best facilitates the primary arm's goal. It takes features from both the anticipated goal action and the preparatory action to compute a target pose.
• Reorient Actor: Executes the physical motion to reorient the object from its initial state to the predicted target state. This ensures the object is in a collision-free, accessible configuration.

D. Re-invocation of Goal Affordance Network (Execution Stage)

• After the preparatory manipulation is complete, the Goal Affordance Network is re-run on the updated scene (containing the reoriented object and the assistant arm).
• This generates the final, precise action for the primary arm to execute the goal (e.g., opening the cap).
• Key Design: Sharing parameters between the anticipatory and execution stages ensures semantic consistency between the "imagined" future and the "executed" reality.

E. Training Strategy

• Supervision: Since the anticipatory stage does not exist in standard demonstrations, the authors construct ground truth by mapping execution-stage labels back to the anticipatory frame using known object pose transformations.
• Loss Functions: Includes $\ell_1$ loss for affordance scores, geodesic distance loss for gripper orientations, and KL divergence for latent space regularization.

3. Key Contributions

1. New Task Formulation: Introduction of Bimanual Preparatory Manipulation, a challenging domain requiring asymmetric coordination and long-horizon planning where one arm must modify the environment for the other.
2. BiPreManip Framework: A novel visual affordance-based approach that decouples the problem into "anticipating the goal" and "planning the preparation," enabling robots to reason about future object configurations.
3. Comprehensive Benchmark: Creation of a new benchmark comprising 18 object categories (from ShapeNet and PartNet-Mobility) across three task types: Articulated Manipulation, Edge-Pushing, and Plate-Lifting.
4. Real-World Validation: Successful deployment on a dual-arm robot (ARX-X7s) and demonstration of robot-human handover capabilities, proving the method's applicability beyond simulation.

4. Experimental Results

The method was evaluated in both simulation (SAPIEN platform) and the real world, comparing against strong baselines including ACT (Transformer-based), 3DFA/3DA (Diffusion/Flow-matching), W2A (Single-arm affordance), and Heuristic rules.

• Simulation Performance:
  • Success Rates: BiPreManip significantly outperformed all baselines. For example, in Articulated Manipulation, it achieved 30-85% success rates across categories, compared to 1-38% for ACT and 1-30% for 3DFA.
  • Generalization: The model demonstrated strong zero-shot generalization to unseen object instances and categories, maintaining high performance on novel objects where baselines often failed.
• Real-World Performance:
  • In real-world tests, BiPreManip achieved success rates of 60-80% across various tasks (e.g., 7/10 for Edge-Pushing a book, 8/10 for Articulating a dispenser), whereas baselines like W2A and 3DFA achieved near-zero success (0-2/10).
• Ablation Studies:
  • Removing the Goal Affordance Network (w/o Ant-Aff) caused a significant drop in performance, confirming that anticipating the final goal is crucial for guiding preparatory actions.
  • Removing the Object Pose Predictor (w/o ObjPosePred) also degraded performance, highlighting the necessity of explicitly planning the target object configuration.

5. Significance and Impact

• Bridging Perception and Action: The work advances the field by moving beyond simple reactive policies to anticipatory reasoning. It demonstrates that robots can "think ahead" about how an object should be positioned before acting.
• Complex Coordination: It solves a critical gap in bimanual robotics: handling tasks where direct interaction is impossible. This is essential for complex household and industrial tasks involving lids, covers, and flat objects.
• Human-Robot Collaboration: The framework's ability to anticipate a human's intent (demonstrated in handover scenarios) suggests a pathway for more intuitive and helpful robotic assistants that can prepare objects for human use.
• Generalization: By relying on visual affordances rather than rigid object models, the approach generalizes well to diverse and unseen object geometries, a key requirement for real-world deployment.

In summary, BiPreManip represents a significant step forward in robotic manipulation by enabling robots to perform complex, multi-step, collaborative tasks through the integration of visual affordance learning and anticipatory planning.