CycleManip: Enabling Cyclic Task Manipulation via Effective Historical Perception and Understanding

1. Problem Definition

The paper addresses Cycle-Based Manipulation, a critical yet underexplored area in robotics. These tasks require a robot to perform repetitive, cyclic actions (e.g., shaking a bottle, hammering a nail, mixing chemicals) and autonomously terminate the action after a specific number of cycles or when a condition is met.

Core Challenges:

• Non-Markovian Nature: The correct decision at any given moment depends not just on the current observation, but on the accumulated history of the cycle.
• Visual Ambiguity: In cyclic tasks, consecutive frames often look identical (e.g., the bottle looks the same after every shake). Standard imitation learning policies relying on short observation windows fail to distinguish between "shake #1" and "shake #5," leading to infinite loops or premature termination.
• Computational Cost: Simply expanding the observation window to include more history increases computational overhead significantly, especially when processing high-dimensional visual data (RGB/Point Clouds) at every timestep.
• Lack of Benchmarks: There was no standardized dataset or automatic evaluation tool to assess cyclic manipulation performance.

2. Methodology: The CycleManip Framework

The authors propose CycleManip, an end-to-end imitation learning framework that enhances historical perception and understanding without requiring hierarchical structures or extra auxiliary models.

A. Effective Historical Perception (Cost-Aware Sampling)

To solve the computational bottleneck of long-horizon observation, the authors introduce a Cost-Aware Sampling Strategy:

• Differentiation of Inputs: Observations are split into Low-Overhead (e.g., proprioception, joint positions) and High-Overhead (e.g., RGB images, point clouds).
• Dense Sampling (Low-Overhead): The system employs dense, broad sampling for proprioceptive data. Since encoding this data is cheap, the model can access a long temporal history to track the exact progression of the cycle.
• Sparse/Heuristic Sampling (High-Overhead): For visual data, the system uses a heuristic strategy (right-sided binary sampling + exponential sampling) to select diverse frames from the past. This maintains a consistent number of visual frames ($K_{high}$) while extending the temporal horizon, avoiding the cost of encoding every single frame.
• Pose Difference Encoding: For low-overhead inputs, the model uses end-effector pose differences rather than absolute positions. This removes bias from absolute location and highlights the cyclic movement pattern.

B. Effective Historical Understanding (Multi-Task Learning)

To prevent the model from merely memorizing actions without understanding the task state, the authors introduce a Multi-Task Learning Objective:

• Auxiliary Task: The policy is trained to predict the current task progress (e.g., "what percentage of the total cycles have been completed?").
• Implementation: The observation features are passed through a multi-layer MLP for fusion, followed by a single-layer MLP to predict a discretized progress value (0–10 bins).
• Benefit: This forces the model to learn progression-discriminative features. Instead of just learning "move down," the model learns "move down during the 3rd cycle," enabling it to decide when to stop.

C. Architecture

• Policy: A language-guided diffusion policy.
• Inputs: User language command (encoded via CLIP), sampled historical observations (encoded via Point/Transformer encoders).
• Output: Action prediction via a Diffusion model with FiLM conditioning.
• Loss Function: A combination of Mean Squared Error (MSE) for action prediction and Cross-Entropy (CE) for the auxiliary progress prediction task.

3. Key Contributions

1. CycleManip Framework: A novel end-to-end imitation learning approach that solves cyclic manipulation by combining cost-aware sampling (for perception) and multi-task learning (for understanding).
2. CycleManip Benchmark:
   • Built on the RoboTwin 2.0 simulation platform.
   • Contains 8 diverse cyclic tasks (e.g., bottle shaking, block hammering, egg beating).
   • Features an automatic evaluation system that counts cycles and verifies termination conditions, removing the need for manual review.
3. Comprehensive Evaluation: Extensive experiments in both simulation and real-world settings across diverse robotic platforms (single-arm, dual-arm, dexterous hands, and humanoid robots).

4. Experimental Results

The paper validates the method against State-of-the-Art (SOTA) baselines (DP, DP3, RDT, Pi-0).

• Simulation Performance:
  • CycleManip achieved significantly higher Success Rates (Suc.) across all 8 tasks compared to baselines. For example, in "Bottle Shaking," CycleManip achieved 97% success vs. 33% for RDT and 14% for Pi-0.
  • Cycle Count Deviation (Cyc.): The proposed method had the lowest deviation from the target number of cycles, proving superior temporal reasoning.
• Real-World Performance:
  • Tested on AgileX Piper (grippers), BrainCO Revo2 (dexterous hands), and Unitree G1 (humanoid).
  • CycleManip consistently outperformed the best baseline (DP3). For instance, in "Block Hammering," it achieved 93.75% success vs. 37.5% for DP3.
• Generalization & Plug-and-Play:
  • General Manipulation: The method also improved performance on non-cyclic general tasks from the RoboTwin benchmark, suggesting the historical modeling benefits general reasoning.
  • VLA Integration: When integrated as a plug-and-play component into the Pi-0 (Vision-Language-Action) model, it significantly boosted Pi-0's performance on cyclic tasks (e.g., from 19% to 72% success in bottle shaking).
• Efficiency: The method achieved these gains with only marginal increases in training/inference time and GPU memory usage compared to standard baselines.

5. Significance

• Bridging a Critical Gap: This work moves robot manipulation beyond simple "pick-and-place" or single-step tasks to complex, repetitive daily chores that require temporal reasoning.
• Scalability: By avoiding heavy hierarchical models or external memory modules, the framework is computationally efficient and easily deployable on various hardware.
• Foundation for AGI: The ability to understand "progress" and "termination" in repetitive tasks is a fundamental step toward autonomous robots that can operate reliably in unstructured human environments.
• Standardization: The introduction of a dedicated benchmark and automatic evaluation tool provides a necessary foundation for future research in cyclic robotics.