GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning

Technical Summary: GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning

Problem Statement
Fine-grained robotic manipulation often fails when inaccurate initial grasps propagate errors, necessitating complex pose corrections that are difficult to achieve with limited demonstrations and parallel-jaw grippers. While recent imitation learning (IL) methods (e.g., RVT2, ACT) have improved high-dimensional action modeling, their policy representations typically treat grasp-pose priors as implicit, leaving the system sensitive to initial grasp quality. Furthermore, existing Transformer-based approaches suffer from quadratic computational costs relative to sequence length, creating latency issues for real-time deployment with multi-view RGB-D observations. The core challenge is to strengthen grasp-sensitive representations without collecting separate grasp datasets or incurring prohibitive computational overhead.

Methodology: GPA-RAM Framework
The authors propose GPA-RAM, a unified framework comprising two complementary modules: Grasp-Pretraining Augmentation (GPA) and Robotic Attention Mamba (RAM).

1. Grasp-Pretraining Augmentation (GPA):

   • Mechanism: GPA incorporates grasp-pose priors directly from expert task demonstrations without requiring additional data collection or manual grasp-pose annotation.
   • Process: A grasp-pose detector is pre-trained on the grasp configurations contained within the expert demonstrations. During the main policy training, the detector's output head is removed, and its frozen feature backbone is retained. These grasp-sensitive features are spatially aligned and fused with the task-policy features via a "Pre-trained Location Fusion" (PLoFusion) module.
   • Integration: This design introduces an explicit manipulation prior, allowing the policy to leverage learned grasp geometries for downstream tasks like stacking or insertion.

2. Robotic Attention Mamba (RAM):

   • Architecture: To address the computational inefficiency of global self-attention in Transformers, RAM employs a hybrid architecture combining attention mechanisms with linear-time state-space modeling (Mamba).
   • Function: Multi-view RGB-D observations are rendered into virtual viewpoints. RAM uses attention for view-specific feature interaction and Mamba blocks to efficiently model long-range spatial dependencies with linear scaling relative to sequence length.
   • Variants:
     • Discrete Policy: Employs a coarse-to-fine RAM architecture. A coarse stage predicts a 3D position to define a local region, which is then re-rendered for a fine stage to generate precise spatial representations for keyframe prediction.
     • Continuous Policy: Utilizes a single-stage RAM integrated with an action-chunking decoder (adapted from ACT) to generate continuous action sequences at high frequencies (e.g., 50 Hz).

Key Contributions

• GPA Framework: A modular augmentation that transfers grasp-pose priors from demonstrations to manipulation policies, improving precise grasping without extra annotation.
• RAM Architecture: A hybrid network combining attention and state-space models to enable efficient spatial feature extraction and real-time action generation, reducing reliance on computationally expensive global self-attention.
• Comprehensive Validation: The framework is validated across four platforms (simulated Franka Panda, simulated ALOHA, physical UR5, and physical ARX R5) covering both discrete keyframe prediction and continuous bimanual manipulation.

Experimental Results
The paper reports significant performance improvements across multiple benchmarks:

• RLBench (Discrete Tasks): GPA-RAM achieved an average success rate of 87.5% across 18 tasks. This outperforms the previous state-of-the-art (SOTA) ARP+ (84.9%) by 2.6 percentage points and RVT2 (79.3%) by 8.2 percentage points. The model also achieved the best Average Rank (2.28), indicating superior consistency.
  • Specific Gains: In the "Insert Peg" task, GPA-RAM reached 95.0% success compared to ARP+'s 78.4%. In "Stack Cups," it achieved 84.8% vs. ARP+'s 80.0%.
• ALOHA (Continuous Bimanual Tasks):
  • Cube Transfer: 98% success rate (improvement of 12% over ACT).
  • Bimanual Insertion: 56% success rate (improvement of 40% over ACT, which achieved 16%).
  • Efficiency: The system operates at approximately 71 frames per second (FPS), surpassing the ACT baseline (61 FPS) and RT-1 (30 FPS).
• Real-World Deployment: On physical UR5 and ARX R5 robots, GPA-RAM demonstrated robustness in tasks such as block-to-drawer placement (90–100% success) and plate transferring (100% success), outperforming baselines in handling contact-rich insertions and constrained environments.

Significance and Claims
The paper claims that GPA-RAM successfully bridges the gap between precise manipulation and efficient real-time execution. By explicitly modeling grasp priors through GPA, the system mitigates the error propagation caused by inaccurate initial grasps, a common failure mode in prior IL policies. Simultaneously, the RAM architecture resolves the latency bottlenecks associated with Transformer-based methods, enabling high-frequency control suitable for dynamic environments.

The authors emphasize that their approach is modular; GPA can augment existing architectures (demonstrated on both RVT2 and ACT), and RAM provides a scalable alternative to global attention for spatial reasoning. The results suggest that combining explicit grasp priors with efficient state-space modeling is a viable path toward robust, generalizable robotic manipulation without the need for extensive, specialized grasp datasets.