💻 computer science

Remember what you did?: Learning Behavioral Memories for Partially Observable Object Manipulation

This paper introduces the Compressed Action Memory Policy (CAMP), a novel approach that enables robots to master long-horizon, contact-rich manipulation tasks under partial observability by learning a self-supervised, compressed representation of their own action history to implicitly track progress and recover from failures without external supervision.

Original authors: Kuancheng Wang, Seungho Yeom, Jinglin Cao, Yuheng Zhi, Nikhil Shinde, Michael Yip

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kuancheng Wang, Seungho Yeom, Jinglin Cao, Yuheng Zhi, Nikhil Shinde, Michael Yip

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The Robot with Amnesia

Imagine you are trying to solve a puzzle, but every time you make a move, someone covers your eyes for a second. When you open them, you see the puzzle, but you don't remember how you got there. Did you just try to move a piece to the left and fail? Did you already move that piece to the right?

This is the problem robots face in "contact-rich" tasks (tasks where they have to push, slide, or touch things). A robot's camera sees the current picture, but it doesn't know the history of what it just tried.

The Result: The robot gets confused. It might push a block into a wall, realize it's stuck, and then push it into the wall again because it forgot it already tried that. It has "amnesia."

The Solution: CAMP (The Robot's "Mental Notebook")

The authors created a new system called CAMP (Compressed Action Memory Policy). Think of CAMP not as a robot that sees better, but as a robot that remembers better.

Instead of trying to remember every single pixel of the past (which is too much data), CAMP keeps a "mental notebook" of its own actions. It asks itself: "What did I just try to do?"

Here is how it works, broken down into simple steps:

1. The "Compressed" Memory (The Summary)

If you tried to write down every single movement you made in a day, your notebook would be huge and messy. CAMP is smart about this. It uses a mathematical trick (called DCT) to write a summary of its past actions.

Analogy: Imagine you are describing a movie to a friend. You don't list every frame; you say, "First, the hero ran left, then he jumped over a fence, then he fell." You keep the shape of the story but throw away the tiny details.
Why it helps: This summary is small enough to fit in the robot's "brain" but detailed enough to tell it, "Hey, you already tried pushing that block to the left, so don't do it again."

2. The Training (Learning from Mistakes)

CAMP is trained using a "self-supervised" method. This means the robot learns by looking at its own past, not by being told what to do by a human teacher.

The Game: The robot is shown a video of a human doing a task. It tries to guess what the human's past actions were based on the current picture.
The Lesson: If the robot guesses wrong, it learns. Over time, it gets really good at looking at the current scene and saying, "Ah, based on where things are now, I must have tried to push the block here earlier."

3. The "Two-Step" Training (Warm-up then Fine-tune)

The paper found a specific way to teach this that works best:

Warm-up: First, the robot practices only remembering the past actions. It builds a strong memory bank.
Freeze & Learn: Then, they lock that memory in place so it doesn't get confused.
Fine-tune: Finally, they let the robot learn how to use that memory to actually move its arm.

Analogy: It's like a student first memorizing the rules of a game (warm-up), then practicing the game without changing the rules (freeze), and finally learning how to play strategically (fine-tune). If you try to learn the rules and play the game at the exact same time, you get confused and forget the rules.

The Results: From 0% to 70%

The researchers tested this on robots doing tricky tasks, like pushing a "T" shaped block into three different spots without hitting the same spot twice, or finding a hidden button.

Without Memory (Old Robots): They failed almost everything. They would push the block, get stuck, and push it again, going in circles. Success rate: 0%.
With CAMP: The robot remembered, "I already tried the left spot, it didn't work. I'll try the middle one." Success rate: Up to 94% in simulations and 70% on real robots.

Why This Matters

Most robots rely on "Vision-Language-Action" models, which are like asking a human, "Remember to push the red block." But you have to tell them exactly what to remember every time.

CAMP is different. It teaches the robot to remember for itself. It learns that its own history of movements is the most important clue to solving the puzzle. It turns a "partially observable" problem (where you can't see the whole picture) into a "clear" problem by filling in the missing gaps with memory.

Summary

The Problem: Robots forget what they just tried, so they repeat mistakes.
The Fix: CAMP gives the robot a "compressed memory" of its own past actions.
The Magic: It doesn't need a human to tell it what to remember; it learns to remember by trying to reconstruct its own past.
The Outcome: Robots can finally solve complex, multi-step puzzles where they have to learn from failure, just like a human would.

Technical Summary: Compressed Action Memory Policy (CAMP)

Problem Statement

Long-horizon, contact-rich robotic manipulation tasks are inherently partially observable. In these scenarios, a single visual observation rarely captures the robot's full action context, including prior attempts, physical interactions, or the current phase of task progress. Consequently, standard visuomotor policies and Vision-Language-Action (VLA) models suffer from "amnesia," acting on aliased signals that fail to distinguish between states that look identical visually but differ in their interaction history.

Existing solutions face significant limitations:

Memoryless Policies: Fail to track progress or learn from failed attempts in contact-rich environments.
VLA-based Approaches: Often require tedious, non-scalable human prompting to dictate what to remember, or rely on predicting high-dimensional future/past visual observations, which is computationally prohibitive and prone to learning task-irrelevant details.
Visual Memory: Methods relying on visual token memory or keyframe retrieval struggle when the critical information is encoded in physical contact history rather than visual appearance.

Methodology: Compressed Action Memory Policy (CAMP)

CAMP addresses these challenges by leveraging the robot's own action history as a highly informative, self-supervised signal. The core insight is that by learning to reconstruct past actions, a policy can naturally extract and retain only the most critical behavioral information required to disambiguate partially observable states.

The CAMP architecture consists of three primary components:

1. Memory Module Pretraining (Self-Supervised)

A recurrent memory module (implemented as a two-layer LSTM) is pretrained to summarize the interaction history.

Input: The module processes observation-action pairs, specifically utilizing a fixed third-view camera (for consistent spatial reference) and proprioception.
Objective: Instead of memorizing raw action sequences, the module predicts a compressed representation of the past action trajectory using the Discrete Cosine Transform (DCT).
Loss Function: The training objective combines:
- Reconstruction Loss ( $L_{rec}$ ): Supervises the prediction of DCT coefficients in the frequency domain. A frequency weight down-weights high-frequency coefficients, forcing the memory to capture the low-frequency structure (shape) of the trajectory while discarding step-level noise.
- Temporal Consistency ( $L_{cons}$ ): Ensures that the reconstruction of a fixed past action remains consistent as the episode advances, preventing the memory account from drifting.
Outcome: The hidden state $h_t$ becomes a compact, latent summary of behavioral history rather than a raw buffer.

2. Discretization (Vector Quantization)

To improve generalization, the continuous hidden state $h_t$ is discretized using a Vector Quantizer (VQ).

The state is mapped to the nearest entry in a learned codebook.
This discretization forces the memory to encode coarse-grained behavioral states rather than fine-grained trajectory identities, which the authors found crucial for generalization across demonstrations.
The quantized state is projected to a memory code $m_t$ that serves as the sole history signal for the action head.

3. Diffusion Action Head

The action head follows the Diffusion Policy framework, predicting a chunk of future actions.

Conditioning: The diffusion model is conditioned on the concatenation of visual features (from both third-person and wrist cameras), proprioception, and the discretized memory code $m_t$ .
Training Schedule: A staged warm-up-then-finetune schedule is employed. The memory module is pretrained and frozen while the action head learns to adapt to it. Subsequently, the memory is unfrozen and jointly finetuned with the action head. This prevents the action loss from eroding the long-horizon information installed during pretraining.

Key Contributions

Self-Supervised Memory Formulation: Introduces action history reconstruction via DCT compression as a scalable, self-supervised signal to capture task-relevant behavioral memory without external supervision.
The CAMP Architecture: A novel framework integrating a recurrent memory module (trained via DCT action compression) with a diffusion policy, enabling long-horizon, contact-rich manipulation in partially observable environments.
Novel Benchmarks: The release of Memory-T-Bench (2D contact-rich pushing) and Memory-Manip-Bench (3D vision-based manipulation), specifically designed to evaluate memory-dependent tasks involving tracking progress and learning from failure.
Extensive Validation: Empirical demonstration of CAMP's superiority over state-of-the-art baselines across simulation and four real-robot setups.

Experimental Results

Simulation Benchmarks

Memory-T-Bench: CAMP significantly outperforms memoryless Diffusion Policy (DP) on partially observable variants.
- Multi-Goals: 94.0% (CAMP) vs. 56.0% (DP).
- Swap-Shuffle: 92.0% (CAMP) vs. 48.0% (DP).
- On fully observable tasks (standard Push-T), CAMP performs comparably to DP (98.0% vs. 94.0%), demonstrating no penalty for memory when it is not needed.
Memory-Manip-Bench: CAMP achieves an average success rate of 64.3%, surpassing the strongest baseline (DP at 40.9%) by over 23%. It leads on 6 of 7 tasks, with the largest margins on tasks requiring state recall (e.g., Swap-Block: 86.0% vs. 18.0%).
Data Efficiency: CAMP's success rate scales monotonically with the number of demonstrations (42% to 94% as demos increase from 80 to 320), whereas memoryless policies show no consistent gain, indicating the bottleneck for memoryless policies is partial observability, not data volume.

Real-Robot Experiments

Evaluated on four tasks: Push-T-Multi-Goals, Swap-Can, Wipe-Plate, and Probe-Insert.

Baselines: All memoryless baselines (ACT, DP, $\pi^{0.5}$ , MemoryVLA) collapsed to 0/10 success on Push-T-Multi-Goals, Swap-Can, and Wipe-Plate.
CAMP: Achieved 7/10 success on all three tasks and 11/15 on Probe-Insert.
Failure Modes: Baselines failed due to oscillating between goals, stalling after partial swaps, or entering infinite loops (reusing brushes/plates) because they could not track which sub-goals were completed. CAMP successfully resolved these by remembering past interactions.

Ablation Studies

Discretization: Removing vector quantization drops performance (94% $\to$ 60%), confirming that coarse behavioral states generalize better than continuous trajectory identities.
Compression: Replacing DCT with raw waypoints or DFT reduces performance, highlighting the necessity of low-frequency compression.
Training Schedule: Joint training from scratch (66%) or pretraining followed by immediate finetuning (46%) fails. The warm-up-then-finetune schedule is critical to preserving the pretrained memory while allowing it to adapt to the policy.
Reconstruction vs. Utility: The finetuned memory, while achieving higher policy success (94%), has a worse reconstruction loss (1.46) compared to the pretrained memory (0.35). This indicates the policy benefits from a compressed summary aligned with decision-making, not a perfect reconstruction of the past.

Significance and Claims

The paper claims that CAMP is, to the authors' knowledge, the first policy to demonstrate substantial success on contact-rich, partially observable manipulation tasks purely through learned memory.

Key takeaways emphasized by the authors:

Memory resolves partial observability: By compressing action history, CAMP turns partially observed problems into effectively Markovian ones for the policy.
Self-supervision is sufficient: Robots can learn robust behavioral memory from their own action history without needing explicit task labels or human prompting.
Scalability: The approach avoids the computational cost of visual prediction and the scalability issues of manual prompt engineering.

The authors note limitations: the method still struggles with extremely long-horizon memory requirements, and extending it to dexterous manipulation remains an open direction for future work.