Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning

This paper introduces MIKASA, a comprehensive benchmark suite featuring a classification framework and two distinct datasets (MIKASA-Base and MIKASA-Robo) to systematically evaluate and advance memory-enhanced reinforcement learning agents across diverse scenarios, with a specific focus on tabletop robotic manipulation.

The Gist

Imagine you are trying to teach a robot to do chores around the house. You might tell it, "Go to the kitchen, find the red cup under the towel, and bring it to me."

If the robot has a "short-term memory" like a goldfish, it will see the red cup, but the moment the towel covers it, the robot forgets it exists. It will wander around confused, asking, "Where did the cup go?"

This paper introduces a new way to test how good robots (and AI agents) are at remembering things, especially when they can't see them all the time. The authors call their new testing suite MIKASA.

Here is a simple breakdown of what they did and why it matters:

1. The Problem: The "Goldfish" Robot

Right now, many AI benchmarks (tests for AI) are like testing a human's memory by asking them to solve a math problem they can see on a piece of paper. But real life is different. Real life is like a game of Shell Game (where a ball is hidden under one of three cups).

• The Issue: If a robot can't remember which cup the ball is under once it's covered, it fails.
• The Gap: Scientists didn't have a standard "report card" to test if a robot is good at remembering different kinds of things (like locations, colors, or sequences of events). Every researcher built their own tiny test, making it impossible to compare robots fairly.

2. The Solution: The MIKASA "Memory Gym"

The authors built a massive, standardized gym called MIKASA (Memory-Intensive Skills Assessment Suite for Agents). Think of it as a Universal Driver's License Test for robot memory.

They organized the tests into four main categories, using simple analogies:

• Object Memory (The "Where's Waldo" Test):
  • The Task: A robot sees a red ball, then a cup covers it. The robot must remember the ball is still there and touch the right cup.
  • Real Life: Finding your keys that you put under a magazine.
• Spatial Memory (The "Mental Map" Test):
  • The Task: A robot sees a peg in a specific spot, then has to rotate it to a new angle without losing track of where it started.
  • Real Life: Walking through your house in the dark because you remember where the furniture is.
• Sequential Memory (The "Recipe" Test):
  • The Task: The robot sees a red cube, then a blue one, then a green one. Later, it has to pick them up in that exact order.
  • Real Life: Remembering the steps to bake a cake: flour, then eggs, then sugar.
• Memory Capacity (The "Grocery List" Test):
  • The Task: The robot has to remember the colors of seven different cubes shown all at once, then pick them all out later.
  • Real Life: Trying to remember a 7-digit phone number you just heard.

3. The "Reality Check": Robots Are Bad at This

The authors tested many of the smartest robots and AI models currently available (including some that can talk and see, called VLA models) on these 32 new tasks.

The results were humbling:

• In a perfect world (Full Memory): When the robot could see everything perfectly (like a video game with no fog), the robots were amazing. They got 100% of the tasks right.
• In the real world (Partial Memory): As soon as they had to remember something because it was hidden or happened in the past, the robots crashed and burned.
  • Even the "smartest" models, which can write poetry and chat, failed to remember a simple color if it was hidden for just a few seconds.
  • It's like a super-genius who can solve complex physics equations but forgets your name five seconds after you introduce yourself.

4. Why This Matters

The paper argues that for robots to truly help us in the real world (cleaning, cooking, caring for the elderly), they need better memory, not just better eyes or stronger arms.

Currently, robots are like amnesiacs. They can react to what they see right now, but they struggle to connect the dots between what happened a moment ago and what they need to do next.

The Takeaway:
MIKASA is a new tool that forces AI researchers to stop building robots that only live in the "now." It pushes them to build robots that can actually remember the past, so they can handle the messy, hidden, and complex tasks of real life.

In short: We are building robots with great eyes but terrible memories. MIKASA is the test that proves it, and the roadmap to fix it.

Technical Summary

Here is a detailed technical summary of the paper "MEMORY, BENCHMARK & ROBOTS: A BENCHMARK FOR SOLVING COMPLEX TASKS WITH REINFORCEMENT LEARNING" (MIKASA).

1. Problem Statement

Reinforcement Learning (RL) agents often struggle with Partially Observable Markov Decision Processes (POMDPs), where the agent lacks full state information and must rely on historical data to make decisions. While memory mechanisms are crucial for real-world robotics (e.g., tracking occluded objects, multi-step manipulation), the field lacks a unified, standardized benchmark to evaluate memory capabilities.

• Fragmentation: Existing benchmarks (e.g., POPGym, DMLab-30, MemoryGym) focus on specific domains (navigation, abstract puzzles) or specific memory types, making cross-study comparisons difficult.
• Robotics Gap: Most robotic benchmarks treat tasks as fully observable MDPs or use artificial noise, failing to capture the complex spatio-temporal dependencies and occlusion challenges inherent in real-world manipulation (e.g., a robot needing to remember an object hidden under a cloth).
• Evaluation Limitations: Current evaluations often fail to distinguish between an agent's inability to perceive versus its inability to remember, and existing methods often rely on "shortcuts" (e.g., large action chunks) rather than true long-horizon memory.

2. Methodology

The authors propose MIKASA (Memory-Intensive Skills Assessment Suite for Agents), a comprehensive framework consisting of a taxonomy, a general RL benchmark, and a specialized robotic benchmark.

A. Taxonomy of Memory Tasks

The paper introduces a classification framework dividing memory-intensive tasks into four distinct categories to ensure systematic evaluation:

1. Object Memory: Tracking object properties (e.g., color, shape) when occluded (Object Permanence).
2. Spatial Memory: Remembering locations and navigating based on past observations.
3. Sequential Memory: Recalling ordered sequences of events or actions.
4. Memory Capacity: Managing multiple pieces of information simultaneously (working memory span).

B. MIKASA-Base (General RL Benchmark)

A unified, Gymnasium-based framework consolidating existing open-source memory tasks (e.g., from POPGym, MiniGrid, MemoryGym) under a consistent API. It serves as a diagnostic tool to isolate specific memory mechanisms before moving to complex visual inputs.

C. MIKASA-Robo (Robotic Manipulation Benchmark)

The core contribution is a suite of 32 robotic manipulation tasks built on the ManiSkill3 simulator.

• Design: Tasks are "atomic" (isolating memory from complex physics/perception) but realistic in their partial observability.
• Observation Modes: Supports State (full info), RGB (visual only), and RGB+joints (standard memory testing configuration).
• Reward Structures: Includes both Dense (continuous feedback) and Sparse (binary success only) rewards to test robustness.
• Task Examples:
  • ShellGame: Memorize a ball's location under cups, then interact with the correct cup.
  • RememberColor/Shape: Memorize an object's attribute, wait for occlusion, then select it from distractors.
  • Intercept: Predict the trajectory of a moving object and catch it.
  • TakeItBack: Move an object to a target, then return it to its original (unobserved) position.

3. Key Contributions

1. Memory Task Classification: A formal taxonomy categorizing memory tasks into Object, Spatial, Sequential, and Capacity types, bridging cognitive science concepts with RL.
2. MIKASA-Base: A standardized, open-source framework for evaluating memory in general RL, enabling fair comparisons across different algorithms.
3. MIKASA-Robo: The first large-scale, open-source benchmark (MIT License) specifically for memory-intensive robotic manipulation, featuring 32 tasks across 12 categories.
4. Datasets: Release of expert-quality datasets (1,000 successful trajectories per task) for Offline RL and Imitation Learning research.
5. Real-World Validation: Experiments conducted on a physical SO-101 robot arm to verify simulation-to-real transfer.

4. Experimental Results

The authors evaluated a wide range of baselines, including Online RL, Offline RL, and Vision-Language-Action (VLA) models.

A. Online RL (PPO, SAC, TD-MPC2)

• State Mode: PPO-MLP achieves 100% success rate when given full state information, proving the tasks are solvable and the environments are well-calibrated.
• RGB+joints Mode (Memory Required):
  • MLP (No Memory): Fails completely (near 0% success) on tasks requiring memory.
  • LSTM (Memory): Outperforms MLP on simple tasks (e.g., 3 colors) but performance collapses as complexity increases (e.g., 5 or 9 colors, sequential tasks).
  • Sparse Rewards: Even LSTM-based agents fail to learn most tasks under sparse reward conditions, highlighting the extreme difficulty of memory-intensive manipulation without dense feedback.

B. Offline RL (Decision Transformer, RATE, BC, CQL, Diffusion Policy)

• Trained on 1,000 expert trajectories per task using sparse rewards.
• Result: None of the models, including those with explicit memory (RATE, DT), could solve the majority of the 32 tasks. Success rates were near zero for high-capacity and sequential memory tasks.
• Implication: Current offline RL methods struggle significantly with long-horizon dependencies in partially observable robotic settings.

C. Vision-Language-Action (VLA) Models (Octo, OpenVLA, $\pi_0$, SpatialVLA)

• Evaluated on tasks like ShellGame and RememberColor.
• Findings:
  • Models perform well on fully observable tasks (Task 1).
  • Performance degrades significantly when the task introduces occlusion and requires long-horizon memory (Task 3).
  • Chunking Limitation: Increasing action chunk sizes ($K=8$) allows models to "guess" trajectories based on early cues but fails when the task requires strict sequential recall or when chunks are small.
  • Real-World: Experiments on a physical robot confirmed simulation results: VLA models fail to retain information across occluded intervals, confirming that memory, not perception or embodiment, is the primary bottleneck.

5. Significance and Future Impact

• Standardization: MIKASA provides the first unified framework to objectively compare memory mechanisms across different RL architectures and domains.
• Identifying Gaps: The benchmark reveals that state-of-the-art models (including Transformers and VLAs) lack robust long-horizon memory capabilities, failing even on simple robotic tasks when occlusion is introduced.
• Research Direction: The results underscore the urgent need for new architectures that explicitly model long-term memory and temporal reasoning, rather than relying on implicit context windows or short-term recurrence.
• Reproducibility: By releasing code, datasets, and detailed customization guides, MIKASA lowers the barrier for researchers to develop and test memory-enhanced agents for real-world robotics.

In conclusion, MIKASA demonstrates that while RL agents can solve fully observable manipulation tasks, they currently lack the robust memory mechanisms required for complex, partially observable real-world scenarios. The benchmark serves as a critical tool for advancing the next generation of memory-centric robotic agents.