💻 computer science

Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization

This paper introduces Instance-Oriented Memory (IOM), an object-centric framework that amortizes exploration costs for manipulating objects with hidden states by recording and reusing short manipulation procedures via a vision-language model, thereby significantly reducing redundant probing while maintaining or improving task success rates.

Original authors: Haizhou Ge, Haochen Ouyang, Zhixing Chen, Yufei Jia, Yue Li, Lu Shi, Lei Han, Guyue Zhou, Ruqi Huang

Published 2026-07-28

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Haizhou Ge, Haochen Ouyang, Zhixing Chen, Yufei Jia, Yue Li, Lu Shi, Lei Han, Guyue Zhou, Ruqi Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot arm trying to open a microwave. It doesn't know if the door is latched or free until it actually tries to pull it. If the latch is stuck, the robot has to fiddle with a handle first, then pull. If the latch is already free, that handle-twisting is just wasted effort. This is the daily struggle for robots in a world full of "hidden states"—things like locked doors, unlatched cabinets, or caps that are already unscrewed. Robots have to poke and prod to figure out what's going on, which is slow and clumsy. Scientists in the field of robotics are trying to teach machines to stop guessing and start remembering. The big question is: if a robot solves a puzzle once, can it remember the solution so it doesn't have to solve the same puzzle from scratch every single time?

This paper introduces a clever new trick called Instance-Oriented Memory (IOM). Think of it like a robot's "sticky note" system for specific objects. Usually, robots remember general rules like "how to open a door." But this new system remembers this specific door and exactly how it behaves. The researchers found that by recording a short, efficient "cheat sheet" after the robot figures out a hidden state just once, they can cut the number of unnecessary moves by 16% to 30% on future tries. Even cooler, they tested this using a pre-made AI tool (a Vision-Language Model) that didn't need any special training to learn the trick. The robot learned once, wrote the note, and then skipped the boring part every time after, all without ever failing at the task.

The "Try Once, Then Optimal" Story

Imagine you are a robot chef trying to open a microwave. The first time you walk up to it, you don't know if the latch is stuck or free. So, you play it safe: you reach out, try to twist the handle, and then pull the door. If the latch was free, that twist was a total waste of time. If it was stuck, you had to do it. Either way, you learned something: "This specific microwave needs a twist."

Now, imagine you have to open that same microwave every morning for a year. A "dumb" robot would repeat that twist-and-pull routine every single day, just in case. It's like checking your pockets for your keys every morning even though you know you left them on the table. It's safe, but it's inefficient.

The researchers behind this paper, Haizhou Ge and their team, asked: "Why does the robot keep re-exploring?" They realized that existing robot memories are like a library of "successful stories." They help the robot remember how to succeed, but they don't help it remember which object it is facing to skip the unnecessary steps.

Their solution is Instance-Oriented Memory (IOM). It's a three-step process that turns a robot from a forgetful explorer into a smart rememberer.

Step 1: The "Try Once" (Exploration)

The robot encounters a new object, say a microwave with a hidden latch. It doesn't know the state, so it has to probe. It tries a sequence of moves. Maybe it fails, maybe it succeeds, but crucially, it reveals the secret. It finds out, "Ah, this latch is stuck, I need to twist first."

Step 2: The "De-Redundified" Cheat Sheet (Distillation)

Here is the magic. The robot takes that long, messy sequence of moves (twist, pull, maybe a retry) and strips away the fluff. It realizes, "Oh, I know this microwave is stuck now. I don't need to check if it's free; I just need to twist and pull." It writes down a tiny, perfect instruction: "Twist, then Pull." It saves this note in a special memory book, labeling it with a picture of this specific microwave.

Step 3: The "Recall" (Amortization)

The next day, the robot sees the same microwave. Instead of poking around to see if the latch is stuck, it looks at its memory book. It recognizes the microwave, pulls up the "Twist, then Pull" note, and goes straight to work. It skips the "checking" phase entirely.

The paper calls this "amortizing exploration." It's like paying for a movie ticket once and getting to watch the movie every day after without buying a new ticket. The "cost" of figuring it out is paid once, and the savings are reaped every time after.

How They Tested It (The Lab vs. The Real World)

The team didn't just talk about this; they built it and tested it in four different scenarios:

Microwave (in a computer simulation)
Door (in a computer simulation)
Bottle (on a real robot arm)
Cabinet (on a real robot arm)

In all these tasks, the robot had to deal with hidden states: Is the door locked? Is the bottle cap already off? Is the cabinet latch engaged?

They compared three types of robots:

The "Random" Robot: It has no memory. It tries to open things from scratch every time, often doing unnecessary steps.
The "Oracle" Robot: This robot has a magical cheat sheet that knows the answer perfectly. It represents the absolute best possible performance.
The "VLM" Robot: This is the star of the show. It uses a standard, off-the-shelf AI (a Vision-Language Model) to look at the video of the first attempt, figure out the trick, and write the note. It didn't get any special training for this specific job; it just used its general smarts.

The Results: Less Wiggling, More Success

The numbers are pretty impressive. When the robots had to open these objects again and again:

The Oracle Robot (the perfect one) cut the number of moves down by 16% to 30% compared to the robot that re-explored everything.
The VLM Robot (the one using the standard AI) managed to capture 69% to 88% of that saving. In other words, using a pre-made AI tool, the robot got almost all the benefits of a perfect memory without needing to learn anything new.

On the real robot arm, the results were even better than in the simulation. The robots that used the memory system didn't just save time; they actually became more successful at opening things than the ones that didn't use memory.

What If the Memory is Wrong?

A big worry with memory systems is: "What if the robot remembers the wrong thing?" What if it thinks the microwave is stuck, but it's actually free? If the robot blindly follows a wrong note, it might break the door.

The researchers designed IOM to be a "soft bias." This is a fancy way of saying the memory is a suggestion, not a command. The robot still listens to its sensors. If the note says "Twist," but the door opens easily without twisting, the robot's feedback loop kicks in and it stops twisting.

They tested this by giving the robot a wrong note on about 12% of the door instances. The result? The robot didn't fail. It just did a few extra moves to correct itself. The success rate stayed the same, proving that the system is safe. It's like having a GPS that suggests a route, but if you see a roadblock, you just turn and keep driving.

Why This Matters

The paper argues that we shouldn't just teach robots to be better at doing tasks; we should teach them to be better at remembering specific objects. Current robot memories focus on "Did I succeed?" but this new system focuses on "What is this object, and how do I handle it?"

By treating every object as a unique individual with its own history, the robot stops wasting energy on "probing" things it already knows. The authors found that this approach works in both computer simulations and on real, physical robots. They even noted that for some tricky objects, like a bottle where the cap looks almost the same whether it's on or off, the robot can't "see" the difference. But by remembering the state from the first interaction, it can skip the visual guesswork entirely.

In short, this paper shows that robots can learn to be efficient by keeping a simple, object-specific diary. They try once, write down the trick, and then breeze through the rest of their lives. And the best part? They can do this using tools we already have, without needing to build a whole new brain for every single job.

Technical Summary: Instance-Oriented Memory for Cross-Episode Exploration Amortization

1. Problem Statement

Robots manipulating articulated objects with hidden internal states (e.g., a latched microwave, a locked door, or a bottle with an unscrewed cap) face a fundamental inefficiency: they must "probe" the object to reveal its state before acting. While a robot that has successfully manipulated a specific instance once learns the necessary state, existing cross-episode memory systems fail to leverage this knowledge.

Current approaches to cross-episode memory primarily target task success and robustness. They organize memories around states or successful trajectories, retrieving action snippets or priors based on the current observation to improve the probability of success. However, they do not recognize recurring object instances to skip the redundant exploration phase. Consequently, a robot re-explores the same hidden state every time it encounters the same object, incurring pure operational overhead.

The paper identifies a gap: there is no mechanism to amortize exploration across repeated encounters with the same object instance. The goal is to transition from a "probe-every-time" strategy to a "probe-once, recall-later" strategy without compromising success rates.

2. Methodology: Instance-Oriented Memory (IOM)

The authors propose Instance-Oriented Memory (IOM), an object-centric framework designed to distill and reuse manipulation procedures. IOM is modular, consisting of three interchangeable components:

A. Procedure Distillation

Upon the first encounter with an object (whether the attempt succeeds or fails), the system records the full interaction sequence. A distillation module analyzes this sequence, the task prior, and the revealed hidden state to extract a de-redundified procedure.

Input: An encounter trajectory (observations and actions) and a task prior (instructions, candidate set).
Output: A minimal sequence of operations ( $\hat{K}$ ) that completes the task without redundant probes.
Mechanism: The module strips away "retries" (operations that were necessary only because the state was unknown at the time). Crucially, distillation relies on revelation of the state, not just task completion; a failed attempt that reveals the state yields the same optimal procedure as a successful one.

B. Instance-Keyed Memory

The distilled procedure is stored in a memory module keyed by the object's identifiable features ( $\phi(x)$ ), such as its visual appearance.

Write: After the first encounter, the system writes the procedure $\hat{K}$ to the memory at address $\phi(x)$ .
Read: On subsequent encounters, the system queries the memory with the object's features.
- Hit: If the object is recognized, the stored procedure $\hat{K}$ is retrieved.
- Miss: If the object is new, the system defaults to an "uncovering probe" (a sequence guaranteed to reveal the state), initiating the distillation process again.

C. Procedure-Conditioned Policy

The retrieved procedure is not executed as a rigid script but injected as a soft bias into a history-dependent policy.

Conditioning: The policy $\pi$ takes the current observation, history, and the retrieved procedure $K$ as inputs.
Soft Bias: The procedure guides the initial actions but does not override the policy's feedback loop. If the retrieved procedure is incorrect (e.g., the object's state has changed or the memory is wrong), the policy relies on interaction feedback to recover and retry, ensuring that success rates do not regress.

Instantiation

The authors instantiate this framework using off-the-shelf Vision-Language Models (VLMs) without task-specific training:

Distillation: A VLM parses the video of the first encounter and the task description to generate the de-redundified procedure text.
Memory: The VLM acts as the associative store, matching the visual appearance of the object in new episodes to the stored procedure.
Policy: The authors adapt the AdaManip policy (using flow matching) to accept the VLM-generated procedure as a language conditioning input.

3. Key Contributions

The IOM Framework: A novel object-centric approach to cross-episode memory that targets operational efficiency rather than just success rates. It isolates the value of memory by demonstrating that storing a single de-redundified procedure can significantly reduce manipulation operations on repeat encounters.
Training-Free Instantiation: The authors demonstrate that a general-purpose VLM can perform both procedure distillation and instance recognition "out of the box," requiring no fine-tuning or task-specific training data.
Robustness via Soft Bias: The framework ensures that incorrect memories do not lead to failure. Because the procedure acts as a soft bias on a feedback-driven policy, the robot can recover from wrong recollections, maintaining high success rates even when the memory is imperfect.

4. Experimental Results

The system was evaluated on four articulated-object tasks: two in simulation (microwave, door) and two on a real robot (bottle, cabinet).

Efficiency Gains:
- An Oracle memory (perfectly distilled procedures) reduced manipulation operations by 16–30% across all tasks compared to a baseline that re-explores every time.
- The VLM-based instantiation recovered 69–88% of the oracle's savings without any task-specific training.
Success Rates:
- Success rates did not regress with the introduction of memory. In real-robot experiments, success rates actually improved (e.g., from 0.70 to 0.90 for the cabinet task), likely because the memory helped avoid the harder, locked instances that caused failures in the random baseline.
Robustness:
- In cases where the VLM recalled a wrong procedure (e.g., ~12% of door instances), the system successfully recovered via retries. The "cost" of the error was limited to extra operations, not task failure.
Amortization:
- The efficiency gains were realized specifically on repeat encounters. The first encounter always requires exploration, but subsequent encounters skip the redundant probing steps.

5. Significance and Claims

The paper claims that IOM addresses a specific inefficiency in robotic manipulation: the repetitive cost of probing hidden states for known objects.

Shift in Memory Paradigm: Unlike existing memories that organize by state to improve success, IOM organizes by object instance to improve efficiency.
Practicality: The use of off-the-shelf VLMs suggests that complex procedural memory can be implemented without expensive, task-specific training pipelines.
Safety and Reliability: The "soft bias" mechanism is critical; it allows the system to benefit from memory while retaining the safety net of exploration if the memory is wrong.
Limitations: The authors acknowledge that the system relies on the object's identifiable features being unique enough to distinguish between instances requiring different procedures (avoiding perceptual aliasing). If two visually similar objects require different procedures, the current keying mechanism may recall the wrong one. They suggest future work could expand the key to include broader context (e.g., surrounding scene) to resolve this.

In summary, the paper demonstrates that robots can learn to "try once, then optimize" by distilling a single successful (or revealing) interaction into a reusable, object-specific procedure, significantly reducing the operational cost of manipulation without sacrificing reliability.