💻 computer science

Implicit State Estimation via Video Replanning

This paper proposes a novel video-based planning framework that enables dynamic adaptation to environmental uncertainties by integrating real-time interaction data to update model parameters and filter failed plans, achieving implicit state estimation without explicit modeling.

Original authors: Po-Chen Ko, Jiayuan Mao, Yu-Hsiang Fu, Hsien-Jeng Yeh, Chu-Rong Chen, Wei-Chiu Ma, Yilun Du, Shao-Hua Sun

Published 2026-02-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Po-Chen Ko, Jiayuan Mao, Yu-Hsiang Fu, Hsien-Jeng Yeh, Chu-Rong Chen, Wei-Chiu Ma, Yilun Du, Shao-Hua Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to open a mysterious door in a dark hallway. You reach out, push it, but it doesn't budge. A "standard" robot might just stand there, confused, or try to push it even harder, thinking it just needs more strength.

This paper, "Implicit State Estimation via Video Replanning," describes a smarter way for robots to learn from their mistakes—not by calculating complex math equations, but by "imagining" the future through video.

Here is the breakdown of how it works using everyday analogies.

1. The Problem: The "Stubborn Robot" Syndrome

Most current robots are like people who have memorized a map but have never actually walked the streets. They see a picture of a task and try to execute it perfectly. But if the world changes—if the door is actually a "pull" door instead of a "push" door—the robot gets stuck. It doesn't realize that its "map" (its plan) is wrong because it isn't paying attention to the fact that its last attempt failed.

2. The Solution: The "Mental Movie" Method

Instead of just following a list of coordinates, this framework uses Video Planning.

Think of it like a professional athlete. Before a basketball player takes a shot, they don't just move their arms; they "see" the ball going through the hoop in their mind. This robot does the same. It generates a "mental movie" (a video plan) of what it thinks will happen if it performs an action.

3. The Secret Sauce: Learning from "Ouch!" Moments

The real breakthrough here is how the robot handles failure. The researchers call this Implicit State Estimation.

The Analogy: The "Trial and Error" Chef
Imagine you are cooking a new soup. You add salt, taste it, and think, "Too salty!" You don't need a chemistry degree to know you shouldn't add more salt; you just learned something about the "state" of your soup through that failed taste test.

This robot does exactly that:

The Memory Bank (Retrieval): The robot keeps a library of "videos" of things it has done before. If it encounters a heavy box, it looks through its memory for a video of a similar heavy box to get a "feeling" for how it might move.
The Rejection Filter (The "Not That Again" Rule): This is the cleverest part. When the robot generates several "mental movies" of how to solve a task, it looks at its "Failed Plan Buffer" (a scrapbook of its recent mistakes). If one of its new mental movies looks too much like a movie that ended in a failure, it says, "Nope, not that one!" and picks a different strategy.

4. How it works in practice (The Door Example)

The First Guess: The robot sees a door and imagines a video of itself pushing it. It executes the push.
The Failure: The door stays shut. The robot records this "failed interaction" video.
The Update: The robot looks at its memory. It realizes, "Wait, my 'pushing' movie resulted in a stuck door. Let me look for a different kind of movie."
The New Plan: It generates a new mental movie of itself pulling the door. Because this movie looks nothing like the "failed push" movie, the robot selects it and successfully opens the door.

Summary: Why does this matter?

In the past, if a robot didn't know the weight of an object or the friction of a floor, a human had to program those specific numbers into its brain.

This paper shows that if we give a robot the ability to imagine outcomes and remember its failures, it can figure out the "rules" of the world on its own, just by watching and trying. It turns a robot from a rigid machine into an adaptive learner that can navigate the messy, unpredictable real world.

Technical Summary: Implicit State Estimation via Video Replanning

Implicit State Estimation via Video Replanning presents a novel framework for robotic decision-making that enables agents to adapt to unknown environmental dynamics (such as mass, friction, or interaction modes) using video-based planning. Unlike traditional methods that require explicit system identification or hand-specified belief models, this approach leverages past interaction failures to implicitly refine its understanding of the world.

1. The Problem: Uncertainty and Partial Observability

Existing video-based planning frameworks (which generate imagined future trajectories to guide robot actions) suffer from a critical limitation: they are "memoryless" regarding failures.

In environments with partial observability—where a robot cannot visually determine if a door should be pushed or pulled, or how heavy an object is—current planners often repeat the same incorrect strategy. When a plan fails, these systems typically generate a new plan from the same initial observation, failing to incorporate the "knowledge" gained from the failed interaction. This leads to suboptimal performance and high failure rates in unstructured or uncertain environments.

2. Methodology: The ISE Framework

The authors propose Implicit State Estimation (ISE), a framework that integrates interaction-time feedback into the planning loop without explicitly modeling unknown physical parameters ( $\theta$ ). The architecture consists of three core components:

A. Video-based Planning with State Embeddings:
- State Embeddings: The system learns a latent representation (a vector) for each object that summarizes its physical properties. These are constructed by encoding interaction videos using a vision encoder (e.g., DINOv2 or CLIP).
- Retrieval & Refinement: At test time, when the robot encounters an object, it retrieves the most similar state embedding from a database of past successful interactions. To handle misalignment, it uses a Generative Identification Module to optimize (refine) this embedding via gradient descent, minimizing the discrepancy between the generated video and the actual observed interaction.
- Video Plan Generator: A diffusion-based model that predicts a sequence of future frames conditioned on the current observation and the refined state embedding.
B. Rejection-based Replanning:
- To prevent the robot from repeating mistakes, the framework maintains a Failed Plan Buffer.
- Instead of generating one plan, the system samples $N$ candidate plans using different retrieved embeddings.
- A Rejection Module selects the candidate plan that is most visually dissimilar (using raw-pixel or feature distance) to the previously failed plans. This forces the agent to explore alternative hypotheses (e.g., "if pushing failed, try pulling").
C. Action Module:
- The selected video plan is converted into low-level robot commands using a training-free method (e.g., point-tracking or optical flow), allowing the robot to execute the "imagined" trajectory in the real world.

3. Key Contributions

Implicit vs. Explicit Estimation: Proposes a way to resolve environmental uncertainty through latent embedding optimization rather than explicit parameter estimation.
Integration of Failure Data: Introduces a mechanism to treat failed interactions not as setbacks, but as critical data points for updating the agent's "belief" via the retrieval and rejection modules.
Meta-World System Identification Suite: Introduces a new benchmark designed specifically to test online adaptation to hidden physical parameters (mass, friction, interaction modes).

4. Results and Evaluation

The framework was evaluated on the new Meta-World suite and real-world door-opening tasks:

Efficiency: The ISE framework significantly reduced the average number of replans required to achieve success compared to standard video planning (AVDC), Behavior Cloning (BC), and Offline RL (CQL).
Robustness: In real-world experiments (opening a door that could be pushed or pulled), the method achieved a 75% success rate (15/20), whereas a naive video planner only achieved 40% (8/20).
Ablation Success: Experiments confirmed that both the Retrieval Module (to find the right hypothesis) and the Rejection Module (to avoid wrong hypotheses) are essential for high performance.
Scaling: The method demonstrated strong scaling properties, performing better as the amount of offline experience data increased.

5. Significance

This work represents a significant step toward adaptive, generalizable robotics. By bridging the gap between modern generative video models and classical system identification, the authors provide a path for robots to learn from trial-and-error in a way that mimics human intuition. The ability to adapt to "unseen" physical properties through visual feedback alone makes this framework highly applicable to real-world deployment in unstructured environments where perfect modeling is impossible.