Technical Summary: ReSYNC – Recovery-Driven Synthesis of Relational Concepts

1. Problem Statement

Intelligent robots operating in complex, "big worlds" inevitably encounter failures (e.g., slipping on ice, getting trapped in mud, or failing to grasp an object due to occlusion). While existing approaches can learn reactive recovery skills to escape specific failure states, they suffer from poor scalability. Training a separate policy for every distinct failure mode is inefficient, and purely reactive methods treat failures as isolated events rather than exploiting their shared structural patterns.

The core challenge addressed by this work is how to enable robots to not only recover from failures but also autonomously acquire the abstract knowledge (concepts and skills) necessary to avoid them in the future. Specifically, the paper seeks to bridge the gap between local, reactive recovery and global, proactive planning by learning relational concepts (predicates) and skills jointly from failure-recovery experience, without requiring hand-designed abstractions or explicit supervision.

2. Methodology: ReSYNC

The authors propose Recovery-Driven Synthesis of Relational Concepts (ReSYNC), a framework that progressively discovers and refines state abstractions from failure-recovery experience. ReSYNC operates through an incremental, dual-learning process driven by a staged training curriculum designed to elicit failures.

2.1 Core Components

• Relational Factored MDP: The environment is modeled as a Relational Factored Markov Decision Process where states are factored by typed objects with continuous features.
• Dual-Learning Loop: ReSYNC alternates between two phases at each stage of the curriculum:
  1. Skill Learning (Recovery): The robot uses Reinforcement Learning (RL) to learn a recovery skill ($\pi_{C_N}$) that transitions the system from a failure state to a state where the original planner can resume.
  2. Concept Learning (Abstraction): The robot discovers new relational predicates ($\Psi$) and refines its abstract planning model (operators) to explain the learned recovery behavior, enabling the planner to reuse the skill in new contexts.

2.2 Key Technical Mechanisms

A. Failure Mining and Skill Learning

• Mining: When a failure occurs, ReSYNC identifies the failure object ($o_F$) and a relevant object ($o_R$). It fits pairwise Gaussian distributions over their relative states to resample failure scenarios, expanding a small set of observed failures into a larger dataset ($D_F$).
• RL Training: A recovery MDP is constructed where the goal is to reach a state from which the symbolic planner can succeed. Using PPO, the robot learns a relational recovery policy and a terminal function. This new skill is added to the library but initially lacks a symbolic description.

B. Concept Learning via "Dreaming"

To make the new skill usable by the symbolic planner, ReSYNC must discover predicates (e.g., IsOpen) and operators that ground the skill.

• The Data Gap: Standard recovery trajectories only show the skill being used in the original failure context (e.g., pulling a drawer to grasp a hammer). They lack examples of the skill being reused in different contexts (e.g., pulling a drawer to place an object).
• Dreaming: ReSYNC employs a self-supervised data generation process called "Dreaming." It creates counterfactual tasks by resampling failure objects relative to new reference objects. The robot plans in these counterfactuals, invokes the learned recovery skill upon failure, and records the successful transitions. This generates diverse data (e.g., pulling a drawer near a target object) that allows the system to disambiguate general concepts (like IsOpen) from specific, coincidental states (like HammerInOpenDrawer).
• IVNTR Integration: The system uses the IVNTR framework (a bilevel learning approach) to jointly discover predicate symbols and train their neural classifiers using the dreamed data.

C. Progressive Learning and Refinement

• Compositional Dreaming: As training progresses, the dreaming process becomes compositional, combining failure objects and state distributions from previous stages to generate increasingly diverse trajectories.
• Concept Fine-tuning: Instead of relearning all concepts from scratch, ReSYNC retains previously learned predicates and incrementally expands the library. Crucially, it fine-tunes the classifiers of existing concepts using new data to prevent "semantic drift" (where new skills alter state distributions, making old predicates inaccurate).

3. Key Contributions

1. First Joint Learning of Skills and Concepts: ReSYNC is the first approach to jointly synthesize skills and relational concepts directly from task-driven interactions without explicit supervision, relaxing the assumptions of prior work that required fixed skill libraries or hand-designed concepts.
2. Recovery-Driven Abstraction: It introduces a mechanism to convert local, reactive recovery experiences into global, proactive planning capabilities by discovering abstract predicates that generalize recovery skills.
3. Self-Supervised "Dreaming": The paper proposes a novel data generation technique ("Dreaming") that creates counterfactual scenarios to enable the discovery of reusable concepts, addressing the data scarcity inherent in online skill learning.
4. Sim-to-Real Transfer: The framework is demonstrated on real-world hardware, successfully transferring learned non-prehensile manipulation skills and abstract planning capabilities to physical robots.

4. Experimental Results

The authors evaluated ReSYNC across four simulated domains (Icy Transport, Blocked Stacking, Cluttered Drawer, Rearrange) and two real-world domains (Cornered Insertion-S and L).

• Generalization: On compositional novel tasks (unseen test distributions), ReSYNC achieved an average success rate of 70%, significantly outperforming strong baselines (Recovery Chaining, Deep Skill Graphs, VLM planners, and hierarchical RL) by 20–70%.
• Sample Efficiency: ReSYNC solved training tasks with approximately 22x fewer episodes compared to Test-Time Recovery Learning (TTRL) baselines, demonstrating the efficiency of reusing learned concepts for symbolic reasoning.
• Progressive Learning: The system successfully learned from multiple curriculum stages without catastrophic forgetting, maintaining high performance on earlier tasks while adapting to new complexities.
• Real-World Performance: In real-world experiments, ReSYNC learned a non-prehensile recovery skill (repositioning a stuck block) and generalized it to new scenarios via discovered predicates, outperforming reactive baselines that could not anticipate failures.

5. Significance and Claims

The paper claims that ReSYNC represents a significant step toward robots that autonomously acquire abstractions for scalable, failure-aware planning in the physical world.

• Shift from Reactive to Proactive: The work argues that treating failures as opportunities for discovery allows robots to move beyond reactive recovery to proactive failure avoidance through abstract planning.
• Scalability: By learning relational concepts rather than specific policies for every failure, the approach offers a path to scaling robot intelligence to "big worlds" where failure configurations are combinatorial.
• Autonomy: The system reduces the need for manual engineering of skills and concepts, relying instead on the robot's own interaction history to build its planning model.

The authors acknowledge limitations, including the reliance on a user-specified curriculum for failure elicitation and the use of Gaussian distributions for state sampling, suggesting future work could automate curriculum design and utilize advanced generative models for state sampling.