Technical Summary: OPINE-World

Problem Statement

Building agents that adapt to unfamiliar tasks requires learning how an environment behaves from interaction. While deep network-based world models offer flexibility, they are data-hungry and transfer poorly beyond their training distribution. Conversely, program-synthesized world models are data-efficient and reusable but have historically been limited to structured-state environments with a pre-defined object vocabulary. A single program search does not scale to pixel-rendered environments where the object structure itself must be hypothesized flexibly from raw observations.

The core challenge addressed by this work is learning an object-centric programmatic world model online from interaction in an environment where:

1. The object vocabulary and partition into mechanical types are unknown.
2. The goal and action semantics are withheld.
3. The agent must infer the object structure, dynamics, and goal relevance solely from raw pixel frames and sparse rewards.

Methodology: OPINE-World

OPINE-World is an LLM agent that learns an object-centric programmatic world model through a loop of hypothesis and test. It couples two cooperating agents and a periodic critic, operating over a shared replay buffer of interactions.

1. Object-Centric Programmatic Model

The world model is represented as a Python program factored by object type. Instead of a monolithic transition function, the model holds a response rule $f_\tau$ for each mechanical type $\tau$. This rule predicts the updated state of an object based on its current attributes, the action taken, and a read-only local context. This factorization allows a rule learned from one object to transfer to all objects of the same type, provided the type partition is approximately correct.

2. Two Cooperating Agents

The system decouples acting from modeling into two specialized LLM agents:

• The Action Agent (Goal-Directed): Interacts with the live environment, proposes hypotheses about dynamics and goals, and probes them. It maintains a natural-language world model and selects actions based on the current synthesized model or exploration needs.
• The Synthesizer Agent: Reads the shared interaction buffer and the action agent's hypotheses. It synthesizes code (the world model) that generalizes observed transitions into rules. It submits candidate programs to a verifier.

3. Verification and Counterexample-Guided Synthesis (CEGIS)

A candidate program is admitted only if it reproduces every recorded transition in the buffer exactly (exact replay test).

• Acceptance: The program must predict the next state attribute-for-attribute and object-for-object. It is also run twice to ensure determinism (rejecting models dependent on hidden state).
• Repair: Synthesis is triggered by counterexamples—transitions where the live model mispredicts the observed outcome. A deferral window allows counterexamples to accumulate before a rewrite, and a stall guard prevents infinite loops on unfixable errors.
• Goal Recognition: Once a level is cleared, a Boolean goal predicate is synthesized and verified against the same exact-replay standard.

4. Ontology Error and Exploration Steering

Since the object partition is unknown, OPINE-World employs a Bayesian diagnostic called ontology error ($\eta_t$) to steer exploration.

• Effect Signatures: Instead of tracking exact value deltas, the system tracks coarse categorical signatures of changed attributes (e.g., "x changed," "pixels changed," "gone").
• Local Effect Tables: These tables count signatures per row (defined by type, action, and local context).
• Uncertainty Measures: The system calculates two normalized entropies:
  1. Type Uncertainty ($U_{type}$): How unsure the system is about an object's mechanical type.
  2. Row Uncertainty ($U_{row}$): How mixed the effect distribution is for a specific type-action-context combination.
• Ontology Error: The noisy-OR of these uncertainties ($\eta_i = 1 - (1-U_{type})(1-U_{row})$) identifies objects and contexts the current types cannot yet explain. The action agent is steered to explore high-error regions, prompting the synthesizer to refine types or add context features.

5. Planning

Once a model is admitted and at least one level is cleared, a planner searches the verified model for a path to the goal. The plan is executed step-by-step in the live game. Any mismatch between the predicted and observed state aborts the plan, records a counterexample, and returns control to the action agent.

Key Contributions

1. OPINE-World Agent: An LLM agent that learns object-centric programmatic world models online via a hypothesis-and-test loop involving two cooperating agents, counterexample-guided synthesis, and model-based planning.
2. Ontology Discovery: A method for discovering the object ontology (partitioning objects into mechanical types) directly from interaction, extending program-synthesized world models from structured-state inputs to pixel-rendered environments with unknown object structures.
3. Evaluation on ARC-AGI-3: A demonstration on the ARC-AGI-3 benchmark (where object vocabulary, goals, and action semantics are withheld) where OPINE-World solves 20 of 25 games without per-game training, surpassing strong baselines.

Experimental Results

The system was evaluated on the ARC-AGI-3 public evaluation set (25 games, 183 levels).

• Performance: OPINE-World solved 20 of 25 games and cleared 160 of 183 levels without per-game training.
• Action Efficiency: It achieved an action-efficiency score of 78.4 (relative to a human baseline), compared to 63.8 for a strong single-agent coding baseline (baseline1) and 63.2 for a vision agent that pre-trained on the evaluation set.
• Comparison to Baselines:
  • Single-Agent Coding (baseline1): Solved 14 games. OPINE-World solved all 14 of these plus 6 additional games that defeated the baseline. On the 6 games the baseline failed, OPINE-World cleared them using approximately 25% of the actions the baseline spent before failing.
  • Program Synthesis (WorldCoder) & Neural Latent Models: These approaches solved 0 games under the benchmark's action budget.
  • Human Baseline: On the 20 games OPINE-World won, it used fewer actions than the human reference in 16 cases.

Significance and Claims

The paper claims that OPINE-World demonstrates that programmatic world models can be learned online from raw pixels without pre-defined object vocabularies or goals. By separating acting from modeling and using a rigorous exact-replay verification loop guided by a Bayesian ontology error, the system achieves high sample efficiency and transferability.

The authors emphasize that the system does not rely on optimism constraints or synthesized reward functions (unlike prior work such as WorldCoder) but instead relies on exact replay consistency and goal predicates derived from observed rewards. The results suggest that factoring models by object type and discovering the ontology dynamically allows agents to generalize across layouts and solve complex skill-acquisition tasks with significantly fewer interactions than neural latent models or single-agent coding approaches.

Limitations

The authors note several limitations:

1. Observable-Markov Scope: The exact-replay test assumes a deterministic transition function under a correct representation. Hidden state degrades the timing signal.
2. Inferred Perception: Object pairing and type partitioning are inferred from raw frames, which is harder than using engine-provided object identities.
3. Planner Scale: The planner uses bounded forward search, which may fail on games with high branching factors.
4. Single Run: Results are based on a single run per game, lacking variance estimates.