🤖 AI

Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goals

The paper introduces Goal Force, a novel framework that trains video generation models on synthetic causal primitives using explicit force vectors as goals, enabling zero-shot generalization to complex real-world physics tasks and serving as an implicit neural physics simulator for precise planning.

Original authors: Nate Gillman, Yinghua Zhou, Zitian Tang, Evan Luo, Arjan Chakravarthy, Daksh Aggarwal, Michael Freeman, Charles Herrmann, Chen Sun

Published 2026-03-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nate Gillman, Yinghua Zhou, Zitian Tang, Evan Luo, Arjan Chakravarthy, Daksh Aggarwal, Michael Freeman, Charles Herrmann, Chen Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot how to play pool.

The Old Way (Text Instructions):
You tell the robot, "Hit the white ball so the black ball goes into the corner pocket."

The Problem: The robot might hit the white ball too hard, too soft, or at the wrong angle. It might even hit the wrong ball entirely. Text is too vague for the messy, physical world.

The "Force Prompting" Way (Previous Tech):
You grab the white ball with a digital hand and physically shove it toward the black ball.

The Problem: This is like doing the work for the robot. You are telling it exactly what to do, not what to achieve. It doesn't learn how to plan; it just reacts to your push.

The "Goal Force" Way (This Paper):
You don't tell the robot how to hit the ball. Instead, you point at the black ball and say, "I want this ball to fly that way with this much speed." You define the result (the "Goal Force").

Then, you let the robot figure out the cause.

Robot's Brain: "Okay, the black ball needs to go left. I can't hit it directly because a stick is in the way. Ah! I see the white ball. If I hit the white ball, it will bounce off the wall, hit the black ball, and solve the problem!"
The Result: The robot generates a video showing the white ball striking the black ball, achieving your goal perfectly.

The Big Idea: Teaching the Robot to "Think" in Physics

This paper introduces a new framework called Goal Force. It teaches video generation models (AI that makes movies) to act like a physics simulator inside their brain.

Here is how they did it, using a simple analogy:

1. The Training Camp (The "Domino" School)

The researchers didn't show the AI complex real-world videos first. That would be too confusing. Instead, they built a training camp using simple, synthetic toys:

Falling Dominoes: One domino falls, hits the next, which hits the next.
Bouncing Balls: One ball hits another, sending it flying.

They taught the AI a simple rule: "If you want Object B to move this way, you must hit Object A that way." They trained the AI on thousands of these simple cause-and-effect chains.

2. The "Magic" Leap (Zero-Shot Generalization)

Here is the most impressive part. After training only on simple balls and dominoes, they tested the AI on complex, real-world scenarios it had never seen before.

The Golf Test: They asked the AI to make a golf ball go into a hole. The AI didn't just push the ball; it figured out it needed to use a golf club to hit it.
The Rose Test: They asked the AI to move a rose. The AI figured out it should grab the stem (which is sturdy) rather than the petals (which would rip off).
The Human Test: They asked a human hand in the video to knock over a chess piece. The AI figured out the hand needed to swing a specific way to make it happen.

The AI didn't memorize these specific scenes. It learned the underlying laws of physics (mass, momentum, collision) from the simple dominoes and applied them to the complex world. It became an "Implicit Neural Physics Simulator."

3. How It Works (The "Three-Channel" Remote Control)

To talk to this AI, the user provides a special "remote control" signal with three buttons (channels):

Direct Force Button: "Push this object right now." (Used for simple tasks).
Goal Force Button: "I want this object to end up moving right." (The AI has to plan the steps to get there).
Mass Button: "This object is heavy; that one is light." (Helps the AI understand how hard to hit things).

When you press the Goal Force button, the AI looks at the scene, thinks, "Okay, to get that result, I need to create a chain reaction," and then generates a video showing exactly how to do it.

Why This Matters

Better Planning: It moves AI from just "guessing" what happens next to actually planning how to make things happen.
No External Engines: Usually, to simulate physics, you need a heavy, complex physics engine (like in video games). This AI learned the physics inside its own brain. It doesn't need external tools; it just "knows" how the world works.
Robotics: This is a huge step for robots. Instead of programming a robot with millions of specific rules for every possible situation, we can just tell the robot the goal, and it figures out the physical steps to get there.

In a Nutshell

Think of Goal Force as teaching a video AI to be a director rather than a cameraman.

A cameraman just records what happens when you push a button.
A director understands the script, the physics of the scene, and the actors' movements. They know that to get the hero to the castle, they need to build a bridge, not just teleport the hero.

This paper teaches video models to be directors of the physical world, capable of figuring out the "how" to achieve the "what."

1. Problem Statement

Recent advancements in video generation have enabled the creation of "world models" capable of simulating potential futures for robotics and planning. However, specifying precise goals for these models remains a significant bottleneck:

Text Instructions: Often too abstract to capture physical nuances (e.g., "shoot the ball" lacks specific force and trajectory details).
Target Images: Frequently infeasible for dynamic tasks, as users cannot easily render the exact lighting or state of a moving object in a future frame.
Direct Force Limitations: Existing methods (e.g., Force Prompting) allow users to specify a direct force vector, which immediately acts on an object. This limits the model to simulating immediate effects rather than planning the antecedent actions (causes) required to achieve a desired outcome.

The core challenge is enabling video models to reason about causal chains: given a desired final state (a "goal force" on a target object), the model must generate the video of the preceding events (e.g., a tool striking an object, or one object colliding with another) that logically leads to that state.

2. Methodology: Goal Force Framework

The authors introduce Goal Force, a framework that conditions video generation on explicit goal force vectors and intermediate dynamics, teaching the model to act as an implicit neural physics planner.

A. Multi-Channel Physics Control Signal

The core innovation is a novel 3-channel control tensor $\tilde{\pi} \in \mathbb{R}^{f \times 3 \times h \times w}$ that encodes physical properties spatially and temporally:

Channel 0 (Direct Force): Encodes the immediate "cause" (e.g., a hand pushing a block). Represented as a moving Gaussian blob.
Channel 1 (Goal Force): Encodes the desired "effect" or outcome on a target object (e.g., "the block must move left"). This is the primary input for the Goal Force task.
Channel 2 (Mass): Encodes privileged physical information (relative mass) as a static Gaussian blob. This is optional; if omitted, the model relies on its internal priors to infer mass from appearance.

Normalization: Force and mass values are not calibrated to absolute physical scales but follow a relative scale normalized within each dataset, allowing the model to learn concepts like "small poke" vs. "large poke" without absolute units.

B. Training Paradigm: Implicit Planning via Masking

The model is trained on a curated dataset of synthetic causal primitives (elastic collisions, falling dominos, swaying flowers) generated using Blender and PhysDreamer.

Curriculum: The dataset includes 3k domino videos, 6k rolling ball videos (colliding and non-colliding), and 3k plant videos.
Masking Strategy: During training, the model is presented with a video and a control signal where causal information is randomly masked:
- Goal $\to$ Plan: The model receives the Goal Force (Channel 1) but the Direct Force (Channel 0) is zeroed out. It must infer and generate the antecedent action (e.g., a pendulum swinging) to achieve the goal.
- Action $\to$ Outcome: The model receives the Direct Force (Channel 0) but the Goal Force is zeroed out. It must simulate the resulting collision and secondary forces.
- Mass: The mass channel is also randomly masked to teach the model to rely on visual priors when physical data is missing.

C. Architecture

Base Model: Built on Wan2.2, a Mixture-of-Experts (MoE) diffusion model.
ControlNet: A ControlNet module is added to condition the model on the physics signal $\tilde{\pi}$ .
Fine-tuning: Only the high-noise expert (responsible for global structure and low-frequency dynamics) and the ControlNet layers are fine-tuned. The base model remains frozen.
Training: 3,000 steps on synthetic data using 4 NVIDIA A100 GPUs.

3. Key Contributions

Goal Force Task Definition: A shift from specifying direct forces (cause) to declaring desired goal forces (effect), enabling the model to plan antecedent causal chains.
Implicit Neural Physics Simulator: A training paradigm using multi-channel control signals that teaches video models to reason about physics (force propagation, mass, causality) without requiring an external physics simulator at inference time.
Zero-Shot Generalization: Demonstrating that training on simple synthetic data (balls, dominos) allows the model to generalize to complex, out-of-domain scenarios involving tool use, human-object interactions, and multi-object collisions.

4. Experimental Results

A. Comparison to Baselines

vs. Text-Only: In a human study (N=40) across four categories (two-object collisions, multi-object, human-object, tool-object), Goal Force significantly outperformed text-only baselines (both zero-shot and fine-tuned) in Goal Force Adherence (e.g., 73.4% vs 66.9% on two-object collisions) while maintaining high visual quality and motion realism.
vs. Prior Force Methods: Unlike PhysGen, PhysDreamer, or Force Prompting (which only simulate direct effects), Goal Force successfully generates the cause to achieve a goal. Prior methods often misinterpret goal forces as direct pokes on the target, violating causality.

B. Visual Planning Capabilities

Physical Accuracy: The model correctly identifies valid initiators in scenes with "natural blockers." For example, in a pool scene where a stick blocks one ball, the model correctly selects the unblocked ball to initiate the collision (98% accuracy).
Diversity: The model avoids mode collapse. In a task to topple the 6th domino in a line, the model samples from a diverse distribution of valid initiators (any of the preceding 5 dominos), achieving a diversity score of 0.6577 compared to a deterministic baseline of 0.39.
Mass Awareness: The model leverages mass information to adjust projectile speeds. If the target is heavier, the projectile moves faster; if the projectile is heavier, it moves slower. This holds true even in out-of-distribution scenarios with different backgrounds and lighting.

5. Significance and Impact

Paradigm Shift: Moves video generation from "what if" (simulating a direct effect) to "how to" (planning the sequence of actions to achieve a goal).
Robotics and Planning: Provides a mechanism for visual planning where robots can simulate the physical consequences of actions before execution, using goal forces as a high-level interface.
Simulator-Free Inference: Unlike hybrid approaches that require 3D geometry or internal physics engines during inference, Goal Force relies solely on the generative model's learned priors, making it more flexible and applicable to real-world, unstructured environments.
Open Source: The authors release all datasets, code, model weights, and interactive demos, fostering further research in physics-aware generative AI.

In summary, Goal Force demonstrates that by grounding video generation in fundamental physical interactions (forces and mass), large-scale generative models can emerge as capable, implicit physics simulators capable of complex causal reasoning and planning.