🤖 machine learning

QuantWAMs: Calibrating at the Right Granularity for World Action Models

QuantWAMs is a post-training quantization framework that aligns quantization decisions with the specific calibration context of World Action Models through three novel strategies, enabling efficient W4A4 deployment with minimal performance degradation on both simulation benchmarks and real-robot manipulation tasks.

Original authors: Jiacheng Zhou, Jinfan Lv, Ruixuan Li, Longtai Zhang, Yan Wang, Wenqiang Zhang, Lizhe Qi

Published 2026-07-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiacheng Zhou, Jinfan Lv, Ruixuan Li, Longtai Zhang, Yan Wang, Wenqiang Zhang, Lizhe Qi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to cook a meal or build a tower of blocks. To do this, the robot doesn't just look at a picture and guess; it uses a "World Action Model." Think of this model as a super-smart, futuristic movie director. It watches the current scene, predicts what will happen next, and simultaneously decides what the robot's arms should do to make that future happen. It's a closed loop: the robot moves, the world changes, the robot sees the change, and it decides the next move. This is incredibly powerful, but it's also like trying to run a blockbuster movie on a tiny, battery-powered calculator. The math required to predict the future and move the arms is so heavy that it usually needs massive, expensive computers.

To make these robots practical for our homes and factories, scientists want to shrink these giant computer brains down to fit on smaller chips. They do this using a technique called "quantization," which is like compressing a high-definition movie into a smaller file size. You trade a tiny bit of picture quality for a huge gain in speed and storage. However, the standard ways of compressing these files were designed for static images or simple text, not for a robot that is constantly moving and reacting to the world. If you compress a robot's brain using the old rules, it might start hallucinating, forgetting how to hold a cup, or crashing into walls because the "compression" messed up the delicate balance of its decision-making loop.

This is where the new paper, QuantWAMs, steps in. The researchers realized that to shrink a robot's brain without breaking it, you can't just use a one-size-fits-all compression tool. You have to calibrate the compression to the specific "vibe" of the robot's world. They found that standard methods make three big mistakes: they look at the robot's brain in isolation (ignoring how actions change the world), they treat all parts of the brain as if they are the same (ignoring that video and action parts are different), and they use the wrong kind of data to decide what to shrink.

The team proposed a new method called QuantWAMs that fixes these errors by treating the robot's brain like a complex orchestra rather than a solo instrument. Here is how they did it:

First, they realized that some parts of the robot's brain are "coordinate-compatible," meaning they speak the same language and can share information, while others are totally different. Imagine a choir where the sopranos and tenors can share a sheet of music, but the drummers need their own. The old methods tried to give everyone the same sheet, causing chaos. QuantWAMs carefully groups only the compatible parts together to share their "outlier" data (the weird, loud notes that are hard to compress), ensuring the compression doesn't distort the unique voice of each section.

Second, they changed how they decide which parts of the brain get to stay high-quality and which can be compressed. Instead of looking at the video part and the action part separately, they looked at how they work together. It's like judging a dance duo: you don't just rate the dancer's footwork and the partner's lift separately; you rate how well they move in sync. By analyzing the "joint" gradient (the combined effort of video and action), they could pinpoint exactly which layers of the brain were most critical for the robot's success and protect those with higher precision, while safely compressing the rest.

Third, and perhaps most importantly, they stopped testing the compressed brain in a "fake" world. Standard methods test the robot on a static set of pictures, like a driver's test on a closed track. But a real robot is always driving on a bumpy road where every turn changes the next view. The authors created a "fixed-intervention audit." They took the robot's real, full-precision movements, paused them, and asked: "If we had used the compressed brain at this exact moment, would the robot have recovered?" This allowed them to adjust the protection schedule dynamically, ensuring the robot stays safe even when the world gets messy.

The results of this new approach are impressive, though the authors are careful to note they are based on simulations and specific real-world trials. When they tested QuantWAMs on two different robot models (Fast-WAM and LingBot-VA) using a very aggressive compression setting (W4A4, which means using 4-bit precision for both weights and activations), the robot's performance barely dipped. In simulation, the success rate was only 0.2 to 0.7 percentage points lower than the original, uncompressed version. For example, on a task called RoboTwin 2.0, the compressed robot succeeded 91.7% of the time compared to the original's 91.9%.

More importantly, this compression came with massive benefits. The memory required for the robot's video and action blocks dropped to about 29% of what the original version needed. This made the robot 1.4 to 1.6 times faster at making decisions. They even took the compressed robot out to the real world using an AgiBot G2 robot to perform tasks like picking up an apple, stacking blocks, and folding a towel. In these real-world trials, the compressed robot succeeded in 17 out of 30 attempts, compared to 19 out of 30 for the full-precision version, proving that the compressed brain could actually function in a physical environment.

The paper suggests that by respecting the unique, closed-loop nature of robot learning—where actions change the world, and the world changes the next action—we can finally make these powerful AI models small enough to run on the robots we will one day see in our daily lives. It's not a magic bullet that solves everything instantly, but it's a crucial step toward making robot brains both smart and small.

Technical Summary: QuantWAMs

Problem Statement

World Action Models (WAMs) represent a promising paradigm for general-purpose robotic manipulation by jointly predicting future observations and the actions required to realize them. However, their deployment on real-world robots is hindered by high computational costs arising from iterative denoising and closed-loop execution, where every generated action alters the observations for subsequent predictions.

Existing Post-Training Quantization (PTQ) methods are ill-suited for WAMs due to three fundamental mismatches:

Objective Mismatch: Standard PTQ optimizes open-loop objectives (e.g., perplexity, single-image reconstruction) where forward passes are scored independently. WAMs operate in a closed loop where early quantization errors propagate and compound through state transitions.
Structural Mismatch: Standard methods often assume homogeneous transformer architectures. WAMs frequently employ coupled pathways (e.g., dual-stream Mixture-of-Transformers or shared-backbone designs) where computation is distributed across heterogeneous streams.
Distributional Mismatch: Calibration distributions in standard PTQ often profile sensitivity on states unreachable in a closed-loop setting, leading to "phantom" denoising-step peaks that misdirect the precision budget.

Methodology: QuantWAMs

QuantWAMs is a PTQ framework designed to align quantization decisions with the specific calibration context of WAMs, defined by model structure, rollout distribution, and task objective. It introduces three core strategies:

1. Shared-Basis Outlier Calibration

Standard outlier preservation often pools activation statistics across all modules indiscriminately. QuantWAMs restricts pooling to coordinate-compatible modules.

Mechanism: It pools activation evidence only across modules that share a common ordered basis (e.g., shared value pathways in Fast-WAM or shared input columns in LingBot-VA).
Theoretical Basis: The method models channel statistics as a mix of cross-member heterogeneity ( $\tau^2_c$ ) and within-member sampling variation ( $\sigma^2_c$ ). It derives a "pooling crossover" point ( $N^*$ ) to determine when pooling data across modules lowers estimation risk compared to per-context estimation.
Outcome: This prevents unstable masks caused by sampling noise when the number of calibration trajectories ( $N$ ) is low, while preserving the energy of critical outlier channels.

2. Co-Training-Objective Saliency

Instead of scoring layers based on a single stream or fusing gradients post-hoc, QuantWAMs allocates weight precision using the joint video–action gradient.

Mechanism: It computes empirical-Fisher scores from the combined loss ( $\ell_{co} = \lambda_v \ell_v + \lambda_a \ell_a$ ) of the pre-trained WAM.
Differentiation: Unlike "post-hoc fusion" which combines marginal objectives after the outer product, this method retains the coordinatewise interaction term ( $\Xi_L$ ) present in the joint gradient.
Granularity: Precision is assigned at a layer granularity (rather than element or column) to ensure calibration stability. The top 20% of candidate Linear layers are upgraded to higher precision (e.g., W8) based on these joint scores.

3. Fixed-Intervention Rollout Auditing

To address the closed-loop nature of WAMs, the method revises denoising-step protection schedules using reachable closed-loop states.

Mechanism: It performs a "fixed-intervention" audit on recorded FP16 rollouts. For each state in a trajectory, it replays the step under an unprotected (low-precision) intervention to measure the local discrepancy ( $\ell_t$ ) against the FP16 baseline.
Advantage: This avoids the "self-masked" problem of observational profiles (where a protected step appears insensitive because the protection is active) and the "unreachable state" problem of synthetic inputs.
Application: The profile is used to audit and replace the set of protected denoising steps ( $T_q$ ) with a new set ( $T_{replay}$ ) that better reflects sensitivity on actual closed-loop trajectories, without changing the total precision budget.

Key Contributions

Context-Aware Formulation: The paper formulates PTQ for WAMs around three specific contexts (structural, distributional, and objective) that determine deployment-time precision decisions.
Theoretical Derivation: It derives conditions for when activation evidence may be pooled across modules (governed by sample heterogeneity and trajectory count) and characterizes the sample-heterogeneity crossover for Top-K mask recovery.
Joint Gradient Saliency & Auditing: It proposes a method to jointly score video–action gradients at a calibration-stable layer granularity and utilizes fixed-intervention replay on real rollouts to revise denoising-step schedules.

Experimental Results

The framework was evaluated on two WAM implementations (Fast-WAM and LingBot-VA) across RoboTwin 2.0, LIBERO, and real-robot manipulation with an AgiBot G2.

Simulation Performance: Under a dominant W4A4 (4-bit weights, 4-bit activations) setting, QuantWAMs achieved simulation success rates differing from FP16 by only 0.2–0.7 percentage points.
- On Fast-WAM (RoboTwin 2.0): 91.7% vs. 91.9% (FP16).
- On LingBot-VA (LIBERO-Long): 98.0% vs. 98.5% (FP16).
Efficiency: For targeted video and action blocks, QuantWAMs reduced peak weight-and-activation memory to approximately 29% of the FP16 baseline and provided 1.4–1.6× block-level speedups.
Real-Robot Validation: On an AgiBot G2 performing three tasks (picking an apple, stacking blocks, folding a towel), the quantized policy (QuantWAMs) achieved a 56.7% success rate (17/30 trials), compared to 63.3% for FP16 and 40.0% for a baseline quantization method (Atom*).
Ablation Studies: Controlled experiments confirmed that:
- Shared-basis pooling improves mask recovery and closed-loop success.
- Joint co-training saliency outperforms post-hoc fusion and single-stream objectives.
- Fixed-intervention rollout auditing yields better schedule placement than synthetic or observational profiles.

Significance and Claims

The paper claims that QuantWAMs establishes the feasibility of deploying high-performance WAMs on resource-constrained real robots by addressing the specific structural and dynamic challenges of closed-loop control.

Modest Claims: The authors explicitly state that their results do not establish formal non-inferiority or equivalence to FP16, particularly in the real-robot trials where the sample size (10 trials per task) is underpowered for statistical comparison.
Scope Limitations: The work is limited to two specific WAM architectures, benchmark-specific calibration, and block-level efficiency measurements. It does not claim to solve end-to-end control-loop latency or transfer to unseen tasks.
Core Insight: The primary contribution is the demonstration that quantization decisions must be made within the specific "calibration context" of the model's deployment (structure, distribution, and objective) rather than relying on generic, open-loop heuristics.