💻 computer science

Same Weights, Different Robot: A Deployment Safety View of VLA Policies

This paper argues that Vision-Language-Action (VLA) deployment safety requires treating action normalization metadata as an integral part of the executable policy, demonstrating that identical model weights can produce drastically different and unsafe physical behaviors when paired with mismatched unnormalization conventions.

Original authors: Jianwei Tai

Published 2026-06-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jianwei Tai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very talented robot chef. You give this chef a specific recipe book (the model weights) and a list of ingredients (the prompt). You assume that if the recipe book and the ingredients are the same, the chef will cook the exact same dish every time.

This paper argues that for robots, this assumption is dangerous.

Here is the simple breakdown of why, using the paper's own findings:

1. The "Secret Translator" Problem

The paper says that the robot's brain (the AI model) doesn't actually tell the robot's arms what to do directly. Instead, the brain speaks a "normalized" language—a code where numbers are scaled between 0 and 1 to make them easy to learn.

Before the robot moves, a translator (called an unnormalizer) converts those 0-to-1 numbers back into real-world movements (like "move arm 5 inches forward").

The Catch: This translator needs a specific "dictionary" or "metadata key" to know how to do the conversion.

Key A might say: "0.5 means move 2 inches."
Key B might say: "0.5 means move 5 inches."

If you download the same recipe book (the AI model) but accidentally use Key B instead of Key A, the robot receives the exact same instructions from the brain, but it moves its arm completely differently.

2. The "Silent Swap"

The authors call this a "silent interface failure."

What you see: The AI model is identical. The code is identical. The safety check says "Pass."
What happens: The robot is actually running a different "policy" (a different set of physical rules) because the translator used the wrong dictionary.

It's like giving a pilot the same flight plan but swapping the map for a different continent. The pilot follows the instructions perfectly, but they end up in the wrong place.

3. The "Drift" Certificate

The paper introduces a tool called ExecSpec. Think of this as a pre-flight checklist that doesn't even need to run the robot.

It looks at the "dictionary" (the metadata) intended for the robot.
It compares it to the dictionary actually being used.
It calculates a "Drift Score." This score tells you exactly how far off the robot's physical movements will be, even before the robot turns on.

If the score is high, it means the robot is about to do something totally different than intended, even though the AI brain is the same.

4. The "Robot Gym" Experiments

To prove this isn't just theory, the authors tested this on a robot training simulator called LIBERO. They took successful robot movements (where the robot picked up an object perfectly) and replayed them, but with a twist:

Scenario: They kept the robot's "brain" and the "movement code" exactly the same.
The Change: They swapped the "dictionary" (metadata) for a similar-looking one from a different dataset (like swapping a "Goal" dictionary for a "Spatial" dictionary).

The Results were dramatic:

LIBERO-Goal: A task that worked 28 out of 28 times with the right dictionary dropped to 2 out of 28 times with the wrong one.
LIBERO-Spatial: A task that worked 26 out of 26 times dropped to 0 out of 26 with the wrong dictionary.
LIBERO-Object: All 28 successful attempts failed completely with the wrong dictionary.

The robot didn't fail because it was "dumb" or because the AI made a mistake. It failed because the translator changed the meaning of the numbers.

5. The Main Takeaway

The paper concludes that a robot policy is not just the AI model weights.

To truly know what a robot will do, you must check three things together:

The AI Model (The Brain).
The Prompt (The Instruction).
The Metadata/Dictionary (The Translator).

If any of these three change, you have a different robot policy, even if the weights look identical. The paper suggests that safety checks must include verifying this "dictionary" before the robot ever moves, or else you might think you are deploying a safe robot when you are actually deploying a different, potentially dangerous one.

In short: You can't just check the "engine" (the AI weights) to see if a car is safe; you also have to check the "steering calibration" (the metadata), or the car might drive in a circle even if the engine is perfect.

Technical Summary: Same Weights, Different Robot

Problem Statement
Current deployment models for Vision-Language-Action (VLA) policies treat the neural network checkpoint (weights, tokenizer, prompt) as the sole unit of policy identity and safety review. This paper argues that this assumption creates a critical "deployment-safety gap." In practice, a VLA model outputs normalized action coordinates, which must be transformed into physical controller commands via an action unnormalization step. This step relies on metadata (e.g., dataset-specific quantile statistics) to map normalized values back to physical units.

The core problem is that identical model weights can instantiate different executable robot policies if the metadata used for unnormalization is incorrect. For instance, substituting a plausible sibling metadata key (e.g., statistics from a different task suite) results in a different physical action law, even though the model's internal output remains unchanged. This constitutes a "silent interface failure" where safety audits certify the checkpoint but miss the actual executable policy reaching the robot controller.

Methodology
The authors formalize the VLA policy not as a model, but as an executable tuple $\Pi_{exec} = (f_\theta, I, Z, U_k, C)$ , where:

$f_\theta$ : The learned model.
$I$ : Image and language processors.
$Z$ : Normalized action representation.
$U_k$ : The action unnormalizer selected by metadata key $k$ .
$C$ : Controller conventions (gripper semantics, control frequency, etc.).

Two deployments are defined as executable-equivalent only if they induce the same physical action law under all admissible histories. The paper demonstrates that changing $k$ (while holding $f_\theta, I, Z, C$ constant) breaks this equivalence.

To detect this, the authors derive a closed-form metadata mismatch transform for quantile-style normalizers. If a deployment uses an intended key $k_c$ but a substituted key $k_w$ , the physical action displacement $\Delta_i$ for a normalized action $z$ on dimension $i$ is:
$\Delta_i(z; k_c, k_w) = (q^{(w)}_{01,i} - q^{(c)}_{01,i}) + z_i [ (q^{(w)}_{99,i} - q^{(w)}_{01,i}) - (q^{(c)}_{99,i} - q^{(c)}_{01,i}) ]$
This transform is deterministic and independent of the model inference. Based on this, they propose ExecSpec, a pre-rollout certificate that measures action-space semantic drift (mean displacement, tail displacement, axis attribution) on a calibration set without requiring model inference or simulation rollout.

Key Contributions

Executable Policy Identity Criterion: The paper defines a policy identity that includes action-space metadata, proving that checkpoint equality is insufficient for physical policy equality.
Closed-Form Certificate: It derives an exact, model-independent certificate for quantile-style unnormalization mismatches, allowing for the detection of semantic drift before any robot movement.
Empirical Witnesses: The authors provide extensive replay evidence across four LIBERO suites (Goal, Spatial, Object, Long) demonstrating that plausible metadata substitutions are sufficient to flip task success rates.

Results
The authors conducted experiments using replayed normalized trajectories (fixing the model output and initial state) to isolate the effect of metadata substitution.

LIBERO-Goal: Using the intended key GOAL-B, substituting LONG-V2 resulted in a mean drift of 0.199 over six non-gripper dimensions. This reduced task success from 28/28 (intended) to 2/28 (substituted). A dose-response curve showed success dropping monotonically as the metadata key interpolated from correct to incorrect.
LIBERO-Spatial: Substituting LONG-V2 for SPATIAL-V2 reduced success from 26/26 to 0/26.
LIBERO-Object & Long: All plausible sibling substitutions reduced success to 0/28 (Object) or 0/23 (Long) under full substitution.
Controls: The paper rules out simpler explanations such as no-op filtering, raw vs. correct replay artifacts, gripper convention mismatches, and synthetic corruptions. The failures were driven specifically by non-gripper action dimensions where the metadata statistics differed.

Significance and Claims
The paper claims a narrower deployment-safety view: action-space metadata is a load-bearing component of the executable policy and must be versioned and checked alongside model weights.

Not a General Failure Claim: The authors explicitly state they do not claim that all VLA deployments fail under metadata mismatch, nor that normalization is the only hidden hazard.
Not a Hardware Safety Guarantee: The results are based on simulation replay and do not certify closed-loop or hardware safety. They do not predict hardware failure rates.
Core Insight: The significance lies in identifying a hidden action-space semantics layer. If a deployment changes the unnormalizer while keeping the model fixed, it changes the executable policy. This mismatch can be detected via a static certificate before rollout, preventing the deployment of a policy that is semantically different from the one audited.

The paper concludes that deployment manifests must include action unnormalization keys, statistics hashes, and controller conventions to ensure reproducibility and safety, treating these fields as integral to the policy object rather than mere configuration details.