💻 computer science

Uncovering Vulnerability of Vision-Language-Action Models under Joint-Level Physical Faults

This paper demonstrates that Vision-Language-Action models are vulnerable to joint-level physical faults that disrupt the action-to-motion interface and proposes J-PARC, a lightweight residual calibration framework that infers latent fault regimes from joint dynamics to adaptively correct actions and restore robustness without compromising fault-free performance.

Original authors: Minsoo Jo, Taeju Kwon, Junha Chun, Youngjoon Jeong, Taesup Kim

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minsoo Jo, Taeju Kwon, Junha Chun, Youngjoon Jeong, Taesup Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: When the Robot's Body "Sickens"

Imagine you have a brilliant, highly trained chef (the VLA Model) who can look at a messy kitchen, read a recipe, and perfectly plan how to chop vegetables and stir a pot. This chef is so smart that they can handle new recipes and different kitchen layouts.

However, there's a catch: the chef doesn't cook with their own hands. They give instructions to a robotic arm (the Embodiment) to do the actual work.

This paper asks a question that no one had really tested before: What happens if the robotic arm gets sick?

Maybe a joint in the arm gets stuck (like a rusty hinge), moves only halfway (like a broken knee), or gets too stiff because of friction (like a joint with arthritis). The chef (the AI) still thinks everything is fine and gives the exact same instructions. But because the arm is broken, the knife misses the vegetable, or the pot spills.

The paper shows that these "sick" arms cause the robot to fail, even if the AI brain is perfect. To fix this, the authors built a "physical therapist" for the robot called J-PARC.

The Problem: The "Broken Arm" Effect

The researchers tested what happens when they lock up different joints on a robot arm (like a Franka Panda robot) while it tries to do tasks like putting a bowl in a drawer.

1. It's not just about "can't reach":
You might think if a joint is locked, the robot just can't reach the target. But the paper found something more subtle. Even if the robot could physically reach the target, the movement felt "wrong."

Analogy: Imagine trying to walk a straight line while wearing heavy, stiff boots. You might still be able to reach the finish line, but your gait is so weird that you stumble, lose your balance, and miss the target. The robot's "gait" (how it moves) gets messed up by the friction or lock, causing it to drift off course.

2. The "Drift" Effect:
When the robot makes a small mistake because of a broken joint, it sees a slightly different view of the world. The AI brain thinks, "Oh, I'm not where I expected to be," and tries to correct it. But because the arm is still broken, the correction makes things worse.

Analogy: It's like driving a car with a flat tire. You steer left to go straight, but the car pulls right. You steer harder left, but the car pulls harder right. Eventually, you spin out of control. The robot gets stuck in this loop of trying to fix a problem it can't actually fix with its broken body.

3. Different Joints, Different Problems:
The paper found that breaking different joints causes different types of failures.

Analogy: If your shoulder is locked, you can't lift your arm high. If your wrist is locked, you can't turn a doorknob. Similarly, locking "Joint 0" (the base) messes up horizontal movement, while locking "Joint 4" (near the hand) messes up grabbing things. The robot fails in very specific ways depending on which part is broken.

The Solution: J-PARC (The "Physical Therapist")

The authors created a new system called J-PARC (Joint-level Physical-fault Aware Residual Calibrator).

Think of the main AI (the Chef) as frozen. You don't want to retrain the Chef because they are already great at cooking; you just want to fix the arm.

How J-PARC works:

The Detective: J-PARC watches the robot's recent movements. It looks at the history of how the arm moved and asks, "Hey, something feels off. Is the arm stiff? Is a joint stuck? Which one?" It figures out the "sickness" without needing a manual to tell it.
The Therapist: Once it knows the arm is sick, it adds a tiny "nudge" to the Chef's instructions.
- If the Chef says, "Move the hand 10cm forward," and the arm is stiff, J-PARC might say, "Okay, but actually, push it 12cm forward to overcome the stiffness."
- It only tweaks the position (where the hand goes), not the rotation or the grip, because those are harder to fix without breaking the arm further.
The Safety Net: If the arm is healthy, J-PARC does nothing. It stays quiet. This means the robot works exactly as well as before when there are no faults.

The Results: Does it Work?

The researchers tested this in two ways:

In Simulation (The Video Game): They simulated broken joints (locked, limited range, high friction) on a virtual robot.
- Result: Without J-PARC, the robot failed often. With J-PARC, the success rate went up significantly. Even when they tested it on a type of "sickness" (friction) it had never seen before, it still worked. It was like the therapist could guess the right treatment even for a new injury.
In the Real World (The Real Robot): They put a real robot arm (a Trossen WidowX) to the test with a bowl pick-and-place task. They physically locked a joint during the task.
- Result: The normal robot failed every time. The robot with J-PARC succeeded in most trials. The video showed the robot's hand drifting off course without J-PARC, but staying on a smooth, successful path with J-PARC.

Summary

This paper proves that giving a robot a "sick" body breaks its ability to do tasks, even if its brain is perfect. The solution isn't to retrain the brain, but to add a smart, lightweight layer (J-PARC) that acts like a physical therapist. It watches the robot's movements, figures out what's wrong with the joints, and gently nudges the instructions to compensate, allowing the robot to keep working even when parts of its body are damaged.

Technical Summary: Uncovering Vulnerability of Vision-Language-Action Models under Joint-Level Physical Faults

Problem Statement

The deployment of Vision-Language-Action (VLA) models in real robotic systems faces a critical robustness gap: while these models demonstrate strong generalization to semantic and perceptual variations, they often lack robustness to embodiment-side faults. Specifically, VLA policies assume that the robot body faithfully converts predicted actions into intended motions. However, real-world robots are susceptible to joint-level physical faults such as actuator degradation, hardware failures, safety-imposed limits, collision damage, and wear-induced friction.

These faults alter the action-to-motion interface, disrupting the learned closed-loop relationship between commanded actions, realized motion, and subsequent observations. Unlike conventional action noise (which perturbs the policy output directly), joint-level physical faults change the robot's kinematic or dynamic capability itself. Consequently, the same commanded action can result in different realized states depending on the affected joint and fault severity. This leads to a closed-loop execution mismatch, where the robot drifts from the nominal state trajectory expected by the VLA policy, potentially causing unrecoverable failures even if the task remains physically feasible.

Methodology: J-PARC

To address this vulnerability, the authors propose Joint-level Physical-fault Aware Residual Calibrator (J-PARC), a lightweight framework designed to operate on top of a frozen VLA policy without fine-tuning the base model.

Core Architecture

J-PARC functions as a residual correction module that adjusts the translational components of the VLA's action output based on the inferred execution context.

Input Context: The module observes recent joint dynamics and execution history ( $H_t$ ), which includes base actions, end-effector/gripper states, and the difference between commanded and executed actions.
Fault Regime Inference: A pre-trained encoder ( $g_\psi$ ) processes recent joint motion to infer a latent joint-fault regime ( $u_t$ ). This encoder is trained to detect the presence of a fault and classify the specific affected joint, but it remains frozen during the residual training phase.
Residual Prediction: A transformer-based residual calibrator ( $f_\theta$ ) takes the base action chunk, execution history, and the inferred fault regime as input. It predicts a residual chunk ( $R_{\theta,t}$ ) specifically for the translational arm commands ( $\Delta x, \Delta y, \Delta z$ ).
Action Correction: The final executed action is the sum of the base VLA action and the predicted residual. The rotation and gripper commands from the base policy are preserved to avoid unnecessary joint reconfiguration.

Training Strategy

J-PARC utilizes a reference-guided residual target approach:

Data Collection: The system pairs fault-free reference rollouts with rollouts under locked-joint conditions starting from the same initial state.
Target Generation: A "teacher" mechanism compares the faulty execution state against the fault-free reference trajectory. If the deviation exceeds a threshold, the target residual is calculated to redirect the end-effector toward the fault-free waypoint. If the deviation is within a "demonstration tube," the target residual is zero.
Optimization: The residual calibrator is trained to minimize the mean squared error between its predicted residual and the teacher-generated target, while the base VLA policy and the fault encoder remain frozen.

Key Contributions

Identification of Vulnerability: The paper establishes joint-level physical faults (locking, range limits, increased friction) as a critical, distinct robustness challenge for VLA deployment, separate from perceptual or command-space noise.
Analysis of Heterogeneous Effects: Through extensive analysis, the authors demonstrate that fault effects are joint-dependent. Tasks degrade heterogeneously based on which joint is affected and its usage in the task. Furthermore, performance drops occur even in feasible scenarios (e.g., increased friction), indicating that the issue is not merely physical infeasibility but a closed-loop execution mismatch.
J-PARC Framework: The proposal of a lightweight, residual calibration framework that infers latent fault regimes from execution history and adapts actions accordingly, preserving the base policy's behavior in fault-free environments.
Empirical Validation: Experiments on the LIBERO benchmark and real-world hardware (Trossen WidowX AI) show that J-PARC significantly improves success rates under joint faults while maintaining performance in nominal conditions.

Experimental Results

The authors evaluated J-PARC on two frozen VLA backbones (OpenVLA-OFT and $\pi_0.5$ ) across the LIBERO benchmark suites.

Locked Joint Faults: Under joint-locking conditions, J-PARC improved the average success rate from 42.3% to 46.8% (OpenVLA-OFT) and from 48.4% to 55.2% ( $\pi_0.5$ ). Notably, it outperformed baselines like Constrained Inverse Kinematics (CIK) and RobustVLA, which failed to consistently improve performance or even degraded it.
Unseen Friction Faults: Crucially, J-PARC was trained only on locked-joint data. When tested on increased joint friction (a different fault mode not seen during training), it still improved average success rates (e.g., from 46.4% to 55.1% for OpenVLA-OFT), demonstrating generalization to unseen execution mismatches.
Fault-Free Preservation: In fault-free environments, J-PARC preserved the base policy's performance, confirming that the residual module does not interfere with nominal operation.
Real-World Validation: On a physical robot performing a bowl pick-and-place task, the base policy failed completely under joint locks, whereas J-PARC succeeded in most trials by keeping the end-effector trajectory close to the fault-free reference path.

Significance and Claims

The paper argues that embodiment-side robustness is a necessary condition for reliable VLA deployment, not merely a secondary hardware concern. The authors claim that:

Current robustness methods focusing on action noise or perceptual perturbations are insufficient for handling persistent physical faults that alter the robot's dynamics.
The mismatch between commanded actions and realized motions accumulates over time, requiring adaptive action calibration that conditions on the current execution context rather than static constraints.
J-PARC offers a practical, lightweight solution that enhances robustness without the computational cost of fine-tuning large VLA models or the need for extensive adversarial training data.

The authors remain modest regarding limitations, noting that their real-world validation is currently restricted to specific joints (4 and 5) and a single task due to safety constraints, and that the method currently relies on access to fault-free reference trajectories for training. Future work is suggested to explore automatic reference construction and broader action spaces.