💻 computer science

Compliance for Free: Learning Identifiable Impedance via Bilateral Teleoperation

This paper proposes a method using four-channel bilateral teleoperation to identify and record operator stiffness from existing joint-torque sensors, enabling the fine-tuning of vision-language-action models to generate direction-dependent compliance labels at zero annotation cost for contact-rich tasks.

Original authors: Harsha Guda, Adrià Colomé, Carme Torras

Published 2026-09-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Harsha Guda, Adrià Colomé, Carme Torras

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots have become remarkably good at seeing the world and deciding where to move next. Modern artificial intelligence can look at a picture of a messy room and tell a robot arm exactly how to pick up a cup or open a drawer. But there is a final, critical step where these machines often stumble: the moment they touch something. While a robot can be told where to go, it struggles to know how hard to push. Tasks that require a gentle touch, like wiping a whiteboard without scratching it, or inserting a peg into a tight hole, depend on a property called compliance. This is the ability to yield or resist pressure in a controlled way. For a long time, teaching robots this skill has been difficult because the standard ways humans show robots what to do only record the robot's final position. They miss the invisible force the human was applying, leaving the robot to guess whether it should be stiff or soft.

A team of researchers in Barcelona has found a way to solve this missing piece of information without adding expensive force-torque or tactile sensors. They discovered that by using a specific type of remote control setup, they could mathematically separate the human's intended target from the force they applied using only the joint-torque sensors already present on the robot arm. In their experiments, a human operator controlled a robot arm through a system where the operator's own arm moved a second, identical robot arm. By watching how the operator's arm moved compared to how the robot arm actually moved, the researchers could calculate exactly how stiff or soft the operator wanted the robot to be at every single moment. They used this new data to train an artificial intelligence model that can now take a simple spoken instruction, such as "wipe this mark firmly," and adjust its own stiffness to match, rather than just moving to a location.

The core of the problem the researchers tackled is a confusion that happens whenever we try to teach a robot by watching a human. Imagine a human pushing a robot arm against a wall. If the robot ends up in a certain spot, we do not know if the human pushed hard against a stiff spring or gently against a soft spring; both scenarios could result in the robot ending up in the exact same place. Standard recording devices only see the final position, so they cannot tell the difference. This makes it impossible to teach a robot the concept of "firm" versus "gentle" just by watching where the robot goes. The researchers realized that to fix this, they needed a second, independent measurement of where the human intended to go, distinct from where the robot actually ended up.

To solve this, they used a four-channel bilateral teleoperation system. In this setup, a human holds a "leader" robot arm, and a "follower" robot arm tries to copy it. Crucially, the system does not just send position commands; it also sends force information back and forth. When the follower arm hits an obstacle, the human feels that resistance in their own hand. Because the human is actively feeling the contact, their own arm's movement represents their true intention, while the follower's movement shows how the robot reacted to the environment. By comparing the leader's path with the follower's path, the researchers could calculate the difference between where the human wanted to be and where the robot actually was. This difference, combined with the force the robot felt (which was estimated using the robot's native joint-torque sensors), allowed them to work backward and determine the exact stiffness the human was applying.

Using this method, the team collected a large set of demonstrations where humans wiped a whiteboard. They instructed the humans to wipe marks on the board either "normally" or "firmly." Because of their new calculation method, they could extract a precise label for the stiffness used in every single moment of the wiping motion, without needing any special force sensors on the robot's wrist. They then used these labels to fine-tune a large vision-language model, a type of artificial intelligence that understands images and text. They taught this model to output not just a target position, but also a target stiffness value for every move it makes.

The results showed that this approach worked exactly as intended. When the researchers tested the new robot policy, they found that it could genuinely change how hard it pressed based on the language instruction. When told to wipe "firmly," the robot increased its contact force to an average of 9.1 Newtons. When told to wipe "normally," it reduced that force to 6.4 Newtons. This change was statistically significant and consistent. In contrast, other robot policies tested in the same study failed to change their behavior based on the instruction. Some policies that were given force data as an input still moved too rigidly, while others that tried to guess the stiffness based on fixed rules could not adapt at all. Only the policy trained with the new, extracted labels successfully linked the word "firmly" to a physical increase in pressure.

The researchers also verified that the robot learned to be selectively stiff. It did not just become uniformly harder in all directions; it became stiff in the direction of the wiping motion to resist being pushed off course, while remaining softer in other directions. This anisotropic behavior, or direction-dependent stiffness, is a sophisticated trait that mimics how a human hand naturally adjusts to a task. The robot achieved a 50 percent success rate in removing ink from the board, which was the highest among the five different policies tested, and it did so while triggering fewer safety stops caused by excessive force.

Despite these successes, the study acknowledges certain limitations. The method relies on the robot having sensors that measure the torque in its joints, which are common in research robots but not always in cheaper commercial models. The technique also required the robot to stay in a specific posture during calibration to ensure the force calculations remained accurate. Furthermore, the rotational stiffness, or how the robot resists twisting, was harder to measure accurately because the wiping task did not involve enough twisting motion to generate clear data. The researchers noted that while their method works well for this specific task, it is not a universal solution for every type of contact a robot might encounter.

The significance of this work lies in how it bypasses the need for expensive, specialized force-torque or tactile hardware to teach robots about touch. By using the existing joint-torque sensors on a robot arm and a clever mathematical approach to interpret human intent, the researchers created a way to generate high-quality training data for compliance at zero extra cost. This suggests that robots could learn to handle delicate or variable-force tasks much more effectively in the future, simply by observing humans through a system that captures both movement and the feeling of resistance. The study demonstrates that the key to better robot touch may not be better sensors, but better ways of reading the signals we already have.

Technical Summary: Compliance for Free: Learning Identifiable Impedance via Bilateral Teleoperation

1. Problem Statement

Vision-Language-Action (VLA) models have revolutionized robot manipulation but consistently fail at contact-rich tasks (e.g., wiping, insertion) because they output only pose targets, not the necessary compliance (stiffness) to modulate interaction forces. While some recent works attempt to output force or compliance, they rely on hand-specified interaction frames, privileged simulation data, or dedicated force/torque (F/T) and tactile sensors during training and inference.

The core obstacle is a fundamental identifiability problem in standard demonstration interfaces (e.g., VR controllers, SpaceMice, unilateral bimanual rigs). These interfaces record only the robot's realized pose ( $x_f$ ) and the resulting contact force ( $f$ ). However, contact force is the product of both the operator's intended equilibrium pose ( $x_{eq}$ ) and their stiffness ( $K$ ). Mathematically, a soft spring pulled far and a stiff spring pulled slightly exert the same force. Without an independent measurement of $x_{eq}$ , the operator's intended equilibrium and their stiffness are confounded; no amount of pose-only data can disentangle them. Consequently, existing interfaces cannot provide compliance supervision in principle.

2. Methodology

A. Resolving Identifiability via Bilateral Teleoperation

The authors propose using four-channel bilateral teleoperation to break the identifiability ambiguity. In this setup, a leader arm (controlled by the human) and a follower arm (the robot) are actively coupled.

Key Insight: The leader arm's trajectory ( $x_l$ ) serves as an independent, physical measurement of the operator's intended equilibrium ( $x_{eq}$ ).
Mechanism: By substituting $x_{eq} = x_l$ into the impedance law $f(t) = K(t)(x_{eq}(t) - x_f(t)) + D(t)\dot{e}(t)$ , the displacement error $e(t)$ becomes directly observable. This allows $K(t)$ and $D(t)$ to be identified via regression using only the joint-torque sensing already present on the manipulator, eliminating the need for external F/T sensors.

B. Sensorless Wrench Estimation

Since the method relies on joint-torque sensors rather than wrist F/T sensors, the authors implement a sensorless wrench estimation procedure:

Gravity Compensation: They identify the tool's mass and center of mass using static poses to subtract the gravity component from measured joint torques.
Bias Correction: They model the residual error (dependent on configuration and velocity) using a bank of Random Fourier Features fitted on free-space sweeps.
Noise Floor: A per-axis noise floor ( $\sigma_{f,i}$ ) is measured from held-out free-space data to ensure force estimates are above the sensor noise.

C. Compliance Label Extraction

The authors extract per-timestep, anisotropic compliance labels through a masked regression process:

Contact Frame: A contact frame is fitted to the wiping trajectory using Singular Value Decomposition (SVD) to define the surface normal.
Windowed Regression: Over sliding 300ms windows, a regularized nonlinear least-squares problem solves for stiffness ( $k$ ) and damping ( $d$ ) per axis.
Identifiability Mask: A binary mask is applied to retain labels only when:
1. The regression matrix is well-conditioned.
2. Displacement is significant.
3. Force exceeds the noise floor.
4. The system is in contact.
  This yields a dataset of $(x_{eq}, \log K)$ pairs with validity masks, requiring zero annotation cost.

D. Compliance-Output VLA Policy

The extracted labels are used to fine-tune a pretrained VLA (SmolVLA backbone):

Architecture: The policy takes vision (scene + wrist camera), language instructions (e.g., "wipe the left mark firmly"), and a force-history token as input.
Output: It predicts a chunk of future actions containing both an absolute target pose ( $x_{eq}$ ) and a stiffness target ( $\log K$ ).
Training: Uses flow-matching for pose and Huber loss for stiffness, applied only on masked (identifiable) timesteps.
Control: A low-level Cartesian impedance controller tracks the policy's chunked outputs. The system employs temporal ensembling and energy-tank gating to ensure stability and passivity.

3. Key Contributions

Sensorless Extraction Procedure: A method to resolve the pose-only identifiability ambiguity using bilateral teleoperation, yielding per-timestep compliance labels using only native joint-torque sensors.
Offline Benchmark: Demonstration that the extracted labels carry learnable signal, with a learned predictor outperforming a per-axis constant predictor by $1.25\times$ in aggregate RMSE on translational axes.
Instruction-Conditioned Compliance Policy: A VLA policy trained on these labels that successfully tracks chunked impedance targets. It is the only policy among five baselines whose realized contact force changes in response to language instructions regarding force "manner" (e.g., "firmly" vs. "normally").

4. Experimental Results

The method was evaluated on a Franka Research 3 robot performing a whiteboard wiping task with instructions to wipe "normally" or "firmly."

Instruction Conditioning: The policy successfully modulated its output stiffness based on the instruction.
- Demonstrated Stiffness: "Firm" demonstrations showed higher stiffness ($146.2$ N/m) than "normal" ($127.2$ N/m).
- Policy Output: The policy reproduced this gap ($141.2$ vs. $116.4$ N/m).
- Realized Force: Crucially, the realized contact force responded to the instruction: "Firm" resulted in $9.12$ N RMS force, while "Normal" resulted in $6.40 $N RMS ($ p=0.023$, Cohen's $d=0.89$ ).
Baseline Comparison:
- Fixed-K: Used constant stiffness; failed to modulate force based on instruction.
- Force-In: Consumed force as input but output only pose; failed to modulate force.
- Hybrid: Predicted stiffness but via a separate regression head; the commanded stiffness did not translate to realized force changes.
- Ours: The only policy where the realized contact force statistically responded to the language instruction.
Task Performance: The proposed policy achieved the highest success rate (50.0%) and ink removal (40.8%) while maintaining lower peak and RMS contact forces compared to the Fixed-K baseline, resulting in fewer protective stops (1 vs. 6).

5. Significance and Claims

The paper claims that the primary barrier to learning compliant manipulation is not model architecture but the demonstration interface. Standard interfaces conflate equilibrium and stiffness, making compliance supervision impossible without external hardware or simulation.

By utilizing four-channel bilateral teleoperation, the authors demonstrate that:

Compliance can be extracted for free: No additional F/T or tactile sensors are required at training or inference; the leader arm provides the necessary equilibrium signal.
Compliance is a learnable action: A VLA can be trained to output stiffness targets that are physically realized by the robot.
Instruction-conditioned force is achievable: The system can modulate contact force based on natural language ("firmly" vs. "normally") without relying on rigid constraints or privileged information.

The authors emphasize that predicting compliance is insufficient; the policy must also track it. Their approach achieves this by decoding pose and stiffness through a shared target, resulting in genuine anisotropic behavior where the robot resists deflection off the stroke path when instructed to wipe "firmly," rather than simply pressing harder into the board.

6. Limitations

The authors acknowledge several limitations:

Configuration Dependence: Sensorless wrench estimation requires a fixed nullspace posture and constrained workspace.
Rotational Impedance: Rotational channels are weakly identified in this specific task (wiping barely excites orientation), and the extracted rotational stiffness often saturates at the controller's lower bound.
Scope: The evidence is limited to a single task (wiping) and platform (Franka Research 3).
Ablation Power: The force-channel ablation study was underpowered due to a small rollout budget, leaving the causal role of the force input token open to further investigation.