💻 computer science

Behavioral Cloning for Robotic Connector Assembly: An Empirical Study

This paper presents an empirical study demonstrating that behavioral cloning, which fuses force-torque sensing with visual data from human teleoperation demonstrations, can achieve over 90% success in automating the challenging task of robotic wire harness connector assembly across various geometries and poses.

Original authors: Andreas Kernbach, Daniel Bargmann, Werner Kraus, Marco F. Huber

Published 2026-02-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andreas Kernbach, Daniel Bargmann, Werner Kraus, Marco F. Huber

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to plug a USB cable into a computer port in a dark room. You can't see perfectly, so you wiggle the cable, feel for the resistance, and gently nudge it until it clicks into place. Humans do this intuitively, combining what they see with what they feel.

Now, imagine trying to teach a robot to do the exact same thing. This is the challenge the authors of this paper tackled: automating the assembly of wire harnesses (the bundles of wires in cars, planes, and cabinets).

Here is the story of their experiment, broken down into simple concepts.

The Problem: The "Fiddly" Task

In factories, robots are great at moving heavy things in straight lines. But plugging in a connector is different. It's like trying to thread a needle while wearing boxing gloves.

The Wires are Squishy: Cables bend and move, making it hard to know exactly where they are.
The Tolerances are Tight: If you push too hard or at the wrong angle, you break the plastic or the wires.
The Old Way: Previously, engineers had to program robots with rigid "search patterns" (like spiraling around the hole). This was like teaching a robot to dance by writing down every single step. It took forever to tune, and if the hole moved even a tiny bit, the robot failed.

The Solution: "Behavioral Cloning" (The Robot Apprentice)

Instead of programming the robot with rules, the authors decided to let the robot learn by watching. This is called Behavioral Cloning (BC).

Think of it like an apprentice chef watching a master cook.

The Master (Human): A human operator holds the robot's arm (using a special 3D mouse controller) and plugs the connector in. They use their eyes (vision) and their hands (feeling the force) to figure out the best way to wiggle and push.
The Apprentice (The AI): The robot records every move the human makes. It learns a pattern: "When I see the socket is slightly to the left and I feel resistance on the right, I should push down and wiggle left."
The Result: The robot builds a "brain" (a neural network) that predicts the next move based on what it sees and feels, just like the human did.

The Experiment: Teaching the Robot

The researchers set up a test lab with a UR5e robot arm, a webcam, and a force sensor (a "smart wrist" that feels pressure).

The Training: They asked humans to plug in connectors 300 times from slightly different angles. This created a dataset of "successful attempts."
The Architecture: They tried different types of "brains" (neural networks) to see which one learned best.
- The Vision: They used a camera to see the socket. Surprisingly, a simple black-and-white camera worked better than fancy color cameras.
- The Feeling: They used the force sensor to feel when the plug hit the edge of the socket.
- The Brain: They tested various AI models. The winner was a specific type of image-processing brain (RegNet) combined with a simple math-based brain for the force sensor.

The Results: A Success Story

The results were impressive, especially for a robot learning a delicate task:

Success Rate: The robot successfully plugged in the connectors over 90% of the time.
Tolerance: It could handle the plug being up to 20mm off-center or tilted by 10 degrees. That's a huge margin of error compared to old methods, which usually required the plug to be almost perfectly aligned.
Speed: The robot was only about 1 second slower than a human.
Data Efficiency: It only needed 300 demonstrations (about an hour of human work) to learn. Compare this to massive AI models that need hundreds of thousands of examples to learn simple tasks.

The "Secret Sauce"

Why did this work so well?

History Matters: The robot didn't just look at the current image; it looked at the last 10 steps of video and force data. It's like remembering, "I was moving left, then I hit a wall, so now I should try moving right." Without this memory, the robot would just freeze.
Simplicity: They didn't need complex 3D maps or perfect lighting. Just a simple camera and a feeling of force were enough.
The "Wiggle": The human operators naturally added a little shaking motion (wiggling) to reduce friction. The robot learned to do this too, which was crucial for getting the plug in.

The Catch and The Future

Is it perfect? Not quite.

Not 100%: It still fails about 1 in 10 times. For a car factory, you usually need 100% reliability.
Specific to the Task: The robot learned how to plug in this specific type of connector. If you give it a totally different shape, it won't know what to do (it's not "zero-shot" smart yet).

The Conclusion:
This paper proves that you don't need a super-complex, expensive robot brain to do delicate assembly work. By simply letting a robot watch a human do the job a few hundred times, you can teach it to handle the "fiddly" parts of manufacturing.

The Metaphor:
Think of this research not as building a robot that thinks like a human, but as building a robot that has muscle memory. It doesn't need to understand why it's plugging in the wire; it just needs to know how to move its hand to make it happen, based on what it has seen before. This is a massive step toward making factories more flexible and less reliant on perfect, expensive precision.

1. Problem Statement

The assembly of wire harnesses in automotive, electrical cabinet, and aircraft production remains largely manual due to the complexity of deformable cables and high variance in connector geometries.

Challenges: Connectors must be inserted with tight tolerances and limited force to avoid damage. While humans use visual and haptic feedback intuitively, programming industrial robots for this task is difficult.
Limitations of Current Methods:
- Rule-based strategies: Rely on pre-defined search patterns (e.g., spiral or stride searches) and require iterative, time-consuming parameter tuning by experts. They struggle with large pose uncertainties (typically limited to $\pm 2$ mm).
- Imitation Learning (IL) gaps: While IL exists, there is a lack of empirical data on suitable neural network (NN) architectures, the number of required demonstrations, and the ability to generalize across different connector geometries without relying on precise pose data.
Goal: To investigate whether Behavioral Cloning (BC) can learn a robust action prediction model for connector insertion using only visual (monocular grayscale) and haptic (force-torque) data, achieving high success rates under significant pose uncertainties.

2. Methodology

A. Problem Formulation

The task is framed as a control problem where a neural network $f_\theta$ predicts a target wrench (force and torque) $\hat{w}_{t+1}$ for a force-torque controller.

Input: A history of $k$ time steps containing camera images ( $I$ ) and measured wrenches ( $w$ ).
Output: A sequence of predicted target wrenches over a horizon $h$ .
Training Objective: Minimize the Mean Squared Error (MSE) between the predicted wrenches and the expert demonstrations. The study explores both single-step prediction ( $h=1$ ) and multi-step prediction ( $h>1$ ) to mitigate compounding errors.

B. Hardware Setup

Robot: Universal Robot UR5e with a Robotiq Hand-E gripper.
Sensors:
- Force-Torque: Integrated sensor in the robot flange.
- Vision: Logitech webcam (VGA resolution) providing fixed-position grayscale images.
Data Collection: Human demonstrations were recorded via teleoperation using a SpaceMouse. The operator inserted connectors while the system logged sensor data and images at 20 Hz.
Tolerance Simulation: Experiments were conducted with start poses varying by up to 20 mm in translation and 10° in rotation, significantly larger than typical rule-based tolerances.

C. Network Architecture

The model is a multi-modal architecture with three components:

Vision Backbone: Processes image history. Evaluated various pre-trained CNNs (ResNet, DenseNet, RegNet, EfficientNet) and Transformers (ViT, Swin).
Sensor Backbone: Processes time-series force-torque data. Evaluated Feedforward networks, LSTMs, 1D-CNNs, and ROCKET models.
Action Head: Maps fused features to the target wrench sequence. Evaluated Feedforward networks and Transformer variants (Encoder, Full Transformer).

3. Key Contributions

Empirical Application of BC: Demonstrated a BC system for connector insertion trained on human teleoperation data, fusing vision and force-torque sensing without explicit pose estimation.
Sensor Sufficiency: Proved that a grayscale monocular camera combined with force-torque sensing is sufficient for achieving >90% success rates, eliminating the need for expensive 3D vision or precise pose estimation.
Architecture Ablation Study: Systematically compared various NN backbones and heads, identifying that CNN-based vision backbones (specifically RegNetx3 2GF) and simple Feedforward networks for sensors/heads outperformed complex Transformer architectures for this specific task.
Data Efficiency Analysis: Investigated the relationship between the number of demonstrations and success rates, identifying a plateau effect around 100–300 demonstrations.

4. Experimental Results

A. Architecture Selection (Car Door Connector)

Vision Backbone: RegNetx3 2GF achieved the highest mean success rate (93.3%) with low variance. Transformer-based backbones (ViT, Swin) performed poorly (<14%), likely due to the limited dataset size and the inductive bias of CNNs being better suited for this task.
Sensor Backbone: A simple Feedforward network outperformed LSTMs, 1D-CNNs, and ROCKET models, achieving 96.7% success.
Action Head: A Feedforward network outperformed Transformer encoders and full Transformers.
Hyperparameters:
- History Length ( $k$ ): Increasing $k$ from 1 to 10 improved success rates significantly. $k=1$ failed to reproduce "wiggling" motions necessary for friction reduction; $k=10$ provided sufficient temporal context.
- Prediction Horizon ( $h$ ): Results were stable between $h=1$ and $h=10$ , though $h=10$ was adopted for consistency with prior work.

B. Data Requirements

Success rate showed a positive correlation with the number of demonstrations.
A plateau was observed after 100 demonstrations, but using 300 demonstrations resulted in a 100% success rate on the evaluation set (90 trials).
The BC approach surpassed a state-of-the-art rule-based stride search (76.7% success) after roughly 200 demonstrations.

C. Generalization to New Geometries

The best-performing model (RegNetx3 2GF + Feedforward Sensor/Head) was tested on four additional connector geometries (Molex, TE, JAE) with varying tolerances (4 mm to 10 mm).

Overall Success Rate: 92.8% across all five connector types.
Robustness: Performance varied by geometry (Std Dev 2.01% to 8.10%), suggesting some geometries are inherently harder for the learned policy.
Speed: The robot was approximately 0.66 seconds slower than human operators but maintained high reliability.
Damage: No connectors were damaged during trials due to the force-controlled nature of the system.

5. Significance and Discussion

Industrial Viability: The system achieves success rates (>90%) comparable to or exceeding large-scale Vision-Language-Action (VLA) models (e.g., RT-1, ALOHA) but with orders of magnitude fewer demonstrations (300 vs. 130,000) and significantly smaller model sizes, making it suitable for edge deployment.
Cost Reduction: By relying on robust insertion strategies rather than high-precision pose estimation, the approach reduces the need for expensive sensors and calibration, handling tolerances up to 10 times larger than typical rule-based systems.
Limitations:
- The method is not zero-shot; it requires specific demonstrations for each connector geometry.
- It does not yet reach the 100% reliability required for full industrial automation without human intervention.
- MSE during training is a poor predictor of actual task success; direct success rate evaluation is necessary.
Future Work: The authors propose using the trained BC model as a pre-training initialization for Reinforcement Learning (RL) to further refine performance and handle distribution shifts.

Conclusion

This study establishes that Behavioral Cloning is a highly effective, data-efficient strategy for robotic connector assembly. By leveraging simple CNNs and force-torque feedback, robots can learn complex insertion strategies that generalize across different geometries and large pose uncertainties, offering a practical alternative to rigid rule-based programming.