💻 computer science

SHaRe-RL: Structured, Interactive Reinforcement Learning for Contact-Rich Industrial Assembly Tasks

SHaRe-RL is a structured, interactive reinforcement learning framework that combines manipulation primitives, human demonstrations, and per-axis compliance to enable safe, sample-efficient online learning for high-precision, contact-rich industrial assembly tasks in high-mix low-volume environments.

Original authors: Jannick Stranghöner, Philipp Hartmann, Marco Braun, Sebastian Wrede, Klaus Neumann

Published 2026-03-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jannick Stranghöner, Philipp Hartmann, Marco Braun, Sebastian Wrede, Klaus Neumann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a clumsy, super-strong robot how to plug a very delicate, oddly shaped electrical connector into a socket. The gap between the plug and the socket is thinner than a human hair (about 0.2 to 0.4 millimeters). If the robot pushes too hard, it breaks the parts. If it wiggles too much, it misses the hole.

This is the challenge of High-Mix Low-Volume (HMLV) manufacturing: factories that make many different products in small batches. They need robots that are as precise as a surgeon but flexible enough to handle new products every day.

The paper introduces a new system called SHaRe-RL to solve this. Here is how it works, explained through simple analogies.

The Problem: The "Trial and Error" Trap

Standard Reinforcement Learning (RL) is like teaching a dog by letting it run around a field and only giving it a treat when it finally sits.

The Issue: In a factory, if a robot "tries" to plug in a connector 1,000 times and breaks the part 999 of those times, the factory loses money, and the robot learns nothing useful. It's too dangerous and too slow to just let the robot guess.

The Solution: SHaRe-RL (The "Smart Apprentice")

SHaRe-RL is a hybrid system that combines three things to teach the robot safely and quickly. Think of it as training a new apprentice using a Master Plan, a Human Mentor, and a Safety Harness.

1. The Master Plan (Structured Primitives)

Instead of asking the robot to figure out the entire process from scratch, the engineers break the job into small, logical steps called "Manipulation Primitives."

Analogy: Imagine a recipe. You don't tell the robot, "Make a cake." You give it steps: "Mix flour," "Add eggs," "Bake."
In the paper: The robot has a script for "Approach the socket" and "Pull back." It only needs to learn the tricky part: "The final wiggle to get the plug in." This reduces the problem from a giant maze to a tiny, manageable puzzle.

2. The Human Mentor (Human-in-the-Loop)

The robot doesn't learn alone. A human operator watches and can step in.

Analogy: Think of a driving instructor. When the student (the robot) is about to crash, the instructor grabs the wheel (takes control) to steer them back to safety.
In the paper: If the robot starts to jam the plug, the human presses a button to take over, fixes the position, and lets the robot try again. The robot learns from these corrections. Over time, the human intervenes less and less, until the robot can do it alone.

3. The Safety Harness (Adaptive Force Limits)

This is the paper's secret weapon. The robot has a "smart muscle" that knows when to be strong and when to be gentle.

Analogy: Imagine a person reaching for a doorknob. In the air, they swing their arm fast. The moment their hand touches the knob, they instantly soften their grip so they don't slam it.
In the paper: The system sets a rule: "If you are in the air, you can push hard. The moment you feel any resistance (contact), your maximum pushing power instantly drops to a safe, gentle level." This prevents the robot from smashing the parts while it is still learning how to wiggle them in.

The Results: From Clumsy to Master

The researchers tested this on a real industrial connector (the Harting HanDD).

Speed: The robot learned to do the task perfectly in just 3 hours of real-world time.
Performance: It eventually became faster (3.5 seconds per task) than the skilled human who taught it (4.5 seconds).
Generalization: Even when they gave the robot a different type of connector it had never seen before, it figured it out immediately without needing to relearn.

Why This Matters

Before this, making robots flexible enough for small factories was too expensive and risky. You needed an expert engineer to program every single movement.

The Old Way: "Here is the code. If the product changes, hire an engineer to rewrite the code." (Slow, expensive, brittle).
The SHaRe-RL Way: "Here is the general plan. Watch the robot learn from a human operator for a few hours, and it will adapt to new products on its own." (Fast, safe, cheap).

In summary: SHaRe-RL is like giving a robot a map, a safety net, and a patient teacher. It allows robots to learn difficult, delicate tasks in the real world without breaking anything, making automation accessible even for small businesses that make many different products.

1. Problem Statement

The paper addresses the challenge of automating High-Mix Low-Volume (HMLV) industrial assembly, particularly in Small and Medium-sized Enterprises (SMEs). These environments require robots to handle diverse product variants with high precision, safety, and reliability while adapting to environmental uncertainties.

Current Limitations: Traditional manual programming is brittle and costly to adapt. Pure Reinforcement Learning (RL) approaches struggle with sample inefficiency, unsafe exploration in contact-rich scenarios (risk of hardware damage), and the difficulty of defining reward functions for sparse success signals.
Specific Task: The authors focus on the insertion of industrial Harting connector modules, a task characterized by sub-millimeter tolerances (0.2–0.4 mm) and complex contact dynamics that are difficult to solve end-to-end.

2. Methodology: SHaRe-RL

The authors propose SHaRe-RL (Structured, Human-in-the-loop, and Real-world RL), a hybrid framework that integrates three complementary sources of prior knowledge into an off-policy RL pipeline:

A. Structured Task Abstraction (Manipulation Primitive Nets)

Instead of learning a monolithic policy for the entire task, SHaRe-RL decomposes the assembly process into Manipulation Primitives (MPs) organized into a state machine (MP-Net).

Hybrid Control: Primitives are defined as tuples containing control policies (force/position), tool commands, and stop conditions.
Adaptive Primitives (AMPs): Only the critical, uncertain phases (e.g., fine alignment during insertion) are controlled by the RL policy. Repetitive or deterministic phases (e.g., approach, retreat) are executed via fixed setpoints.
Benefit: This drastically reduces the effective state-action space, focusing learning only on the "hard" parts of the task.

B. Human-in-the-Loop (HIL) Guidance

The system leverages operator expertise through two mechanisms:

Offline Demonstrations: A skilled operator provides teleoperated trajectories using a SpaceMouse, stored in a demonstration buffer.
Online Corrections: During training, the operator can intervene at any time to correct the robot's actions. This biases data collection toward safe and successful behaviors, helping the policy escape local optima.
Algorithm: The system uses RLPD (a regularized variant of Soft Actor-Critic), which samples equally from the demonstration buffer and the on-policy buffer to update the policy.

C. Adaptive Per-Axis Safety Limits

To ensure safety during exploration in contact-rich environments, the authors introduce a deterministic safety layer:

Mechanism: An adaptive force limit ( $F_{lim}$ ) is applied per axis. In free space, limits are high to allow responsiveness. Upon contact detection ( $F_{meas} > 0$ ), the limit contracts based on a monotone decreasing function.
Theoretical Guarantee: The system is designed to converge to a steady-state force ( $F^*_{lim}$ ) that is strictly bounded, preventing destructive forces while preserving free-space dynamics. This is proven via stability analysis of the recurrence relation.

3. Key Contributions

First Integration of MP-Nets with Off-Policy HIL-RL: SHaRe-RL is the first system to combine manipulation primitive nets with off-policy, human-in-the-loop reinforcement learning.
Adaptive Safety Layer: A novel, provably safe compliance mechanism that bounds interaction forces during exploration without requiring manual tuning or sacrificing free-space performance.
Empirical Superiority: Demonstrated robust performance on a sparse-reward, sub-millimeter tolerance task within a 3-hour real-world training budget, outperforming both state-of-the-art RL baselines and skilled human demonstrators in cycle time.
Generalization: The learned policy successfully generalized to previously unseen connector variants without fine-tuning.
Open Source Implementation: The framework is implemented as an extension of the open-source LeRobot framework, making it accessible for low-cost manipulators.

4. Experimental Results

The system was evaluated on a UR3e robot inserting Harting HanDD connectors.

Performance Metrics:
- SHaRe-RL: Achieved 100% success rate with a cycle time of 3.5 seconds.
- Human Expert: 100% success but with a slower cycle time of 4.5 seconds.
- Unstructured HIL-RL (HIL-SERL): Only reached 65% success with a 7.4s cycle time.
- Behavior Cloning (SHaRe-BC): Reached 80% success but plateaued, unable to surpass the quality of the human demonstrations.
Ablation Studies:
- Removing online interventions or offline demonstrations significantly reduced success rates (to 45% and 35%, respectively), confirming the necessity of both human guidance sources.
- Removing vision input collapsed performance to 10%, highlighting the critical role of visual feedback for resolving geometric ambiguity.
- On-policy variants (PPO) failed (5% success), proving the superiority of off-policy methods for sparse rewards.
Zero-Shot Generalization: The policy trained on one connector variant achieved high success rates (90–100%) on unseen variants, demonstrating that it learned a generalized insertion strategy rather than memorizing specific coordinates.
Safety Validation: In high-force collision tests, the adaptive limits successfully bounded contact forces to a target of 5N (from a 20N command) within 0.3 seconds, preventing hardware damage.

5. Significance and Impact

Bridging the Gap: SHaRe-RL effectively bridges the gap between rigid, hand-engineered control (safe but inflexible) and pure RL (flexible but unsafe/inefficient).
Industrial Viability: By leveraging existing process expertise (MP-Nets) and operator knowledge, the framework makes RL deployment in SMEs economically viable. It reduces the training time from days/weeks to hours and ensures safety without complex model-based safety shields.
Future Directions: The authors suggest that while SHaRe-RL works well for specific skill families, future work should focus on reducing the manual effort required to define the primitive graphs (e.g., learning the graph structure from data) and integrating foundation models for broader generalization across diverse industrial tasks.

In conclusion, SHaRe-RL demonstrates that process expertise alone can effectively guide real-world RL, enabling safe, robust, and economically feasible automation for complex, contact-rich industrial assembly tasks.