Learning Adaptive Force Control for Contact-Rich Sample Scraping with Heterogeneous Materials

Imagine you are a scientist trying to clean a very sticky, messy jar. Inside, there's a mix of powders, crystals, and gooey pastes clinging to the glass walls. Your goal is to scrape every last bit of this material off the sides so you can weigh it or study it.

If you were a human, you'd do this effortlessly. You'd look at the jar, see where the mess is, grab a spatula, and gently wiggle it against the glass. If the stuff is hard, you'd press a little harder. If it's soft, you'd be gentle. You'd adjust your force in real-time based on what you see and feel.

Now, imagine trying to teach a robot to do this. This is the problem the paper solves.

The Problem: Robots Are Too "Stiff"

Traditionally, robots are like rigid machines. They are programmed to move to a specific spot and push with a specific amount of force.

The Flaw: If the robot pushes with the same force on a soft paste and a hard crystal, it will either fail to move the hard stuff or smash the glass jar when dealing with the soft stuff.
The Challenge: In a real lab, every sample is different. Some are sticky, some are dry, some are wet. A "one-size-fits-all" force doesn't work.

The Solution: A Robot with "Feel" and "Brain"

The authors created a system that gives the robot two superpowers: Compliant Touch and Adaptive Learning.

1. The "Compliant Touch" (The Low-Level Controller)

Think of the robot's arm as a springy, rubbery arm rather than a steel rod. This is called a Cartesian Impedance Controller.

Analogy: Imagine holding a toothbrush against a wall. If you push too hard, the bristles bend. If you push gently, they just touch. This robot arm behaves like those bristles. It doesn't fight the glass; it yields to it. This ensures the robot never shatters the expensive glass vial, even if it pushes a little too hard.

2. The "Adaptive Brain" (The High-Level RL Agent)

This is the real magic. The robot has a "brain" (an AI trained with Reinforcement Learning) that acts like a student learning by trial and error.

How it learns: The robot tries to scrape the jar.
- If it pushes too hard and the material doesn't move, it learns, "Okay, I need to change my angle or force."
- If it pushes too soft and nothing happens, it learns, "I need to press harder."
- The Goal: It learns to find the "Goldilocks" force—the exact amount of pressure needed to dislodge the material without breaking the jar.
The Eyes: The robot isn't just guessing. It has a camera (RGB-D) that acts like its eyes. It looks inside the jar, identifies where the messy stuff is, and tells the brain, "Hey, there's a clump of sugar right there, go scrape that!"

The Simulation: The "Video Game" Training Ground

Before letting the robot loose in a real lab, the researchers built a virtual world (a simulation).

The Setup: They created a digital Franka robot, a digital spatula, and a digital jar.
The Secret Sauce: Instead of making the "dirt" look the same every time, they used a special noise generator (Perlin noise) to make the virtual dirt have random "hardness" levels. Some virtual grains were soft like butter; others were hard like rocks.
The Result: The AI played thousands of hours of this "video game," learning to handle every possible type of mess. It learned a general strategy: Look, feel, adjust, and scrape.

The Real-World Test: From Game to Lab

Once the AI was a master in the simulation, they transferred it to a real robot in a real chemistry lab.

The Test: They tried it on five very different materials:
1. Liquid dough (sticky and thick).
2. Cornflour paste (thick and gooey).
3. Dried cornflour (hard and crumbly).
4. Salt crystals.
5. Sugar crystals.
The Comparison: They compared their "Smart Robot" against a "Dumb Robot" (one that just pushes with a fixed, unchanging force).
The Outcome: The Smart Robot was 10.9% better on average.
- On the hardest materials (like sugar crystals), the fixed-force robot struggled, often failing to clean the jar.
- The Smart Robot adapted its force, successfully cleaning the jar almost as well as a human scientist could.

Why This Matters

This isn't just about cleaning jars. It's about accelerating science.

The Old Way: Human scientists spend hours doing repetitive, messy tasks like scraping jars. It's boring, dangerous (if the chemicals are toxic), and inconsistent.
The New Way: This robot can do it autonomously. Because it can adapt to any material, we can finally build "Self-Driving Laboratories" where robots run experiments 24/7, discovering new medicines or clean energy materials much faster than humans ever could.

In a nutshell: The paper teaches a robot to stop being a rigid machine and start acting like a skilled human chemist: looking at the mess, feeling the resistance, and adjusting its grip to get the job done perfectly.

Here is a detailed technical summary of the paper "Learning Adaptive Force Control for Contact-Rich Sample Scraping with Heterogeneous Materials."

1. Problem Statement

The paper addresses a critical bottleneck in autonomous scientific discovery: the inability of current robotic chemists to handle contact-rich manipulation tasks involving heterogeneous materials.

The Challenge: In early-stage materials discovery, samples often adhere to the walls of glass vials in unpredictable ways (ranging from granular powders to cohesive pastes). Humans use spatulas to scrape these materials, relying on dexterity and adaptive force control.
Limitations of Current Systems: Existing robotic systems in labs rely heavily on position control or fixed-force profiles. These approaches fail when material properties (hardness, adhesion, viscosity) vary, as they cannot adapt to the resistance encountered during scraping. Furthermore, standard force control is complicated by tool deformation (bending spatulas), meaning wrist forces do not directly correlate to tip forces.
Goal: To develop an autonomous system capable of scraping heterogeneous materials from vial walls without prior knowledge of the material's physical properties, using visual feedback to guide adaptive force application.

2. Methodology

The authors propose a hierarchical adaptive control framework that decouples high-level decision-making from low-level physical execution.

A. Control Architecture

The system consists of two main layers:

Low-Level Controller (Cartesian Impedance Control - CIC):
- A stable, compliant controller running at 500 Hz.
- It regulates the robot's end-effector to behave like a mass-spring-damper system, ensuring safe interaction with fragile glassware.
- It accepts a target Cartesian wrench (force/torque) and position as a goal, generating compliant joint torques.
High-Level Agent (Reinforcement Learning - RL):
- An RL agent running at 10 Hz that learns a policy to generate the target wrench ( $F^c_{ext}$ ) and vertical position ( $z_D$ ).
- Action Space: A hybrid command vector $[f^c_x, \tau^c_y, z_D]^T$ $[f_{x}^{c}, τ_{y}^{c}, z_{D}]^{T}$ , representing:
  - $f^c_x$ : Normal force against the vial wall.
  - $\tau^c_y$ : Torque for tangential scraping motion.
  - $z_D$ : Vertical sweeping position.
- State Space: Includes robot kinematics, external wrench feedback, and crucially, perception-based state (3D centroids and residue percentages of material clusters).

B. Perception Pipeline

To enable the RL agent to know where to scrape, a multi-stage computer vision pipeline processes RGB-D data from an end-effector-mounted camera:

Localization: YOLO detects the vial.
Segmentation: GrabCut isolates the vial from the background.
Depth Filtering: A dynamic threshold isolates the front-facing material (closest to the camera) to avoid occluded back-wall regions.
Tool Removal: K-means clustering in HSV space removes the spatula (colored green for contrast) from the material mask.
Feature Extraction: The remaining material is clustered into groups, outputting centroids and residue percentages as structured input for the RL policy.

C. Simulation and Training

Environment: Built in MuJoCo using a Franka Research 3 robot.
Material Modeling: Instead of a single rigid body, materials are modeled as hundreds of discrete spheres. Each sphere has a unique dislodgement force threshold procedurally generated using Perlin noise. This creates a highly heterogeneous distribution, forcing the agent to learn a generalizable strategy rather than memorizing a specific material.
Reward Function: Designed to maximize material removal efficiency while minimizing force and avoiding collisions.
- $R_{total} = R_M (\text{material removed} / \text{force}) + R_E (\text{milestone bonus}) - \lambda R_C (\text{collision penalty})$ .
Training Strategy: The agent is trained exclusively in simulation and transferred to the real robot using a zero-shot approach (no fine-tuning on real data).

3. Key Contributions

Novel Adaptive Framework: A hybrid architecture combining a low-level Cartesian Impedance Controller with a high-level RL agent that learns to output target Cartesian wrenches. This separates force learning from joint control, simplifying the RL action space.
Perception-Driven Force Control: Integration of a multi-stage visual pipeline that provides real-time feedback on material location and residue, allowing the robot to operate without a priori knowledge of material distribution.
Robust Sim-to-Real Transfer: Demonstration of successful zero-shot transfer from a complex, procedurally generated simulation to a real-world chemistry lab across five distinct material types.
Empirical Evaluation: Comprehensive testing on diverse materials (liquids, pastes, dried solids, crystals) showing significant improvements over fixed-force baselines.

4. Experimental Results

The system was evaluated on a Franka Research 3 robot with five material types: Liquid Dough, Liquid Cornflour, Dried Cornflour, Crystalline Salt, and Crystalline Sugar.

Baseline vs. RL:
- Fixed Wrench Baseline: Achieved an average relative success rate of 64.44% (relative to human performance). It struggled significantly with crystalline sugar (41.0%) and liquid dough.
- RL Adaptive Method: Achieved an average relative success rate of 75.3%, outperforming the baseline by 10.9%.
Material-Specific Performance:
- The RL agent showed the most significant gains on non-Newtonian materials (e.g., liquid cornflour), where it adapted to shear-thickening behavior.
- Performance on crystalline sugar approached human levels (66.4% relative success vs. 41.0% for baseline), attributed to the good correspondence between the discrete simulation model and rigid crystal behavior.
Perception Accuracy: The perception pipeline achieved high precision in locating material, though accuracy dropped slightly (~15%) when filtering out the spatula, a trade-off deemed acceptable for the task.

5. Significance and Conclusion

This work represents a significant step forward in autonomous scientific experimentation.

Overcoming Heterogeneity: It proves that robots can handle the unpredictable nature of early-stage materials discovery, a task previously limited to human chemists due to the need for dexterity and force adaptation.
Safety and Compliance: By using impedance control, the system ensures safe interaction with expensive glassware and fragile samples.
Scalability: The zero-shot sim-to-real transfer suggests that such systems can be rapidly deployed to new tasks and materials without extensive retraining.
Future Impact: This framework paves the way for "force-aware" robotic chemists capable of performing complex, contact-rich workflows (like powder recovery) autonomously, accelerating the pace of discovery in fields like drug development and solid-state chemistry.

The paper concludes that while challenges remain in modeling complex slurries and tool fouling, the integration of learned adaptive control with compliant execution is a viable path toward fully autonomous laboratory operations.