RL-Based Coverage Path Planning for Deformable Objects on 3D Surfaces

This paper proposes a reinforcement learning-based approach for coverage path planning on 3D surfaces using deformable objects, which leverages harmonic UV mapping and scaled grouped convolutions to process contact feedback in simulation and successfully demonstrates effective wiping tasks on a physical Kinova Gen3 manipulator.

The Gist

Imagine you have a very complex, bumpy, and weirdly shaped object—like a human torso, a car door with a window cut out, or a crumpled piece of fabric. Your goal is to wipe this entire surface clean with a soft sponge.

If you were a human, you'd just look at the object, feel the bumps with your hand, and naturally figure out how to move the sponge to cover every inch without missing a spot or getting stuck in a hole.

Robots, however, are terrible at this. They usually see the world as a flat grid or a rigid box. If you ask a standard robot to wipe a curved, bumpy surface with a squishy sponge, it gets confused. The sponge stretches, the surface curves, and the robot doesn't know where it is or what it's touching.

This paper presents a clever new way to teach a robot how to be a master cleaner for these tricky jobs. Here is the breakdown of their solution, using some simple analogies:

1. The Problem: The "Flat Map" vs. The "Bumpy Ball"

Imagine trying to draw a map of the entire Earth on a flat piece of paper. You have to stretch and squish the continents to make them fit. If you try to navigate a robot using a 3D model of a bumpy surface, the math gets incredibly complicated. The robot has to calculate millions of points in 3D space to know where to move next. It's like trying to solve a puzzle while wearing thick gloves.

2. The Solution: The "Unfolding Trick" (Harmonic UV Mapping)

The authors' first big idea is to flatten the problem.

• The Analogy: Think of a 3D object (like a basketball or a human arm) as a balloon. If you cut the balloon open and lay it flat on a table, it becomes a 2D shape.
• The Tech: They use a mathematical trick called Harmonic UV Mapping. This takes the complex 3D surface the robot needs to clean and "unwraps" it onto a flat 2D square.
• Why it helps: Instead of the robot trying to navigate a 3D maze, it now just has to draw a line on a flat piece of paper. It's much easier to plan a path on a flat map than on a bumpy ball.

3. The Brain: The "Smart Sponge" (Reinforcement Learning)

Once the surface is flattened, they need a brain to figure out the best path. They don't program the robot with strict rules (like "move left, then right"). Instead, they use Reinforcement Learning (RL).

• The Analogy: Imagine a baby learning to walk. It falls down, gets up, tries again, and slowly learns what works.
• The Process: The robot is placed in a virtual video game (a simulator called MuJoCo). It tries to wipe the "flat map" millions of times. Every time it covers a new spot, it gets a "point" (reward). Every time it wastes time or misses a spot, it gets a "penalty."
• The Feature Extractor (SGCNN): To help the robot "see" the map, they use a special type of AI camera (SGCNN) that looks at the map like a human looks at a maze, spotting patterns and boundaries instantly.

4. The Action: From Paper Back to Reality

Once the robot learns the perfect path on the flat 2D map, the system "re-wraps" that path back onto the original 3D object.

• The Result: The robot now knows exactly how to move its arm in 3D space to follow the path it learned on the flat map.
• The "Squishy" Factor: Because the sponge is soft, the robot doesn't need to be perfect. If the 3D model was slightly wrong, the sponge just squishes a little to fill the gap, ensuring the surface still gets wiped clean.

5. The Results: Better Than the Old Ways

The researchers tested this on 10 different objects (bowls, car doors, human models).

• Old Methods: Tried to use rigid rules (like a lawnmower going back and forth in straight lines). These often missed spots or took very long, winding paths.
• Their Method: The AI learned to take a shorter, smoother path that covered more area. It was like comparing a clumsy person mowing a lawn in straight lines versus a professional gardener who intuitively knows exactly where to cut to get the job done fastest.

The Real-World Test

Finally, they took the robot out of the video game and put it in the real world. They used a real robotic arm (Kinova Gen3) to wipe the back of a human mannequin.

• The Outcome: The robot successfully wiped the entire back, avoiding holes (like the armpits) and covering the curves perfectly.

Summary

In short, this paper teaches a robot how to clean weird, bumpy surfaces by:

1. Flattening the 3D world into a 2D map (like unfolding a balloon).
2. Training an AI in a video game to learn the best path on that map.
3. Translating that path back to the real 3D world, letting the soft sponge handle the small imperfections.

It's a bridge between the messy, flexible real world and the rigid, mathematical world of robots, making them much better at tasks like cleaning, disinfecting, or massaging.

Technical Summary

Here is a detailed technical summary of the paper "RL-Based Coverage Path Planning for Deformable Objects on 3D Surfaces".

1. Problem Statement

The paper addresses the Complete Coverage Path Planning (CPP) problem specifically for deformable objects (e.g., sponges, soft cloths) interacting with complex 3D surfaces (e.g., human torsos, car doors).

• Challenges: Unlike rigid objects, deformable objects change shape (stretch, compress, fold) upon contact, making state representation and trajectory planning difficult. Traditional CPP algorithms assume static, rigid environments and fail to account for dynamic deformations, occlusion, and the need for real-time tactile/force feedback.
• Goal: To generate an efficient path for a robotic manipulator to wipe a 3D surface completely using a deformable tool, minimizing path length while maximizing coverage area, without requiring complex real-time tactile sensing hardware.

2. Methodology

The authors propose a novel pipeline integrating simulation, dimensionality reduction, and Reinforcement Learning (RL).

A. Simulation Environment

• Platform: MuJoCo is used to simulate the interaction between a deformable sponge (modeled as a spring-mass system) and 3D target meshes.
• Data Acquisition: The simulator provides low-cost, real-time contact feedback (force and collision data) which is difficult to acquire in the real world.
• Geometry Handling: Complex concave meshes are decomposed into convex hulls to satisfy MuJoCo's collision detection constraints.

B. State and Action Space Reduction (Harmonic UV Mapping)

To overcome the high dimensionality of 3D mesh states, the authors map the 3D surface to a 2D plane:

• Harmonic UV Mapping: The target 3D surface (assumed homeomorphic to a disk) is parameterized onto a 2D domain (square or unit circle) using harmonic mapping. This creates a one-to-one correspondence between 3D surface points and 2D UV coordinates.
• Observation Space: Instead of raw 3D point clouds, the agent observes a multi-scale 2D map consisting of three layers:
  1. Coverage Map ($M_c$): Binary matrix of covered vs. uncovered regions.
  2. Frontier Map ($M_f$): Boundaries between covered and uncovered areas to guide exploration.
  3. Border Map ($M_b$): Defines valid movable ranges (handling holes/inaccessible areas).
• Egocentric View: The maps are translated, rotated, and scaled relative to the agent's current position to eliminate the need for the agent to learn its own pose.

C. Feature Extraction and Policy

• Feature Extractor: A Scaled Grouped Convolutional Neural Network (SGCNN) processes the multi-scale 2D maps to extract spatial features efficiently.
• Action Space: The agent outputs actions in the 2D UV coordinate system (angular velocity $\omega$).
  • Linear velocity is fixed.
  • The 2D action is projected back to 3D space to control the robotic arm's end-effector, ensuring the movement remains perpendicular to the contact surface.

D. Reward Function

The reward function is designed to optimize coverage efficiency and path smoothness:
$$\text{reward} = r_c + r_{\Delta TV} + r_{const}$$

• $r_c$ (Coverage Reward): Proportional to the number of newly covered pixels.
• $r_{\Delta TV}$ (Total Variation Reward): Penalizes the change in Total Variation of the coverage map. This encourages the agent to fill gaps and reduce "streaks" or isolated uncovered holes, promoting a smoother, more contiguous coverage.
• $r_{const}$: A negative constant to encourage efficiency (shorter paths).

3. Key Contributions

1. Dimensionality Reduction via UV Mapping: The introduction of harmonic UV mapping to project 3D deformable surface states and actions onto a 2D plane. This significantly simplifies the RL problem, enhancing computational efficiency and convergence speed.
2. Efficient RL Pipeline: A tailored processing pipeline combining simulation-based force feedback, 2D state representation, and SGCNN feature extraction.
3. Sim-to-Real Validation: Successful deployment of the learned policy on a Kinova Gen3 robotic arm to perform wiping tasks on a reconstructed human torso model, demonstrating feasibility without active tactile sensors (relying on the sponge's physical compliance).

4. Experimental Results

The method was evaluated on 10 bowl-shaped objects and complex geometries (car doors, human torsos) against three baselines: SPONGE (learning-based TSP), Zigzag, and Spiral (rule-based).

• Path Length: The proposed method achieved the shortest total path length (7.54m total across 10 objects), outperforming SPONGE by ~21% and the Spiral method by ~5%.
• Coverage Area: Achieved a high average coverage of 95.5%, outperforming SPONGE (94.2%) and the Spiral method (87.0%). It was comparable to the Zigzag method but with significantly shorter paths.
• Path Smoothness: The cumulative sum of Z-axis rotation angle changes ($S|\Delta\gamma|$) was 133.81, significantly lower (better) than SPONGE's 187.73, indicating smoother end-effector movements.
• Complex Geometries: The agent successfully navigated around holes in car door models, avoiding inaccessible regions while covering the rest of the surface.
• Real-World Performance: Experiments on a human torso model confirmed the system could clean a marked area effectively, validating the sim-to-real transfer.

5. Significance and Limitations

Significance:

• This work bridges the gap between rigid CPP and the complex reality of deformable object manipulation.
• It demonstrates that simulation-based training combined with geometric dimensionality reduction can solve high-dimensional manipulation tasks without expensive real-world sensor data.
• The approach offers a practical solution for medical rehabilitation (e.g., disinfecting patients) and industrial cleaning where surface geometry is complex and objects are soft.

Limitations:

• Kinematic Constraints: The planning occurs in UV space and does not fully account for the robotic arm's joint limits, occasionally resulting in unreachable poses in the real world.
• Sim-to-Real Gap: The spring-mass model in MuJoCo cannot perfectly replicate the deformation physics of a real sponge, though the sponge's compliance helps mitigate errors.
• Static Planning: The current approach is offline. It does not yet handle dynamic objects (e.g., a moving patient) in real-time, which is a target for future work involving visual servoing.