In-Hand Manipulation Planning for Grippers with Active Surfaces

Robotic hands have long struggled with a specific kind of dexterity that human hands perform without thinking: the ability to slide an object across the skin of a finger while holding it. For decades, engineers trying to build robotic hands have focused on mimicking the complex, multi-jointed structure of the human hand, hoping that more joints would lead to more skill. However, these complex designs often become difficult to control and plan for, limiting their use in real-world tasks. A simpler alternative has emerged: grippers with "active surfaces." These are robotic fingers equipped with mechanisms like moving belts or surfaces that can change their stickiness, allowing them to deliberately slide an object along the finger rather than just gripping it in place. While this hardware capability exists, the software that tells the robot how to move has lagged behind. Traditional planning software assumes that a robot's fingers only roll over an object like a wheel on a road, treating any sliding as a mistake to be avoided. This assumption breaks down completely when the robot is designed to slide things on purpose, leaving engineers with powerful hardware that the software cannot fully utilize.

Researchers at Worcester Polytechnic Institute and the RAI Institute have developed a new planning framework that solves this disconnect by changing how the robot "thinks" about the object it is holding. Instead of mapping the object's movement solely based on the object's own surface, their new system tracks the object's position relative to the moving surface of the finger itself. Imagine trying to navigate a city while ignoring the fact that the streets are moving beneath your feet; traditional planners try to map the city while the streets shift, leading to confusion and failure. The new approach acknowledges that the streets are moving and plans the route based on the relationship between the car and the road. By creating a combined state space that accounts for both the object and the finger surface simultaneously, the system can plan complex sequences of sliding and rotating that were previously impossible to calculate.

The researchers tested this new thinking on two very different types of active-surface grippers. The first was a Variable Friction gripper, which uses a mechanism to make parts of its fingers slippery or sticky on command. The second was a Belt Orienting Phalanges gripper, which uses small conveyor belts on its fingers to push objects. In physical experiments with the friction-based gripper, the system successfully planned and executed tasks involving six different object shapes, including hexagons, stars, and curved pieces. The robot could slide objects 40 millimeters across its fingers and rotate them to face different directions with high precision. In these physical trials, the planner generated a path in less than a second on average, and the robot successfully completed 96 percent of the 135 physical attempts. The few failures were not due to the planning logic but rather to the physical properties of the objects, such as a gelatin box shifting its weight during the move, which threw off the final position slightly.

The power of this new framework became even more apparent when the researchers asked the robot to perform a more complex task: reconfiguring a Rubik's cube. The goal was not just to turn the cube, but to slide it within the grip until a specific layer stuck out, making it accessible for a human to twist. This required a sequence of sliding the cube toward a pinch grip, sliding it back to create space, rotating it, and then sliding it out again. Traditional planning methods, which rely on the object's surface as the map, failed to see a continuous path for this task because the contact points on the object changed in ways that their software could not connect. The new planner, however, saw the continuous motion along the finger surface and successfully generated the sequence. In a separate set of experiments using a simulation of the belt-driven gripper, the new planner achieved a 100 percent success rate across all tasks, while existing planning methods failed on the majority of them, often causing the robot to drop the object or collide with its own structure.

The study demonstrates that the limitation was never the hardware's ability to move, but the software's inability to visualize the motion correctly. By abandoning the old rule that sliding is an error and instead treating the finger surface as a dynamic part of the planning map, the researchers enabled robots to perform in-hand manipulation tasks that were previously infeasible. The results show that this approach works across different types of active surfaces, from friction modulation to conveyor belts, suggesting a unified way to program dexterity for a wide range of future robotic hands. The system does not require complex sensors to see the object in real-time; it relies on knowing the shape of the object and the capabilities of the gripper to calculate a path that is physically possible before the robot even begins to move. This shift from object-centric to finger-object-centric planning opens the door for robots to handle complex, real-world tasks with a level of fluidity that was previously the domain of human hands alone.