🤖 machine learning

Learning to Throw Objects Safely in Multi-Obstacle Environments

This paper introduces a potential field state representation combined with reinforcement learning to enable robots to reliably throw objects into target baskets while avoiding random obstacles in cluttered environments, achieving up to 90% success in real-world experiments.

Original authors: Mohammadreza Kasaei, Klemen Voncina, Hamidreza Kasaei

Published 2026-07-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mohammadreza Kasaei, Klemen Voncina, Hamidreza Kasaei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot arm trying to toss a ball into a basket. Now, imagine that the room is full of random furniture, boxes, and other items scattered everywhere. If the robot just guesses where to throw, it might hit a chair, bounce off a wall, or miss the basket entirely. This is the problem the paper solves: teaching a robot to throw things safely and accurately through a messy room.

Here is a breakdown of how they did it, using simple analogies:

1. The Problem: Throwing in a Minefield

Most robots are great at picking things up and placing them down gently. But throwing is faster and can reach places a robot's arm can't physically stretch to. The catch? In a messy room, a straight line to the basket might be blocked by a wall or a box. The robot needs to figure out: "Can I throw this? If so, what arc should I take to avoid hitting the junk?"

2. The Solution: A "Magnetic Map" (Potential Fields)

The researchers realized that telling the robot exactly where every single obstacle is (like listing coordinates for 5 different boxes) is too complicated. If you add a sixth box, the robot's brain has to be retrained from scratch.

Instead, they gave the robot a magnetic map, which they call a "Potential Field."

The Basket is a Magnet: Imagine the basket is a giant magnet pulling the object toward it (a positive force).
The Obstacles are Repellers: Imagine every obstacle is a magnet pushing the object away (a negative force).
The Grid: The robot looks at the room as a flat, grid-like map (like a chessboard). On this map, the "pull" of the basket and the "push" of the obstacles blend together.

This is like giving the robot a weather map where the basket is a low-pressure system (pulling air in) and obstacles are high-pressure systems (pushing air out). The robot just needs to follow the smooth "wind" of this map to find a safe path, regardless of how many obstacles are in the room. This allows one single "brain" to handle a room with 1 obstacle or 100 obstacles without needing to be retrained.

3. The Training: "Hand-Holding" Before "Free-Throwing"

You wouldn't let a toddler try to throw a ball through a minefield on their first try; they'd get hurt or break something. The researchers used a technique called Kinesthetic Teaching.

The Analogy: A human physically grabs the robot's arm and guides it through a safe throwing motion.
The Result: This gives the robot a "safe kernel"—a basic, pre-approved throwing motion that won't crash into anything. The robot doesn't just copy the human; it learns to tweak that safe motion (changing the angle or release time) based on the magnetic map to hit the target. This prevents the robot from making dangerous, random guesses while it is learning.

4. The Learning Process: Trial and Error with a Coach

The robot practices in a computer simulation (a virtual room) using a method called Reinforcement Learning.

The Coach: The computer acts as a coach. If the robot throws the ball into the basket, it gets a "high five" (a reward). If it hits an obstacle or misses, it gets a "frown" (a penalty).
The Algorithms: They tested three different "coaching styles" (algorithms named SAC, DDPG, and TD3). They found that SAC was the best coach, helping the robot learn the fastest and most consistently.

5. The Results: From Virtual to Real

In the Simulation: The robot learned to throw objects (like milk boxes, bananas, and even sneakers) into baskets while dodging up to 5 random obstacles. The "Magnetic Map" approach worked better than the old way of listing obstacle coordinates, especially when the number of obstacles changed.
In the Real World: They took the trained robot to the real world. Even though the real world is messier (cameras aren't perfect, and the robot's hand might be slightly delayed), the robot still succeeded about 90% of the time when throwing unseen objects (like a sneaker) into a basket, even with obstacles in the way.

Summary

The paper shows that by giving a robot a "magnetic map" of the room (where the goal pulls and obstacles push) and teaching it a safe starting throw by hand, the robot can learn to toss objects through cluttered environments. It's a way to make robots faster and more efficient in warehouses or sorting centers without them constantly crashing into things.

Technical Summary: Learning to Throw Objects Safely in Multi-Obstacle Environments

Problem Statement
Robotic throwing offers a mechanism to increase throughput and extend the effective workspace of automated systems by enabling fast object placement beyond immediate reach. However, existing approaches, such as TossingBot, largely assume obstacle-free environments. In practical, unstructured settings (e.g., warehouses or sorting centers), direct throwing trajectories are frequently blocked by obstacles like walls, bins, or other items. The core challenge addressed in this paper is formulating robotic throwing as a Safe Reinforcement Learning (RL) problem: the robot must learn to maximize the success rate of throwing an object into a target basket while strictly avoiding collisions with randomly placed obstacles.

Methodology
The authors formulate the task as a Markov Decision Process (MDP) with continuous state and action spaces, employing off-policy RL algorithms (SAC, DDPG, and TD3). The methodology is built upon three primary pillars:

Safe Initialization via Kinesthetic Teaching:
To prevent unsafe random exploration in cluttered scenes during early training, the throwing motion is initialized using kinesthetic teaching. A human demonstrator physically guides the robot arm to define a safe, physically feasible "throwing kernel." The RL policy does not imitate this demonstration directly but learns to modulate the kernel's parameters (initial/final shoulder joint values, release time, and throw duration) to adapt to varying objects and obstacle configurations.
State Representations:
The paper compares two distinct methods for encoding environmental information:
- Explicit Pose Representation (EPR): Obstacles are encoded by appending their center positions and relative distances to the state vector. While this allows for fast convergence, the state dimensionality grows linearly with the number of obstacles, requiring a separate policy for each specific configuration.
- Potential Field Representation (PFR): This novel approach encodes the basket (attraction) and obstacles (repulsion) onto a fixed-size grid ( $15 \times 15$ ) covering the workspace. Each cell stores a continuous potential value. This representation is processed by a Convolutional Neural Network (CNN) encoder to extract spatial features, which are then concatenated with proprioceptive data. Crucially, PFR provides a fixed-dimensional input regardless of the number of obstacles, enabling a single policy to generalize across arbitrary obstacle counts and configurations.
Reward Function:
The reward structure penalizes collisions (setting reward to 0 if an obstacle is moved) and rewards successful landings (reward of 1). For non-colliding throws that miss the basket, a composite reward function is used: a term based on the distance to the goal (encouraging proximity) and a penalty term that activates only if the object ends up further from the goal than it started, discouraging random directional throws.

Key Contributions

Problem Formulation: The authors formulate robotic throwing in the presence of obstacles as a safe RL challenge, leveraging kinesthetic teaching to bootstrap the learning process and ensure safety during early exploration.
Scalable State Representation: The introduction of the Potential Field Representation (PFR) allows for a single policy to generalize across arbitrary numbers and configurations of obstacles, overcoming the scalability limitations of explicit pose encodings.
Empirical Validation: Extensive experiments in simulation and on a real UR5e robot demonstrate that PFR-based policies achieve high success rates and robust sim-to-real transfer, even with unseen throwable objects.

Results

Algorithm Performance: Among the tested algorithms (SAC, TD3, DDPG), Soft Actor-Critic (SAC) demonstrated the most consistent performance.
PFR vs. EPR: In simulation, PFR consistently outperformed EPR in cluttered scenarios. For instance, with three obstacles, SAC with PFR achieved 90% accuracy on a "Coke Can" compared to 76% with EPR. PFR maintained success rates above 90% with up to three obstacles, whereas EPR performance degraded more significantly as clutter increased.
Grid Resolution: An ablation study on grid resolution revealed that a $15 \times 15$ grid offered the optimal trade-off between training efficiency and spatial fidelity, outperforming coarser ( $10 \times 10$ ) and finer ( $30 \times 30$ ) grids in terms of generalization and computational cost.
Real-Robot Transfer: Policies trained in simulation were deployed zero-shot on a real robot. Using the PFR approach with SAC, the system achieved a 90% success rate in cluttered scenes with a larger target basket and a 70% success rate with a standard-sized basket, using an unseen object (a sneaker). This confirmed the robustness of the sim-to-real transfer.

Significance and Claims
The paper claims that the Potential Field Representation provides a practical and robust solution for safe robotic throwing in unstructured environments. By balancing accuracy, scalability, and transferability, the proposed framework enables robots to learn throwing strategies that generalize to arbitrary obstacle configurations without requiring retraining for each new scene. The authors assert that this approach brings robotic throwing closer to real-world deployment in logistics, manufacturing, and service applications.

The paper acknowledges limitations, noting that failures in real-world experiments were primarily attributed to inaccurate object pose estimation, delays in gripper command execution, and unstable grasp pose selection. However, the overall results demonstrate that combining potential fields with reinforcement learning effectively addresses the challenge of safe, efficient object placement in cluttered environments.