💻 computer science

GRITS: A Spillage-Aware Guided Diffusion Policy for Robot Food Scooping Tasks

The paper presents GRITS, a spillage-aware guided diffusion policy that utilizes a trained spillage predictor to steer robot food scooping trajectories toward safer outcomes, achieving high task success and significantly reduced spillage across diverse and unseen food categories.

Original authors: Yen-Ling Tai, Yi-Ru Yang, Kuan-Ting Yu, Yu-Wei Chao, Yi-Ting Chen

Published 2026-04-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yen-Ling Tai, Yi-Ru Yang, Kuan-Ting Yu, Yu-Wei Chao, Yi-Ting Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to scoop a bowl of loose, slippery rice balls or a pile of tiny, rolling peas into a spoon without making a mess. If you move your hand too fast, the food flies off. If you move too slowly, they might stick to the bowl. If you tilt the spoon wrong, everything spills.

Now, imagine a robot trying to do this. For a long time, robots were like clumsy toddlers learning to walk: they could copy what a human did, but if the food looked slightly different (maybe bigger, stickier, or more slippery), the robot would often spill the food everywhere.

This paper introduces GRITS (a catchy acronym for RobotIc Food Scoop TaskS), which is like giving the robot a "sixth sense" for avoiding spills.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Clumsy Robot"

Most robots learn by watching humans (like a student copying a teacher). They memorize the path the human's hand took.

The Issue: If the teacher scooped a bowl of dry beans, the robot learned that path. But if you give the robot a bowl of wet, sticky noodles, that same path might cause a disaster. The robot doesn't "understand" that the food is different; it just blindly follows the old map.

2. The Solution: The "Spillage Crystal Ball"

The researchers built a special AI tool called a Spillage Predictor. Think of this as a crystal ball that the robot looks into before it moves.

How it works: Before the robot actually scoops, the crystal ball simulates the move in its head. It asks, "If I take this path with these specific peas, will they fly out?"
The Training: You can't train a robot to spill food 4,000 times in a real kitchen (it's a mess!). So, the team built a super-realistic virtual kitchen (a video game world) where they created thousands of fake food items—balls, cubes, cones—and made the robot spill them over and over. The crystal ball learned to recognize the patterns that lead to a mess.

3. The Magic Trick: "Guided Diffusion"

The robot uses a type of AI called a Diffusion Policy.

The Analogy: Imagine the robot is trying to draw a perfect line, but it starts with a messy, scribbled-up sketch (noise). It slowly erases the bad parts and refines the line until it's perfect. This is how the robot plans its movement.
The Twist: Usually, the robot just tries to make the line look like the human's drawing. With GRITS, the "Spillage Crystal Ball" acts as a strict editor. As the robot is refining its movement plan, the editor whispers, "Hey, that part of the path looks risky! If you go there, you'll spill the rice. Let's nudge the path just a tiny bit to the left."
The Result: The robot doesn't just copy the human; it adapts the human's path in real-time to avoid the mess, making tiny, subtle adjustments (less than the width of a fingernail) to keep the food safe.

4. The Real-World Test

The team tested this on a real robot arm in a real lab.

The Challenge: They tried it on 10 different types of food the robot had never seen before (like milk tea, nuts, and weirdly shaped candies).
The Score:
- Old Robots: Spilled food about 40% of the time.
- GRITS: Only spilled food 4% of the time.
- Success Rate: It successfully scooped the food 82% of the time.

Why This Matters

Think of GRITS as the difference between a robot that is a "copycat" and a robot that is a "thinker."

The Copycat sees a bowl of food and says, "I saw a human do this once, so I will do exactly that." (Result: Spill).
The Thinker (GRITS) sees the bowl, checks the food's texture, runs a quick mental simulation, and says, "Okay, that path looks dangerous. I'll adjust my angle slightly to be safe."

This technology is a huge step toward robots that can work in real kitchens, hospitals (helping feed patients), and restaurants without making a huge mess, handling everything from slippery noodles to sticky marshmallows with the delicacy of a human chef.

1. Problem Statement

Robotic food scooping is a critical skill for food preparation and assistive feeding. However, existing learning-from-demonstration (LfD) methods struggle with the dynamic and deformable nature of food.

Core Challenge: Standard policies often fail to account for unseen food states (e.g., a full bowl, varying particle sizes, or friction), leading to spillage.
Limitations of Current Approaches:
- Collecting diverse real-world data to cover all food types and spillage scenarios is costly and labor-intensive.
- Standard Diffusion Policies (DP) generalize well but lack explicit safety constraints, often generating trajectories that result in unintended food loss.
- Post-hoc adjustments (modifying trajectories after generation) often compromise task success (e.g., failing to scoop at all to avoid spillage).

The goal is to develop a policy that ensures high task success rates while minimizing spillage in real-world, unseen scenarios without requiring retraining for every new food type.

2. Methodology: GRITS Framework

The authors propose GRITS (A Spillage-Aware Guided Diffusion Policy), which integrates a spillage predictor as a differentiable guidance signal into the diffusion sampling process.

A. Core Architecture

GRITS decouples training and inference:

Training Phase: A standard Diffusion Policy is trained on expert demonstrations to learn general scooping behaviors (imitation learning) without explicit spillage constraints.
Inference Phase (Guided Sampling): A Spillage Predictor estimates the probability of spillage for a candidate trajectory. This predictor acts as a guidance signal ( $J$ ) that steers the denoising process toward safer trajectories during test time.

B. Spillage Predictor & Simulation

Training a spillage predictor in the real world is impractical due to cleanup costs. The authors utilize Isaac Lab (high-fidelity simulator) to generate a dataset:

Dataset Construction: 4,000 trials involving four primitive shapes (spheres, cubes, cones, cylinders) with randomized physical properties (mass, friction, particle size).
Input Representation: To mitigate the sim-to-real gap, the predictor uses segmented point clouds rather than raw images.
- Food: Reconstructed from depth data.
- Spoon/Bowl: Represented by known CAD models aligned via kinematics.
- Segmentation: Uses SAM2 to isolate relevant pixels.
Network Design:
- Point Cloud Encoder: Uses DP3 (a PointNet++ variant optimized for fixed cameras) to encode the composite point cloud.
- Trajectory Encoder: A fully connected MLP encodes the candidate action rollout.
- Output: A binary classification (spillage vs. non-spillage) via softmax.

C. Guided Diffusion Process

During inference, the diffusion model generates a trajectory $A_t$ . The guidance mechanism modifies the denoising steps:

The model estimates the "clean" trajectory $A_{0|k}$ at step $k$ .
The spillage predictor $f_{spillage}$ evaluates $A_{0|k}$ to compute a risk score.
The gradient of this risk score ( $\nabla J$ ) is computed and used to update the trajectory:
$A_{k-1} = \mu_k - \rho \nabla J + \sigma_k z$
Where $\rho$ is the guidance weight.
Thresholding: Guidance is delayed until step $s=30$ (out of 100) to allow the policy to form a coarse trajectory before refining it, preventing over-correction in early stages.

3. Key Contributions

GRITS Framework: The first application of guided diffusion policies specifically for robotic food scooping, introducing a spillage-aware objective that refines trajectories in real-time.
Sim-to-Real Data Pipeline: A novel pipeline using Isaac Lab to synthesize diverse food shapes and physical properties, training a spillage predictor that generalizes to real-world scenarios.
Composite Point Cloud Representation: A method to reduce the sim-to-real gap by combining depth-reconstructed food with known CAD models of the spoon and bowl, significantly improving predictor accuracy.
Real-World Validation: Extensive experiments on a physical robot platform demonstrating effectiveness across diverse, unseen food categories.

4. Experimental Results

The system was evaluated on a Franka Emika Panda robot with a spoon attachment.

Training Data: 6 food categories (brown rice, soybeans, etc.).
Test Data: 10 unseen categories (varying shapes, sizes, and materials like liquids and mixtures).

Performance Metrics:

Task Success Rate: GRITS achieved 82%, outperforming all baselines.
Spillage Rate: GRITS achieved only 4%, a >40% reduction compared to standard Diffusion Policy baselines.
Comparison with Baselines:
- Rule-based: Failed to adapt to different foods.
- Behavior Cloning / Vanilla DP: High success but high spillage rates.
- SCONE (Active Perception): Reduced spillage but had high failure rates on small quantities.
- Post-hoc Guidance: Reduced spillage (8%) but caused a massive drop in success (40% scoop failure) because it adjusted trajectories after generation without considering task feasibility.

Trade-off: GRITS showed a slightly higher "scoop failure rate" (14%) than vanilla DP, indicating that the safety guidance occasionally prevents the robot from attempting risky but potentially successful scoops. However, the overall reliability (Success - Spillage) is superior.

5. Significance and Impact

Safety in Robotics: This work addresses a critical safety and reliability gap in service robotics. Spillage is not just a failure of task completion but a hygiene and efficiency issue in food service.
Efficiency: By using simulation for the "hard" part (learning spillage dynamics) and real-world data for the "easy" part (general motion), GRITS avoids the prohibitive cost of collecting thousands of real-world spillage examples.
Generalization: The ability to handle 10 unseen categories with different physical properties (e.g., sticky fudge vs. loose nuts) demonstrates strong generalization capabilities of the guided diffusion approach.
Future Direction: The paper suggests extending this framework to other manipulation tasks (cutting, pouring) and incorporating multimodal sensing (force/torque) to further reduce the sim-to-real gap regarding physical properties.

In summary, GRITS successfully bridges the gap between high-level imitation learning and low-level safety constraints, providing a robust solution for delicate robotic manipulation tasks where precision and spill prevention are paramount.