💻 computer science

FlatLab: A Unified Methodology Framework and Simulation-Based Benchmark for Robotic Manipulation of Flat Objects

This paper introduces FlatLab, a unified framework and high-fidelity simulation benchmark that enables robust robotic manipulation of diverse flat objects by decoupling strategy generation from action execution, demonstrating superior generalization to unseen objects compared to existing methods.

Original authors: Xingyu Zhu, Wenshuo Han, Zhouyu Wang, Yuran Wang, Ruihai Wu, Hao Dong, Fan Tang, Hechang Chen, Hyung Jin Chang, Yixing Gao

Published 2026-08-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xingyu Zhu, Wenshuo Han, Zhouyu Wang, Yuran Wang, Ruihai Wu, Hao Dong, Fan Tang, Hechang Chen, Hyung Jin Chang, Yixing Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where robots are like clumsy toddlers trying to learn how to pick up a piece of toast. They can grab a mug or a ball easily because those things have handles or round shapes that fit perfectly in a hand. But try to grab a flat magazine lying flat on a table, a thin wooden board, or a floppy towel, and the robot gets stuck. It's like trying to pick up a pancake with a fork; there's no handle to hold onto, and if you push too hard, it just slides away. This is the "flat object problem" in robotics. Scientists have been trying to solve this for years, mostly by building special, expensive robot hands or by teaching robots one specific trick, like "push the book to the edge of the table." But real life is messy. A thick book needs a different trick than a thin piece of paper, and a floppy towel needs a completely different approach. If a robot only knows one trick, it fails when the object changes. This is why researchers are looking for a smarter way to teach robots to adapt, rather than just memorizing a single move.

Enter FlatLab, a new project that acts like a giant, high-tech video game for robots, designed specifically to solve this flat-object puzzle. The researchers behind FlatLab realized that instead of forcing a robot to learn one giant, complicated move for every possible object, it's better to give the robot a "toolbox" of strategies and a brain that knows which tool to pick. They created a unified framework that splits the job into two parts: a Strategy Generator and an Action Execution Module. Think of the Strategy Generator as a smart manager who looks at a pile of flat objects (like a stack of magazines, a box, or a crumpled shirt) and decides, "Okay, for this thin disk, we'll push it to the edge. For this thick box, we'll use two arms to lift it. For this floppy towel, we'll squeeze the edges to make a fold." Once the manager picks the plan, the Action Execution Module is the worker bee that breaks that big plan down into tiny, simple steps—like "touch," "slide," or "squeeze"—and executes them smoothly.

The team didn't just build the robot brain; they built the entire playground to test it. They created FlatLab, a simulation platform with over 100 different flat objects, ranging from rigid things like books and boxes to soft, squishy things like fabrics. In this digital world, they can test how well their robot handles these objects without breaking anything in the real world. They trained their system using a clever technique called "contrastive learning," which is like teaching the robot to recognize that a thin book and a thin plate are similar enough to use the same "push to the edge" trick, even if they look different. They also used "simulated data transformation," which means they took a digital object and instantly changed its size or material (turning a soft towel into a hard board) to teach the robot that the strategy depends on the shape and material, not just the specific object's name.

When they tested their system, the results were impressive. In their simulations, the robot correctly guessed the right strategy 99.2% of the time on objects it had seen before, and still got it right 78.6% of the time on completely new categories of objects it had never encountered. When it came to actually grabbing the objects, their method succeeded 81.1% of the time on known objects, and managed a 69.0% success rate on totally new, unseen objects. This is a big deal because other methods, like those that try to learn one big "end-to-end" move or just push everything to the edge, struggled significantly, often failing on new types of objects. For example, a method that just pushes things to the edge only succeeded about 24.7% of the time on new objects, while a dual-arm lifting method failed even more often on thin items.

The researchers also took their robot out of the video game and into the real world, using a Baxter robot with two arms. Even with the messy, unpredictable physics of the real world—where friction is different and towels don't always fold exactly like they do in a computer—they achieved an 80.0% success rate on new objects. This suggests that their "toolbox" approach is robust enough to handle the real thing, not just the perfect digital version. However, the authors are careful to note that while this works well in their tests, there is still work to be done. They admit that most of their heavy lifting was done in simulation, and bridging the gap between the perfect digital world and the messy real world is a key challenge for the future. They also point out that they haven't fully tested how their robot handles incredibly complex, cluttered rooms yet. But for now, FlatLab shows that giving robots a flexible strategy and a set of simple tools is a much better way to teach them how to handle the flat, floppy, and tricky objects that surround us every day.

Technical Summary: FlatLab

Problem Statement
Robotic manipulation of flat objects (e.g., magazines, wooden boards, fabrics) presents significant challenges due to the lack of readily accessible grasp affordances when objects lie flush on surfaces and the strong variations in object geometry and material properties. Existing approaches often rely on heuristic pre-manipulation strategies (such as pushing objects to table edges or using specialized end-effectors) and are typically evaluated in closed settings with limited generalization capabilities. Furthermore, the field lacks a unified simulation benchmark that systematically covers both rigid and deformable flat objects, hindering the development of robust manipulation algorithms.

Methodology
The authors propose a unified methodology framework that decouples manipulation into two distinct modules: a Manipulation Strategy Generator and a Robot Action Execution Module.

Manipulation Strategy Generator:
- Input: Object point clouds.
- Output: A predicted manipulation strategy selected from three complementary strategies:
  - Strategy-A: Pushing the object to the table edge (suitable for thin, small rigid objects).
  - Strategy-B: Cooperative dual-arm lifting (suitable for thick, large objects).
  - Strategy-C: Squeezing edges to create graspable folds (suitable for deformable objects).
- Learning Mechanism: The generator learns strategy-centric, object-invariant representations. It utilizes simulated data transformation (augmenting data via material and scale changes) and multi-perspective contrastive learning (NT-Xent loss). This approach ensures the model aggregates features for the same strategy across different objects while decoupling features for different strategies, preventing overfitting to specific object geometries.
Robot Action Execution Module:
- Input: The predicted strategy and the scene point cloud.
- Output: 6-DoF end-effector poses.
- Mechanism: Instead of learning a single long-horizon trajectory, this module decomposes manipulation into reusable action primitives (e.g., touching, sliding, squeezing). It learns position-dependent poses rather than object-specific features.
- Training: The module is trained using a geometrically consistent pose loss comprising a translation term and a geodesic rotation loss (using quaternions) to ensure smooth, physically meaningful trajectories.

FlatLab Benchmark
To enable systematic evaluation, the authors introduce FlatLab, a comprehensive simulation-based benchmark built on NVIDIA Isaac Sim.

Assets: Includes over 100 high-fidelity flat objects spanning 21 categories (rigid and deformable) with configurable physical properties (mass, friction, texture).
Scenarios: Features 18 manipulation scenarios across 6 environments (e.g., Kitchen, Bedroom) with varied desktop backgrounds.
Capabilities: Supports automated multi-modal data collection (point clouds, RGB, depth, kinematic data) and standardized task definitions (e.g., sliding, grasping, dual-arm lifting).
Evaluation: Provides unified protocols and metrics, including success rate, trajectory smoothness, and primitive action counts.

Key Contributions

Unified Framework: A decoupled framework that separates strategy prediction from action execution, enabling robust generalization across unseen objects and categories by adapting strategies to object shape and material.
FlatLab Platform: The first unified simulation benchmark for robotic flat object manipulation, offering high-fidelity physics, automated data collection, and standardized evaluation for both rigid and deformable objects.
Empirical Validation: Extensive experiments demonstrating the framework's effectiveness and generalization capabilities.

Experimental Results
Experiments were conducted on a dataset of 104 flat objects, partitioned into a training set, a test set with unseen objects from known categories (Test $\alpha$ ), and a test set with unseen categories (Test $\beta$ ).

Strategy Discrimination: The framework achieved strategy prediction accuracies of 99.2% (Train), 91.3% (Test $\alpha$ ), and 78.6% (Test $\beta$ ).
Grasping Success Rates: The proposed method achieved success rates of 81.1% (Train), 74.2% (Test $\alpha$ ), and 69.0% (Test $\beta$ ).
Baseline Comparison: The method significantly outperformed baselines, including single-strategy approaches (Slide, Lift) and end-to-end vision-language-action models (Diffusion Policy, OpenVLA, $\pi_0$ , $\pi_0.5$ ). For instance, on Test $\beta$ , the proposed method (69.0%) vastly outperformed OpenVLA (17.8%) and $\pi_0$ (17.3%).
Real-World Transfer: When deployed on a Baxter robot with real-world objects, the method maintained high performance, achieving success rates of 83.6% (Train), 80.0% (Test $\alpha$ ), and 80.0% (Test $\beta$ ).

Significance and Claims
The paper claims that the proposed approach addresses the limitations of single-strategy pipelines and closed datasets by introducing a flexible, strategy-adaptive framework. By decoupling strategy selection from action execution and leveraging contrastive learning on simulated data, the system achieves superior generalization to unseen objects and categories. The introduction of FlatLab fills a critical gap in the field by providing a standardized, high-fidelity platform for evaluating robotic manipulation of flat objects, facilitating reproducible research and the development of more robust embodied intelligence systems. The authors note that while the method performs well in simulation and real-world tests, bridging the sim-to-real gap and studying complex scene interactions remain directions for future work.

Technical Summary: FlatLab

More like this