🤖 machine learning

Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)

This paper presents a prizewinning bimanual garment folding system for the LeHome Challenge 2026 that achieves top rankings by enhancing a vision-language-action policy with a self-supervised reinforcement learning loop, where the policy's own predictions of success and progress drive advantage estimation and failure detection, supported by a comprehensive suite of engineering optimizations including flow-matching, distributed training, and robust sim-to-real transfer.

Original authors: Ilia Larchenko

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ilia Larchenko

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot trying to fold a messy t-shirt or a pair of pants. It sounds simple to us, but for a robot, a piece of cloth is like a chaotic, slippery snake that changes shape every time you touch it. This paper describes how the author, Ilia Larchenko, built a robot brain that won first place in a simulation contest and second place in a real-world contest for this exact task.

Here is the story of how he did it, broken down into simple concepts.

1. The Problem: The "Ghost" in the Machine

The robot had to fold four types of clothes (long/short sleeves, long/short pants) using two arms.

The Catch: The robot didn't know what it was holding. It had to look at the shirt and figure out, "Is this a t-shirt or a pair of pants?" before it could start folding.
The Reward: The robot only got a "Gold Star" (a reward) if the shirt was perfectly folded at the very end. If it messed up halfway, it got nothing. This is like trying to learn to ride a bike but only getting a cookie if you reach the finish line without ever falling.

2. The Solution: A "Self-Teaching" Loop

Instead of just copying human demonstrations (which is like trying to learn to swim by watching a video), the author built a system that learns by doing, failing, and trying again. He calls this a "Flywheel."

Think of it like a three-person team working in a factory:

The Trainer: The teacher who studies the data and updates the robot's brain.
The Runners: A fleet of robots in a video game (simulation) that try to fold clothes thousands of times a day.
The Human Helper: A person who steps in only when the robot gets stuck, fixes the shirt, and lets the robot try again.

These three don't talk to each other directly; they just drop notes in a shared digital mailbox (HuggingFace Hub). The Trainer reads the notes, learns, and drops a new "brain update" back in the box. The Runners pick it up and try again.

3. The Secret Sauce: The Robot is its Own Scorekeeper

Usually, in robotics, you need one brain to decide what to do and a separate brain to guess how well it's doing. This author merged them.

The Magic Trick: The same neural network that decides "grab the sleeve" also predicts "I'm 80% likely to succeed" and "I'm 40% of the way through."
Why it helps: Because the robot knows its own score in real-time, it can learn faster. If it predicts it's going to fail, it knows to try a different move. It's like a student who not only takes the test but also grades their own answers immediately to see where they went wrong.

4. Learning from Mistakes (The "Replay" Strategy)

The author realized that just practicing is boring. You need to practice the hard parts.

Hard Mining: When the robot fails, the system saves that exact moment (like a "Save State" in a video game). It then loads that failed state and tries again, but this time with extra "noise" (random changes in lighting or cloth texture) to make it harder.
The Human Touch: When the robot gets truly stuck, the human takes over for just a few seconds to fix the shirt, then hands control back. The robot learns from this tiny correction. This is called DAgger (a fancy name for "learning from human fixes").

5. The "Magic Glasses" (Inference Optimization)

Even with a smart brain, the robot can be clumsy if it moves too fast or too slow. The author found that by tweaking how the robot "thinks" at the very last second, performance skyrocketed.

The Analogy: Imagine you are taking a multiple-choice test. Instead of just picking the first answer that pops into your head, you quickly generate three different answers in your mind, check which one feels "best" based on your gut feeling (the score), and then pick that one.
Thompson Sampling: The author used a mathematical game (like a slot machine) to automatically figure out the perfect settings for every type of shirt (e.g., "For shorts, move slower; for long pants, grab tighter") without needing to manually test every single combination.

6. The Real-World Challenge: From Video Game to Reality

The hardest part was moving from the computer simulation to a real robot in a real room.

The "Uncanny Valley" of Physics: In the simulation, the clothes looked perfect. In reality, the lighting was different, the camera was slightly crooked, and the robot's arms had a little bit of "slop" (backlash).
The Fix: The author didn't try to make the robot perfect. Instead, he made the training data chaotic. He taught the robot to fold clothes in the dark, in the bright sun, with the camera tilted, and with the robot moving at different speeds.
The Result: By training the robot to be comfortable with chaos, it didn't panic when it saw the real world. It was like training a surfer in a stormy ocean so that a calm beach feels easy.

7. The Outcome

Simulation: The robot folded clothes successfully 79.6% of the time, beating 61 other teams.
Real World: It placed 2nd in the final competition, successfully folding real clothes on a real robot.

The Big Lesson

The author admits this wasn't a perfect scientific experiment with controlled variables. It was a "kitchen sink" approach—throwing every useful tool (reinforcement learning, human feedback, heavy data augmentation, and smart guessing) into the pot until it worked.

The main takeaway is that for robots to handle messy, unpredictable things like clothes, they need to be able to predict their own success, learn from human fixes, and be trained in chaotic, varied environments so they don't get confused when reality doesn't look like the video game.

Technical Summary: Learning to Fold at the LeHome Challenge 2026

Problem Definition

The paper addresses the LeHome Challenge 2026, a competition focused on bimanual robotic manipulation of deformable objects (garments). The task requires a robot with two 6-DOF arms (SO-ARM101 setup) and three RGB cameras to fold four types of garments (long/short-sleeved tops, long/short pants) lying flat on a table.

The challenge is divided into two phases:

Online Round (Simulation): Teams compete in NVIDIA Isaac Sim. Success is binary and geometric, defined by specific keypoint-distance conditions (e.g., sleeves meeting, hem alignment). The dataset includes 10 "seen" garments with scripted demonstrations and 10 "unseen" garments (2 public, 8 private) with no training data.
Real-World Final: Top teams deploy on physical robots. Success criteria include partial credit for intermediate steps, judged by a jury. A critical constraint is that the garment type is not provided as an input; the policy must infer it.

Key challenges identified include the deformable nature of cloth (sensitive to trajectory), sparse binary rewards, generalization to unseen garments without training data, and a difficult sim-to-real transfer where the evaluation robot was never accessible during development.

Methodology

The solution is an engineering case study combining a Vision-Language-Action (VLA) policy with a Reinforcement Learning (RL) loop, rather than a controlled ablation study.

1. Policy Architecture

The system builds upon the $\pi_0.5$ architecture (extended in the BEHAVIOR-1K Challenge 2025), which uses a frozen SigLIP image encoder, a Gemma-2B prefix transformer, and a Gemma-300M action expert that generates 30-step action chunks via flow matching.

Modifications for LeHome:
- Garment-Type Token: A learned embedding replaces text prompts. During inference, a "System-2" bootstrap predicts the garment type at the start of the episode to condition the rest of the rollout.
- Advantage Conditioning: The policy is conditioned on a learned advantage token (RECAP-style), enabling Classifier-Free Guidance (CFG) at inference.
- Multi-Signal AdaRMS: Advantage and garment type are injected into the action expert's layer norms via a learned vector, strengthening conditioning.
- Exclusive Self-Attention (XSA): Applied to prevent tokens from trivially routing their own values, improving training stability.
- Smooth Normalization: Action deltas are normalized per-timestep to handle the growing variance of the random-walk-like trajectory.

2. The Policy as Its Own Value Function

Instead of a separate critic, the VLA model includes auxiliary linear heads that predict:

Success Probability: Acts as the value function ( $V$ ).
Completion Progress: Predicts $t/T$ for potential-based shaping.
Keypoint Distances: Predicts the specific ratios used in the success checker.
Future Predictions: A "Q-function" head predicts the success residual ( $\Delta_{success}$ ) 30 frames ahead, conditioned on the proposed action chunk. This enables Best-of-N selection at inference.

3. RL Training Loop (The Flywheel)

The system uses an asynchronous pipeline communicating via HuggingFace Hub:

Algorithms: Combines AWR (Advantage Weighted Regression) and RECAP (advantage conditioning).
- AWR: Frames with high advantage are sampled more frequently ( $P \propto e^{A/\beta}$ ) rather than re-weighted in the loss, ensuring 100% batch utilization on high-value data.
- RECAP: The advantage is fed as an input token, allowing the model to learn to "predict good actions only" and enabling CFG.
Data Strategy:
- Multi-dataset Sampling: The loader mixes Behavior Cloning (BC) data, RL rollouts, and DAgger (Human-in-the-loop) data.
- Decay: RL data decays over time to prioritize on-policy data.
- Recovery: A DAgger loop allows humans to fix failure states in simulation (though this proved difficult in sim) and on the real robot.
- Rollback: The training process periodically rolls back to older checkpoints to escape local optima using fresh off-policy data.

4. Reward and Advantage Design

Since the true reward is binary, the system engineers dense intermediate rewards:

Checkpoints: Intermediate rewards (0.5) are awarded for reaching specific keypoint-distance milestones.
Failure Withdrawal: If an episode fails, all accumulated reward is withdrawn to maintain a binary return, preventing the model from learning to "game" intermediate states.
Advantage Estimation: Uses Generalized Advantage Estimation (GAE) combining the dampened success probability and completion progress. For stale rollouts, the advantage degrades gracefully to a GRPO-style relative-success signal.
Precision Boost: Successful episodes that satisfy conditions "tightly" receive a bonus advantage to encourage high-quality folds.

5. Inference-Time Optimization

The same checkpoint is optimized at deployment without retraining:

Thompson Sampling: A multi-armed bandit tunes hyperparameters (execution length, guidance scale, noise temperature, candidate count) per garment type during rollout collection.
Best-of-N: Multiple action chunks are sampled from the same prefix; the one with the highest predicted $\Delta_{success}$ is executed.
Soft Inpainting: Overlapping action chunks are anchored to ensure smooth trajectories and mode stickiness.

6. Sim-to-Real Transfer

The real-world transfer was a "sprint" involving:

Model Pruning: Removed sim-only heads (keypoint distances, world model) and advantage conditioning, keeping only the action expert, garment-type head, and completion head.
Data Mixing: Fine-tuned on a mix of organizer BC data, personal teleop/DAgger data, and heavily augmented sim replays.
Alignment & Augmentation: Used a camera-overlay tool to align the physical rig with the target. Applied extreme augmentation (color jitter, noise, camera dropout, motion intensity rescaling) to bridge the domain gap.
DAgger on Real Robot: Collected recovery data by having a human intervene when the policy failed, weighting frames based on proximity to intervention.

Results

Online Round (Simulation): Achieved 1st place out of 62 teams with an overall success rate of 79.63%, beating the second-place team by 6.1 percentage points.
- Performance by type: Long Tops (74.5%), Short Tops (70.0%), Long Pants (80.5%), Short Pants (93.5%).
Real-World Final: Achieved 2nd place. The score was 865 out of a maximum of 1080 points.
- The policy successfully folded unseen garments and handled the physical robot's dynamics, though it relied heavily on DAgger data for recovery rather than autonomous exploration.

Significance and Claims

The author frames this work as an engineering case study rather than a controlled scientific experiment, noting that the system was built iteratively under competition pressure with limited ablation studies.

Practical Recipe: The paper claims the solution is a "recipe" of recombined existing ideas (AWR, RECAP, Flow Matching, DAgger) that can be used together or individually.
Policy as Value Function: It demonstrates that a single VLA model can effectively serve as both the policy and the value function (success/completion predictors), simplifying the training pipeline and enabling efficient Best-of-N selection.
Inference-Time Tuning: The work highlights that significant performance gains can be achieved by optimizing inference hyperparameters (via Thompson sampling) without additional training.
Sim-to-Real via Diversity: It argues that heavy augmentation and deliberate rig randomization, combined with a "sim-to-real-to-real" transfer strategy, can overcome significant domain gaps even without access to the evaluation robot.
Limitations: The author admits the system lacks robust autonomous exploration/recovery. The policy excels at refining known behaviors but struggles to recover from states completely outside its training distribution without human intervention (DAgger). The author suggests that a unified pipeline combining the RL machinery of the sim round with the real-world recovery data of the final round could potentially achieve >90% success.