Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans

This paper presents an autonomous robotic assembly framework that utilizes reinforcement learning with successor features to construct stable block structures without predefined plans, demonstrating adaptability to environmental uncertainties and construction noise through real-world experiments.

The Gist

Imagine you are teaching a robot to build a house, but instead of giving it a detailed architectural blueprint with every brick's exact location, you simply tell it: "Build something that reaches this specific window, but don't touch that tree in the middle."

That is the core idea behind this research paper. The authors have created a robot that can figure out how to build stable structures on its own, without needing a pre-written plan.

Here is a breakdown of how they did it, using some everyday analogies:

1. The Problem: The "Rigid Blueprint" Trap

Traditionally, construction robots are like musicians reading sheet music. They can only play the notes exactly as written. If the floor is uneven, or a brick is slightly crooked, the robot gets confused and stops because the "music" doesn't match reality.

In the real world, construction sites are messy. The ground isn't perfectly flat, and materials vary. The authors wanted a robot that acts more like a jazz improviser. It knows the goal (the melody) but can change its notes (the building steps) on the fly to handle whatever curveballs the environment throws at it.

2. The Solution: The "Goal-Oriented Gardener"

Instead of a blueprint, the robot is given a Target (where the structure needs to end up) and Obstacles (what it must avoid).

Think of the robot as a gardener trying to grow a vine.

• The Goal: The vine must touch a specific trellis (the target).
• The Obstacle: The vine must not grow through a fence (the obstacle).
• The Robot's Job: It doesn't know the final shape of the vine. It just knows it needs to reach the trellis. It tries placing a block (a leaf), checks if it's stable, and then decides where to put the next one.

3. The Brain: The "Crystal Ball" (Reinforcement Learning)

How does the robot decide where to put the next block? It uses a type of Artificial Intelligence called Reinforcement Learning (RL).

Imagine the robot is playing a video game where it gets points for getting closer to the target and loses points if it uses too many blocks or hits an obstacle.

• The "Crystal Ball" Trick: The researchers gave the robot a special superpower called Successor Features.
  • Normally, a robot just looks at the now.
  • This robot looks at the future. It can visualize a "ghost image" of what the structure will look like if it takes a certain action.
  • It's like a chess player who doesn't just think, "If I move here, I capture a pawn," but instead thinks, "If I move here, I control the center of the board in 10 moves."
  • This allows the robot to plan long-term strategies, like building a bridge or an arch, even though it's only placing one block at a time.

4. The Real-World Test: The "Clumsy Hand"

The researchers didn't just test this in a computer simulation; they built a real robot arm with a suction cup gripper to stack 3D-printed blocks.

• The Challenge: In the real world, robots aren't perfect. Sometimes the suction cup slips, or the block lands slightly crooked. This is like trying to stack Jenga blocks while your hands are shaking slightly.
• The Closed-Loop: The robot has a camera (its eyes) that constantly checks the structure after every block is placed. If the block is slightly off-center, the robot sees it, updates its mental map, and adjusts the next block to compensate. It's like a tightrope walker constantly shifting their weight to stay balanced.

5. The Results: 80% Success Rate

The robot was tested on 15 different challenges (like building a bridge over a gap or a tall tower).

• In the computer: It solved almost all of them.
• In the real world: It succeeded in 80% of the tasks.
• The Failures: When it failed, it was usually because the robot's arm physically couldn't reach a spot (like a human trying to reach behind their back) or because the structure was so delicate that a tiny wobble made it collapse.

Why This Matters

This research is a big step toward autonomous construction.

• Current Robots: Need a perfect plan and a perfect environment.
• This Robot: Can handle uncertainty, adapt to mistakes, and invent its own building strategies.

The Big Picture:
Imagine a future where, after a natural disaster, a fleet of these robots arrives at a ruined city. Instead of waiting for engineers to draw up blueprints for every single shelter, you just tell them: "Build a roof over this area, avoiding the debris." The robots would then figure out the best way to stack whatever materials are available to create a stable shelter, adapting to the chaos of the real world in real-time.

This paper proves that robots can move from being rigid "plan-followers" to becoming flexible, creative "problem-solvers."

Technical Summary

1. Problem Statement

The paper addresses the limitations of current robotic construction systems, which rely heavily on rigid, pre-defined architectural blueprints. In dynamic construction environments (uneven terrain, material variability, human error), these fixed plans often fail because they lack adaptability.

The authors propose a framework for autonomous robotic assembly where the robot constructs stable structures without a pre-computed sequence or blueprint. Instead, the task is defined abstractly by:

• Targets: 2D points the structure must reach.
• Obstacles: Regions the structure must avoid.
• Blocks: A set of discrete, rigid units (square and trapezoidal) with dry joints (no adhesive).

Key Challenges:

• Dynamic Action Space: The set of valid block placements changes combinatorially as the structure grows.
• Stability Constraints: Since dry joints offer no inherent reinforcement, every intermediate step must be structurally stable (verified via Rigid-Block Equilibrium).
• Generalization: A single policy must solve diverse structural topologies (columns, arches, bridges) defined by different target/obstacle configurations.

2. Methodology

The authors develop a Goal-Conditioned Reinforcement Learning (RL) framework using Deep Q-Learning (DQN) with Successor Features.

A. Task Formulation

• State ($S$): The current assembly of blocks.
• Action ($A$): The placement (position and orientation) of a new block. The action space is state-dependent ($A(S)$), containing only placements that result in a stable structure and avoid collisions.
• Task ($T$): Defined by target coordinates and obstacle regions.
• Goal: Construct an assembly connecting the ground to all targets while avoiding obstacles, using the minimum number of blocks.

B. Representation Learning

To handle the complex geometry and generalization requirements, the system uses image-based feature representations:

• State Features ($\psi(S)$): A binary image representing the current assembly, formed by summing the features of placed blocks.
• Action Features ($\phi(A)$): A binary image indicating the shape and location of a potential block placement.
• Task Features ($\xi(T)$): A multi-channel image where channels encode obstacle locations and target locations.
• Reward Feature ($\rho(T)$): A smooth scalar field generated by convolving target locations with a Gaussian kernel, providing dense rewards to guide growth toward targets while penalizing block volume.

C. Algorithm: DQN with Successor Features

The core innovation is the use of Successor Features ($\Psi$) to decompose the reward function.

• Decomposition: The Q-value is expressed as $Q^\pi(S, A, T) = \Psi^\pi(S, A, T)^\top \rho(T)$.
  • $\Psi^\pi$ captures the expected future state-action features (long-term intent) independent of the specific task reward.
  • $\rho(T)$ encodes the specific task goals.
• Benefits: This allows a single policy to generalize across different tasks (different targets/obstacles) without retraining, as the policy learns the "how" (successor features) while the task definition provides the "what" (reward).
• Architecture: A U-Net is used to predict the successor features from the input state, action, and task images.

D. Closed-Loop Robotic System

To validate the method in the real world, the authors implemented a closed-loop workflow:

1. Perception: A Zivid 3D camera scans the structure. ArUco markers on blocks allow for 6D pose estimation.
2. State Update: The physical state is reconstructed in the simulation environment to account for placement errors and noise.
3. Decision: The trained policy selects the next action based on the updated state.
4. Execution: An ABB CRB 15000 robot with a custom L-shaped suction gripper places the block.

3. Key Contributions

1. Blueprint-Free Framework: A novel system that generates stable structures from abstract geometric goals (targets/obstacles) rather than fixed plans.
2. Successor Feature RL: Application of successor features to robotic assembly, enabling multi-task generalization and interpretability (visualizing the policy's long-term construction intent).
3. Polygonal Block Handling: The system accommodates non-rectangular (trapezoidal) blocks, allowing for more diverse structural strategies compared to grid-aligned approaches.
4. Real-World Validation: Successful deployment on a physical robot with a closed-loop feedback system, demonstrating robustness against construction noise and physical uncertainties.

4. Experimental Results

The system was evaluated on 15 distinct 2D assembly tasks (varying targets and obstacles).

• Simulation Performance:
  • Trained in just 50 episodes.
  • Achieved a 93.3% success rate (14/15 tasks solved).
  • The policy learned to use fewer blocks over time and discovered complex solutions like arches and counterweights.
• Real-World Performance:
  • Deployed on a physical robot with closed-loop feedback.
  • Achieved an 80% success rate (12/15 tasks).
  • Adaptability: In successful cases (e.g., Tasks 3 and 12), the physical structures deviated from the simulation due to noise, yet the policy successfully adapted its subsequent steps to still reach the targets.
• Failure Analysis:
  • Failures were attributed to: (1) Accumulated errors in long-horizon tasks exceeding the policy's correction capability; (2) Marginal stability cases that passed simulation but collapsed under real-world perturbations; (3) Hardware constraints (gripper collisions).

5. Significance and Future Directions

This work represents a significant step toward adaptive autonomous construction. By moving away from rigid plans, the system can handle the inherent uncertainties of real-world building sites.

• Interpretability: The visual nature of successor features allows engineers to "see" the robot's future construction plan, increasing trust in autonomous systems.
• Scalability: While currently limited to 2D and two block types, the framework is designed to scale to 3D, diverse materials (including irregular/recycled materials), and multi-robot collaboration.
• Applications: The approach is particularly relevant for rapid post-disaster reconstruction, space exploration (using in-situ materials), and construction in unstructured environments where pre-planning is impossible.

The paper concludes that while a "Sim-to-Real" gap remains (particularly regarding stability solvers and hardware constraints), the proposed framework offers a promising path for robust, flexible, and intelligent robotic construction.