Co-Layout: LLM-driven Co-optimization for Interior Layout

This paper presents Co-Layout, a novel framework that integrates large language models with grid-based integer programming and a coarse-to-fine optimization strategy to jointly optimize room layouts and furniture placement, significantly outperforming existing two-stage pipelines in both solution quality and computational efficiency.

The Gist

Imagine you are hiring an architect to design your dream home. Usually, you'd give them a rough idea ("I need a big kitchen and a cozy bedroom"), and they would spend weeks drawing, erasing, and rearranging walls and furniture until everything fits perfectly.

This paper, Co-Layout, introduces a new "super-architect" that does this job in seconds. It combines the imagination of a creative writer (an AI called an LLM) with the mathematical precision of a puzzle solver (Integer Programming).

Here is how it works, broken down into simple concepts:

1. The Problem: The "Two-Step" Mistake

Most current AI tools for interior design work in two separate steps, like a clumsy assembly line:

1. Step A: They draw the rooms (where the walls go).
2. Step B: They try to stuff furniture into those rooms.

The Analogy: Imagine trying to build a house by first building the rooms, and then realizing the sofa is too big for the living room, or the bed blocks the door to the bathroom. You have to tear down the walls and start over. This leads to messy, illogical designs where you can't walk from the kitchen to the bedroom without walking through a closet.

2. The Solution: The "Co-Optimization" Dance

The authors' new method, Co-Layout, does both steps at the same time. It treats the room layout and the furniture placement as one giant, interconnected puzzle.

• The Creative Brain (The LLM): First, you type a simple prompt like, "Design a 100-square-meter apartment for a family of three." The AI (LLM) acts like a creative director. It doesn't draw the floor plan; instead, it writes a detailed "rulebook."
  • Rulebook Example: "The master bedroom must be in the corner. The kitchen needs to be next to the dining room. The bed needs to be 1.8 meters wide. The hallway must connect everything."
• The Math Brain (The Grid Solver): This is where the magic happens. The system takes that rulebook and turns the entire floor into a giant digital grid (like a chessboard or a pixelated map).
  • Every square on the grid is a tiny piece of the puzzle.
  • The computer uses math to figure out exactly which squares become walls, which become the living room, and which squares hold the sofa, ensuring no overlaps and no blocked paths.

3. The Secret Sauce: "Coarse-to-Fine" Strategy

Solving this puzzle for a whole house is incredibly hard for a computer. It's like trying to solve a 10,000-piece jigsaw puzzle all at once. It would take forever.

The Analogy: Think of it like sketching a painting.

1. Coarse Phase (The Rough Sketch): The computer first looks at the house as a tiny, low-resolution grid (like a 6x6 pixel image). It quickly decides, "Okay, the bedroom goes here, the kitchen there." It solves the big picture fast.
2. Fine Phase (The Detail Work): Once it has the rough sketch, it zooms in to the full resolution (the high-definition grid). It uses the rough sketch as a "warm start" (a head start) to fill in the tiny details, like placing the exact position of the coffee table.

This strategy makes the computer hundreds of times faster without losing quality.

4. Why It's Better (The Results)

The paper tested this against other top AI tools (Holodeck and AnyHome).

• Other Tools: Often created "impossible" houses. You might find a bedroom with no door, a sofa blocking the hallway, or a kitchen floating in the middle of a wall.
• Co-Layout: Created houses that actually make sense.
  • Connectivity: You can always walk from the front door to every room.
  • Logic: The furniture fits perfectly within the walls.
  • Aesthetics: The rooms are shaped nicely (not weirdly narrow or jagged).

Summary

Think of Co-Layout as a team where:

• The LLM is the Visionary who understands what you want and sets the rules.
• The Grid Solver is the Engineer who ensures the physics and geometry work perfectly.
• The Coarse-to-Fine strategy is the Efficiency Expert who ensures the job gets done quickly.

Instead of just generating a pretty picture that falls apart when you try to use it, this system generates a functional, logical, and beautiful home layout that respects the rules of real-world living.

Technical Summary

Here is a detailed technical summary of the paper "Co-Layout: LLM-driven Co-optimization for Interior Layout".

1. Problem Statement

Interior design is a complex task requiring the simultaneous consideration of room layout (spatial partitioning) and furniture placement. Existing automated approaches typically treat these as separate, sequential stages (e.g., generating a floorplan first, then placing furniture). This decoupling often leads to suboptimal outcomes, such as:

• Infeasible layouts: Furniture blocking room entrances or corridors.
• Poor connectivity: Disconnected circulation paths or rooms that are inaccessible.
• Constraint conflicts: Failure to satisfy high-level user requirements (e.g., specific adjacencies, privacy hierarchies) because the two stages do not communicate constraints effectively.

The core challenge is to create a unified framework that can translate high-level natural language prompts into precise, geometrically valid, and functionally coherent interior designs that satisfy complex spatial constraints.

2. Methodology

The proposed Co-Layout framework integrates Large Language Models (LLMs) with Integer Programming (IP) using a grid-based representation. The workflow consists of three main phases:

A. LLM-driven Preprocessing (Constraint Extraction)

Instead of manually defining constraints, the system uses a multi-agent LLM workflow to convert user text prompts into a structured scene graph.

• Input: Natural language requirements (e.g., "100 sqm apartment for a family of 3").
• Process: Specialized LLM agents analyze the input to extract:
  • Room Constraints: List of rooms, target areas, corner positions, adjacencies, and privacy requirements.
  • Furniture Constraints: List of furniture items, dimensions, orientation preferences, and relative positioning rules.
  • Environmental Constraints: Entrance location, outdoor spaces, and orientation preferences.
• Output: A structured JSON containing all semantic constraints required for the optimization model.

B. Grid-based Formulation (Integer Programming)

The design space is discretized into a 2D grid (inspired by the architectural concept of "Modulor"), where each cell $(i, j)$ is assigned binary variables.

• Variables:
  • $x^k_{i,j}$: Cell belongs to room $k$.
  • $p_{i,j}$: Cell is a corridor.
  • $f^{k,l}_{i,j}$: Cell is occupied by furniture item $l$ in room $k$.
  • $\nu_{k,l}$: Orientation of furniture (encoded via binary variables).
• Key Constraints:
  • Non-overlapping: Every cell is assigned to exactly one entity (room, corridor, or furniture).
  • Corridor Connectivity: Uses a flow-based formulation to ensure all rooms and open spaces are connected to the entrance via a single continuous network.
  • Accessibility: Ensures every room has at least one boundary cell adjacent to a corridor/open space that is not blocked by furniture.
  • Adjacency & Cornering: Enforces specific room-to-room adjacencies and corner placements.
  • Furniture Containment: Furniture must be fully contained within its assigned room and respect size/shape dimensions.
• Objective Function: A weighted sum of penalties to optimize:
  • Geometric Quality: Room rectangularity and compactness (minimizing perimeter).
  • Functional Goals: Matching target room areas, aspect ratios, and furniture distribution balance.
  • Spatial Relationships: Minimizing deviation from target relative positions of furniture.

C. Coarse-to-Fine Optimization Strategy

Solving the IP problem on a high-resolution grid is computationally expensive (NP-hard). To address this, the authors propose a Coarse-to-Fine strategy:

1. Coarse Phase: The grid is down-sampled (e.g., $12\times10 \to 6\times5$). The optimizer solves for room layout only (ignoring furniture) to find a high-level spatial organization.
2. Mapping: The coarse solution is mapped back to the fine grid as a "warm start" for the room variables.
3. Fine Phase: The full optimization runs on the original high-resolution grid, optimizing both rooms and furniture simultaneously. The coarse solution acts as a guide (via penalty terms) to maintain consistency, significantly reducing the search space and computation time.

3. Key Contributions

1. Unified Co-optimization Framework: The first approach to jointly optimize room layout and furniture placement within a single mathematical model, ensuring global consistency and avoiding the pitfalls of sequential pipelines.
2. LLM-to-IP Pipeline: A novel method to leverage LLMs for extracting complex, structured design constraints from natural language and translating them into a formal Integer Programming problem.
3. Grid-based "Modulor" Representation: A discretized grid formulation that explicitly models complex topological constraints like corridor connectivity and room accessibility, which are difficult to handle with vector-based or image-based methods.
4. Efficient Coarse-to-Fine Strategy: A scalable optimization technique that drastically reduces computation time for large-scale designs without sacrificing solution quality.

4. Experimental Results

The method was evaluated against state-of-the-art baselines (Holodeck and AnyHome) using both quantitative metrics and user studies.

• Quantitative Metrics:
  • Physical Plausibility: Co-Layout achieved 0.00 Object Overlap Rate (OOR) and 0.00 Out-of-Boundary Rate (OOB), significantly outperforming baselines which suffered from overlaps and boundary violations.
  • Aesthetics & Alignment: It scored higher on Image Quality (IQA), Aesthetic Assessment (IAA), and CLIP text-image alignment scores.
• Qualitative Analysis:
  • Baselines often produced layouts with illogical circulation (e.g., requiring passage through a bedroom to reach the living room) or unreachable rooms.
  • Co-Layout consistently generated layouts with clear circulation, logical spatial hierarchies, and fully accessible rooms.
• User Study:
  • In a study with 71 participants, Co-Layout received the highest scores across Semantic Alignment, Layout Rationality, and Path Clearance.
  • It achieved a Mean Reciprocal Rank (MRR) of 0.80, indicating a strong user preference over the baselines.
• Efficiency: The ablation study confirmed that the Coarse-to-Fine strategy reduced computation time by a significant margin (e.g., from ~1800s to ~155s for a $16\times16$ grid) while maintaining solution quality.

5. Significance

This work represents a significant advancement in automated architectural design. By bridging the gap between the semantic understanding of LLMs and the geometric precision of mathematical optimization, Co-Layout solves the "last mile" problem in generative design: ensuring that AI-generated plans are not just visually plausible but structurally and functionally valid.

The framework demonstrates that joint optimization is superior to sequential approaches for complex design tasks. It offers a practical tool for interior designers to rapidly prototype layouts that strictly adhere to user constraints, safety codes (accessibility), and ergonomic principles, potentially revolutionizing how residential and commercial spaces are planned.