🤖 AI

RulePlanner: All-in-One Reinforcement Learner for Unifying Design Rules in 3D Floorplanning

This paper introduces RulePlanner, an all-in-one deep reinforcement learning framework that unifies the handling of complex 3D IC design rules through novel matrix representations, constrained action spaces, and quantitative reward signals, thereby eliminating the need for labor-intensive manual post-processing and demonstrating strong effectiveness and transferability on public benchmarks.

Original authors: Ruizhe Zhong, Xingbo Du, Junchi Yan

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ruizhe Zhong, Xingbo Du, Junchi Yan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an architect tasked with designing a massive, multi-story skyscraper. But this isn't a normal building; it's a microchip. Inside this skyscraper, you have thousands of tiny rooms (called "blocks") that need to hold specific equipment. Your job is to decide exactly where each room goes and what shape it should be.

This process is called Floorplanning.

The Problem: A Messy Rulebook

In the past, architects (chip engineers) had a few simple rules: "Don't let rooms overlap," and "Keep the whole building inside the lot." But as technology gets smaller and more complex (like moving to 3D skyscrapers with stacked layers), the rulebook has exploded.

Now, the rules are incredibly specific and strict:

The "Handshake" Rule: Certain rooms on different floors must line up perfectly so they can "shake hands" (share data) through the floor.
The "Best Friend" Rule: Some rooms must be glued right next to each other because they work together.
The "Edge" Rule: Some rooms must be placed exactly against a specific wall or corner.
The "No Overlap" Rule: Rooms cannot crash into each other.

The Old Way: Current computer programs are like junior architects. They are good at following one or two of these rules, but if you give them the whole complex rulebook, they get confused. They might place the rooms perfectly for the "Handshake" rule but forget the "Best Friend" rule. When this happens, human experts have to spend hours manually fixing the mistakes, like a tired editor correcting a messy manuscript.

The Solution: RulePlanner

The authors of this paper built a new AI system called RulePlanner. Think of it as a super-intelligent, all-seeing architect who never forgets a single rule.

Here is how it works, using simple analogies:

1. The "Magic Map" (Matrix Representations)

Instead of just looking at a list of rules, RulePlanner creates special "Magic Maps" (matrices) for the chip.

Imagine a grid covering the entire floor of the building.
For the "Best Friend" rule, the map lights up in Yellow wherever a room can sit next to its friend, and Red everywhere else.
For the "Edge" rule, another map lights up only near the walls where a specific room is allowed.
The AI looks at all these maps at once, creating a clear picture of where it is allowed to put things.

2. The "Bouncer" (Action Space Constraints)

In many AI systems, the computer tries a move, realizes it broke a rule, and then gets a "penalty" (a negative score) to learn not to do it again. This is slow and messy.

RulePlanner uses a Bouncer. Before the AI even tries to place a room, the Bouncer checks the Magic Maps.

If a spot is Red (forbidden), the Bouncer blocks it immediately.
The AI is only allowed to pick from the Yellow spots (valid moves).
This means the AI never wastes time making illegal moves. It only learns from the good ones.

3. The "Scorecard" (Quantitative Rewards)

Even with the Bouncer, the AI needs to know how well it did. The paper introduces a precise scorecard.

Instead of just saying "Good job," it measures exactly how close two rooms are to their ideal distance.
It gives points for every inch of alignment and every millimeter of separation.
This helps the AI fine-tune its placement to be perfect, not just "okay."

The Results: A Perfect Building

The researchers tested RulePlanner on standard chip designs (like the MCNC and GSRC benchmarks).

The Old Architects: They often failed to follow all the rules at once. They might get the shape right but miss the alignment, or vice versa.
RulePlanner: It successfully followed all seven complex industrial rules simultaneously. In many tests, it achieved a perfect score (0.000 distance error) for placing rooms next to their required partners, while other methods failed completely.

Why This Matters

The paper claims that RulePlanner is extensible. This means if chip designers invent a new rule in the future (like a new way rooms must connect), they can just add a new "Magic Map" to the system. The AI doesn't need to be retrained from scratch; it just learns to read the new map.

In summary: RulePlanner is an AI that doesn't just guess where to put chip parts. It uses a system of "Magic Maps" and a strict "Bouncer" to ensure that every single complex rule of modern 3D chip design is followed perfectly, saving human engineers from hours of tedious manual fixing.

Technical Summary: RulePlanner

Problem Statement

Floorplanning is a critical, NP-hard stage in modern Application-Specific Integrated Circuit (ASIC) design, responsible for determining the coordinates and shapes of functional modules. As technology nodes scale (e.g., to 5-nm) and designs move toward 3D scenarios with multiple stacked layers, the complexity of adhering to hardware design rules has increased significantly. These rules include boundary constraints (aligning blocks with specific terminals), grouping constraints (physically abutting blocks for voltage islands), inter-die block alignment, pre-placement requirements, non-overlap, outline fitting, and shape constraints.

Current state-of-the-art methods, including analytical, heuristic-based, and existing reinforcement learning (RL) approaches, struggle to handle these constraints simultaneously. Most methods are tailored to specific rules (e.g., non-overlap or alignment) and often fail to satisfy others concurrently. Consequently, violations of unhandled rules require labor-intensive, time-consuming manual post-processing by expert engineers. Existing RL methods typically rely on incorporating penalties into reward functions, which is often insufficient for accurately addressing complex, hard constraints without extensive fine-tuning or failing to converge to valid solutions.

Methodology

The paper proposes RulePlanner, a deep reinforcement learning framework designed to unify the handling of diverse hardware design rules in 3D floorplanning. The approach frames the problem as an episodic Markov Decision Process (MDP) solved by an actor-critic agent. The core innovation lies in a unified framework comprising three key components:

1. Novel Matrix Representations for Design Rules

Instead of treating constraints solely as soft penalties, RulePlanner introduces explicit matrix representations to model design rules:

Adjacent Terminal Mask: A matrix $T \in \mathbb{R}^{W \times H}$ where each element represents the Manhattan distance between a potential block position and a required adjacent terminal. This models boundary constraints.
Adjacent Block Mask: A matrix $B \in \mathbb{R}^{W \times H}$ representing the adjacency length between a movable block and already placed blocks. This models grouping constraints (e.g., voltage islands).
Alignment Mask: Used to model inter-die block alignment constraints.

These masks are computed efficiently using GPU parallelization (e.g., meshgrid operations) to handle the computational cost of evaluating all possible positions.

2. Action Space Constraints

Rather than relying solely on reward penalties to discourage invalid moves, RulePlanner applies hard constraints directly to the action space to filter out invalid actions before they are sampled.

Position Filtering: The policy network generates logits for block positions. A binary Availability Mask ( $M$ ) is constructed by combining the binarized adjacent terminal, adjacent block, alignment, and position masks. This mask is applied to the policy's output distribution (via element-wise multiplication and softmax) to zero out probabilities for invalid positions, ensuring the agent only samples placements that satisfy boundary, grouping, and alignment rules.
Aspect Ratio Clipping: The aspect ratio is modeled as a Gaussian distribution but is clipped to the valid range $[AR_{min}, AR_{max}]$ to satisfy shape constraints.

3. Quantitative Analysis and Reward Shaping

The framework employs quantitative metrics to evaluate constraint satisfaction, which are integrated into the reward function:

Metrics: Block-terminal distance (to be minimized), block-block adjacency length (to be maximized), alignment score (to be maximized), Half-Perimeter Wire Length (HPWL), and overlap.
Reward Function: A dense reward scheme is used, incorporating self-adaptive normalization to handle the varying magnitudes of different metrics (e.g., HPWL vs. alignment scores). The reward includes weighted terms for these metrics, guiding the policy to satisfy multiple rules simultaneously.

Model Architecture

Input: The state consists of stacked design rule matrices (masks, canvas, wire mask) and a netlist graph.
Encoder: A Transformer-based encoder processes the netlist graph, while a CNN processes the spatial matrices.
Hybrid Action Space: The agent outputs a discrete position $(x, y)$ and a continuous aspect ratio $AR$.
Training: The system uses Proximal Policy Optimization (PPO) with a hybrid action space.

Key Contributions

Unified Framework for Diverse Rules: RulePlanner is presented as the first framework capable of simultaneously satisfying over seven industrial design rules (boundary, grouping, inter-die alignment, pre-placement, non-overlap, outline, and shape) in 3D floorplanning. The framework is modular and extensible to new rules.
Novel Representations: The introduction of adjacent terminal and adjacent block masks to explicitly model complex constraints, moving beyond simple penalty-based approaches.
Action Space Pruning: The method enforces hard constraints by filtering the action space via binary masks, ensuring that sampled actions inherently satisfy critical design rules, thereby reducing the search space and accelerating training.
Quantitative Metrics: The proposal of specific metrics (e.g., block-block adjacency distance) to quantitatively evaluate and optimize for complex constraints.

Experimental Results

The authors evaluated RulePlanner on public benchmarks (MCNC and GSRC) across three tasks with varying combinations of design constraints.

Constraint Satisfaction: RulePlanner achieved a block-terminal distance of 0.000 across all tested circuits in Task 1, indicating perfect alignment with terminals. In Task 2, it achieved a block-block adjacency length of 0.223 on average, significantly outperforming the best baseline (0.082).
Comparison with Baselines: Compared against analytical, heuristic (B*-3D-SA, WireMask-BBO), and RL-based baselines (GraphPlace, DeepPlace, MaskPlace, FlexPlanner), RulePlanner consistently satisfied all design rules simultaneously. While some baselines (e.g., Analytical, WireMask-BBO) achieved lower HPWL on specific large circuits, they failed to satisfy boundary, grouping, or alignment constraints, often resulting in significant violations.
Generalization and Transferability:
- Zero-Shot: A model trained on the n100 circuit demonstrated strong zero-shot generalization to unseen circuits (n10 to n300), maintaining high performance on constraint satisfaction metrics without fine-tuning.
- Fine-Tuning: Fine-tuning pre-trained weights on larger circuits (n200, n300) achieved comparable or superior performance to training from scratch with significantly reduced computational cost.

Significance and Claims

The paper claims that RulePlanner addresses a critical bottleneck in modern chip design: the inability of current automated tools to handle the multifaceted and simultaneous constraints of 3D IC design. By unifying diverse rules into a single framework through novel matrix representations and action space constraints, the method eliminates the need for extensive manual post-processing.

The authors emphasize that their approach represents a strategic shift in priority: ensuring full compliance with complex design rules is deemed more critical than achieving the absolute lowest HPWL in isolation. The framework's extensibility is highlighted as a key feature for addressing emerging challenges in future chip designs, such as thermal optimization (proposed as a future direction) and other implicit constraints. The work demonstrates that deep reinforcement learning, when augmented with domain-specific structural constraints and representations, can effectively solve the NP-hard problem of constrained 3D floorplanning.