Wrong Code, Right Structure: Learning Netlist Representations from Imperfect LLM-Generated RTL

This paper proposes a cost-effective framework that leverages structurally informative but functionally imperfect LLM-generated RTL to train netlist representation models, effectively overcoming data scarcity and outperforming methods reliant on scarce high-quality labeled datasets.

The Gist

Here is an explanation of the paper "Wrong Code, Right Structure" using simple language and creative analogies.

The Big Problem: The "Gold Standard" Bottleneck

Imagine you want to teach a robot to recognize different types of cars (a Ferrari, a pickup truck, a bus). To do this well, you need thousands of photos of these cars.

In the world of computer chips (hardware), the "photos" are called Netlists. These are the blueprints that show how the tiny electronic switches (gates) are connected.

• The Catch: Real chip designs are like secret recipes guarded by companies (Intellectual Property). They are expensive to buy, and even harder to label. You can't just ask a human to look at a complex chip and say, "Oh, this part is the memory, and that part is the math calculator."
• The Result: Researchers only have a tiny handful of labeled examples. It's like trying to teach a robot to recognize all the cars in the world using only three photos. The robot gets confused and fails when it sees a new car.

The New Idea: The "Imperfect Architect"

The authors of this paper had a clever idea. They asked: What if we use Artificial Intelligence (LLMs) to generate the blueprints for us?

LLMs are great at writing code. They can write thousands of chip designs in seconds. However, they are bad at logic. If you ask an LLM to design a 2-bit multiplier, it might write code that looks perfect but actually calculates the wrong numbers.

The "Wrong Code, Right Structure" Discovery:
The authors realized something surprising. Even though the LLM's code does the wrong math, the shape of the blueprint is often correct.

• Analogy: Imagine an apprentice architect who is terrible at math. If you ask them to draw a house with a kitchen, a bedroom, and a bathroom, they might put the stove in the wrong spot or forget the sink. But, they will still draw a house with three distinct rooms connected by hallways. The structure is right, even if the function is broken.

The paper argues: "We don't need the math to be perfect; we just need the shape to be recognizable."

The Solution: A Three-Step Factory

The team built a system (a pipeline) to turn these "flawed" AI drafts into a super-powerful training dataset. Think of it as a factory with three stations:

1. The Generator (The Creative Apprentice)

Instead of asking the AI to just copy-paste code, they ask it to write a specification (a description of what the chip should do) and then generate the code from scratch.

• Why? This forces the AI to try different architectural styles. Sometimes it builds a "ripple-carry" adder, other times a "carry-lookahead" adder. This creates diversity, which is crucial for the robot to learn how to recognize chips no matter how they are built.

2. The Filter (The Quality Control Inspector)

Since the AI makes mistakes, we can't use all the drafts. We need to throw out the garbage.

• The Trick: They don't check if the math is right (that's too expensive). Instead, they check if the shape looks like a real chip.
• Analogy: Imagine a pile of toy cars. Some are missing wheels, some are painted weird colors, and some are just blocks of wood. The inspector doesn't check if the engine works. They just check: "Does this look like a car?" If it has four wheels and a chassis, it stays. If it looks like a toaster, it gets thrown away.
• They use a mathematical "similarity score" to keep the designs that look structurally similar to real chips, even if the code inside is buggy.

3. The Voter (The Panel of Judges)

For the designs that are supposed to be different (to teach the robot about variety), they use a second AI to act as a judge.

• They generate 10 different versions of a design.
• They ask a second AI: "Which 3 of these look the most interesting and diverse?"
• This ensures the training data isn't just a pile of junk, but a curated collection of unique, high-quality structural variations.

The Result: A Super-Student

They took this massive pile of "imperfect but structurally sound" blueprints and used them to train a Graph Neural Network (a type of AI that studies connections).

The Test:
They tested this AI on real-world chips it had never seen before (like the PicoRV32 and NEORV32 processors).

• Old Way: Models trained on tiny, perfect datasets failed to recognize complex parts of the chip.
• New Way: The model trained on the "imperfect" data performed better than models trained on the scarce, perfect data.

The Takeaway

This paper is a game-changer because it breaks the "Data Bottleneck."

• Before: We were stuck because we couldn't get enough labeled data.
• Now: We can generate millions of "imperfect" examples that teach the AI the shape of the problem.

In simple terms: You don't need a perfect chef to teach someone how to recognize a cake. You just need a bunch of people who can draw a cake, even if their drawings have a few extra sprinkles in the wrong place. As long as the drawing looks like a cake, the student will learn to spot a real cake in the wild.

This allows engineers to analyze, secure, and understand complex hardware designs much faster and cheaper than ever before.

Technical Summary

Here is a detailed technical summary of the paper "Wrong Code, Right Structure: Learning Netlist Representations from Imperfect LLM-Generated RTL".

1. Problem Statement

The Data Bottleneck in Circuit Representation Learning:
Learning effective representations for hardware netlists (gate-level circuits) is critical for downstream tasks like IP piracy detection, reverse engineering, and functional understanding. However, current data-driven approaches (supervised and self-supervised learning) are severely constrained by the scarcity of high-quality, labeled datasets.

• IP Protection: Real-world designs are proprietary, making large-scale annotated datasets virtually impossible to acquire.
• Cost of Annotation: Manually labeling netlists for functional boundaries or component types is prohibitively expensive.
• Limitations of Existing Augmentation: Traditional data augmentation relies on logic rewriting or synthesis variations (e.g., changing optimization targets). These methods preserve the same high-level architecture, failing to provide the architectural diversity needed for models to generalize to complex, unseen real-world designs.

The LLM Challenge:
While Large Language Models (LLMs) can generate Register-Transfer-Level (RTL) code at scale, the output is often functionally incorrect. Historically, this functional noise has prevented the use of LLM-generated code in rigorous circuit analysis, as functional correctness is usually a prerequisite for training.

2. Core Insight

The authors propose a counter-intuitive observation: Even when LLM-generated RTL is functionally imperfect, the resulting synthesized netlists often preserve the structural patterns indicative of the intended functionality.

• Structure and function are partially decoupled.
• A "wrong" code (functional error) can still yield a "right" structure (topological similarity to the golden design).
• This allows for the use of noisy, imperfect data for representation learning if the structural fidelity is maintained.

3. Methodology

The paper proposes an end-to-end framework that transforms imperfect LLM-generated RTL into robust training data for Graph Neural Networks (GNNs). The framework consists of three main stages:

A. Circuit Data Augmentation (The Core Innovation)

This stage generates a massive, diverse dataset of netlists from limited inputs (functional specifications or existing RTL).

1. LLM-Based RTL Generation:
   • The system accepts functional specifications (or extracts them from existing RTL) and prompts an LLM to generate RTL code.
   • Bottom-Up Approach: To handle complexity, sub-modules are generated from their own specifications.
   • Synthesizer Feedback Loop: A debug agent analyzes synthesis error logs and suggests fixes, ensuring all generated code is synthesizable (convertible to a netlist), even if the logic is functionally flawed.
2. Netlist-Level Filtering (Structural Similarity):
   • Since LLM outputs are noisy, a filtering mechanism is applied.
   • Metric: The generated netlist ($G_{gen}$) is converted to a graph and compared against a "golden" reference netlist ($G_{gold}$) using graph embeddings (Mean + Max pooling).
   • Thresholding: Designs with a cosine similarity below a threshold $\tau$ are discarded. This ensures that while the code might be functionally buggy, the structure remains topologically relevant to the target function.
3. RTL-Level Architecture Voting:
   • To ensure architectural diversity (e.g., ripple-carry vs. carry-lookahead adders), a subset of designs is curated differently.
   • The LLM acts as an evaluator, voting on a batch of generated designs based on architectural diversity and complexity.
   • Top-$k$ diverse designs are selected to prevent the model from overfitting to a single implementation style.

B. Netlist Representation Learning

• Graph Construction: Netlists are transformed into undirected graphs where nodes are logic gates and edges are wires.
• Feature Extraction: Node features include connectivity (PI/PO status), functional type (AND, XOR, etc.), and structural properties (in/out degree).
• Training: A Graph Neural Network (specifically GraphSAINT for scalability) is trained using a message-passing paradigm to learn embeddings that capture both structural and functional intent.

C. Downstream Classification Tasks

The learned embeddings are used for:

• Node-Level Classification: Identifying sub-circuit boundaries (e.g., locating a specific multiplier within a flattened netlist).
• Graph-Level Classification: Classifying the entire component's function (e.g., identifying an IP block).

4. Key Contributions

1. Rethinking Imperfect RTL: The paper establishes that netlist structural features are robust to functional errors in source RTL, providing a theoretical basis for using noisy supervision in hardware AI.
2. Cost-Effective Framework: It introduces the first framework to systematically leverage functionally imperfect LLM-generated RTL. It reduces data preparation costs by orders of magnitude compared to manual annotation while achieving superior architectural diversity compared to rule-based augmentation.
3. Scalability to IP-Level: The framework successfully scales from operator-level tasks to complex, real-world System-on-Chip (SoC) scenarios (IP-level partitioning), breaking the data bottleneck.

5. Experimental Results

The framework was evaluated on benchmarks ranging from simple arithmetic operators to full SoCs (PicoRV32 and NEORV32).

• Sub-Circuit Identification (Operator Level):
  • Models trained on the LLM-augmented dataset (specifically the "Voting" set) achieved an F1-Macro of 93.79%, outperforming the baseline trained on scarce real data (90.15%).
  • This demonstrates that synthetic, noisy data can enrich the feature space better than limited high-quality data.
• Generalization to Unseen Architectures:
  • The "Voting" mechanism (selecting diverse architectures) significantly outperformed raw LLM generation and baseline methods, confirming that architectural diversity is key to generalization.
• IP-Level Case Study (SoC Partitioning):
  • Task: Identifying the CPU Core boundary in an unseen SoC (NEORV32) using a model trained on PicoRV32 data.
  • Results:
    • Rule-Based Augmentation: F1 = 58.28% (High false positives due to lack of architectural diversity).
    • LLM-Raw (Noisy, Unfiltered): F1 = 60.44% (Better than rule-based due to diversity).
    • LLM-Filtered (Full Framework): F1 = 68.35%.
  • The structural filtering mechanism significantly improved precision (from 58.90% to 71.02%) by removing non-core components that lacked structural similarity to the target.

6. Significance

This work fundamentally shifts the paradigm of hardware representation learning. Instead of treating LLM errors as a dealbreaker, the authors harness the structural invariance of LLM outputs to solve the data scarcity crisis.

• Practical Impact: It offers a scalable, low-cost solution for generating training data for hardware security and reverse engineering, areas previously stalled by the lack of labeled IP.
• Broader Implication: It suggests that for structural learning tasks, functional perfection in the training data may be less critical than structural diversity and fidelity, a principle that could apply to other domains involving graph-structured data generation.