iScript: A Domain-Adapted Large Language Model and Benchmark for Physical Design Tcl Script Generation

This paper introduces iScript, a domain-adapted Qwen3-8B model and its corresponding benchmark for generating reliable Innovus Tcl scripts, which leverages a novel multi-stage data synthesis pipeline and a two-step verification framework to overcome data scarcity and outperform state-of-the-art LLMs in physical design automation.

The Gist

Imagine you are building a massive, incredibly complex skyscraper. In the world of computer chips (which are like tiny, microscopic cities), this "skyscraper" is the physical design of a microchip. To build it, engineers use a special language called Tcl (Tool Command Language). Think of Tcl as the instruction manual or the "recipe" that tells the construction robots exactly where to place every single brick, wire, and room in the chip.

For decades, writing these recipes has been a nightmare. It requires thousands of lines of code, and if you make a tiny mistake, the whole building collapses (or the chip doesn't work). General AI models (like the ones that write poems or answer trivia) are terrible at this because they haven't seen enough of these specific "recipes," and they don't understand the strict rules of chip construction.

Here is what the iScript paper does, explained simply:

1. The Problem: The "Chef" Who Doesn't Know the Kitchen

Imagine you hire a world-famous chef (a general AI) to cook a very specific, obscure dish from a remote village. The chef has never seen the ingredients, doesn't know the local spices, and has never read the village's secret cookbook. If you ask them to cook it, they will guess, and the result will likely be inedible.

• The Reality: General AI models try to write chip scripts, but they fail because chip data is secret (proprietary), rare, and uses very specific, weird commands that the AI has never learned.

2. The Solution: Training a "Specialist Chef" (iScript)

The authors created iScript, a specialized AI chef trained specifically for chip building. They didn't just give the AI a few recipes; they built a massive training program.

• The "Synthesis Pipeline" (The Cooking School): Since there weren't enough real recipes to teach the AI, they built a factory to create them.

  • Step 1: They took a list of all the valid "ingredients" (commands) and mixed them together randomly to create thousands of fake recipes.
  • Step 2: They ran these fake recipes through a "grammar police" (a syntax checker) to throw out any that were nonsense.
  • Step 3 (The Magic): They used a super-smart AI (a "Teacher") to look at the valid recipes and ask, "What was the chef trying to do here?" The Teacher then wrote a story (called Chain-of-Thought) explaining the logic behind the recipe.
  • Result: They ended up with 10,000 high-quality examples of: The Request ("Build a clock tower") + The Reasoning ("First, I need to lay the foundation, then add the gears...") + The Code (The actual Tcl script).

• The Training: They took a smart base AI (Qwen3-8B) and taught it in two phases:

  1. Language Immersion: Learning the specific vocabulary and grammar of chip scripts.
  2. Logic Training: Learning why to write the code, not just what to write, using the "stories" (Chain-of-Thought) they generated.

3. The Test: The "Driving License" Exam (iScript-Bench)

Before you can drive a truck, you need a test. But how do you test an AI's ability to write chip code without actually building a real chip (which costs millions of dollars)?

The authors created iScript-Bench, a standardized driving test with three levels:

• Level 1 (The Parking Lot): Simple tasks, like "Turn on the lights."
• Level 2 (The City Streets): Combining commands, like "Drive to the store and park."
• Level 3 (The Highway): Complex logic, like "Navigate traffic while avoiding potholes and changing lanes."

They tested iScript against other famous AIs (like GPT-4, Gemini, and Claude). iScript won easily, especially in the complex tasks where the others failed completely.

4. The Grading System: The "Two-Step Check"

How do you grade the exam without a real chip?

1. The Grammar Check: First, they run the code in a tiny, safe "sandbox" (a simulation). If the code has a typo or a syntax error, it fails immediately.
2. The Logic Check: If the code passes the grammar check, a second AI (the "Proctor") reads the code and the original request. The Proctor checks: "Does this code actually do what the user asked?"
   • Cool Trick: They proved this AI Proctor is almost as good as a human expert, but much faster.

The Big Takeaway

This paper is like saying: "We can't just ask a general smart person to build a rocket ship. We need to train a specialist using a factory that creates practice problems, and then test them with a rigorous exam."

iScript is that specialist. It proves that if you give an AI the right training data (synthesized from scratch) and the right way to think (Chain-of-Thought), it can master the incredibly difficult task of writing the code that builds our modern world's computer chips.

In short: They taught an AI to speak "Chip Engineer" fluently, created a test to prove it works, and showed that a specialized AI is far better at this job than a general one.

Technical Summary

Here is a detailed technical summary of the paper "iScript: A Domain-Adapted Large Language Model and Benchmark for Physical Design Tcl Script Generation."

1. Problem Statement

The paper addresses critical bottlenecks in the Physical Design (PD) stage of modern ASIC implementation, specifically regarding the generation of Tcl scripts used to control EDA tools (e.g., Cadence Innovus).

• Data Scarcity: High-quality, public PD Tcl scripts are rare due to proprietary workflows, making it difficult to train general Large Language Models (LLMs).
• Domain Complexity: PD scripts require deep domain knowledge. Commands are highly specialized, and there is strong semantic coupling between steps (e.g., placement regions must align with floorplan geometry). General LLMs struggle with this sparse vocabulary and complex logic.
• Evaluation Gap: There is no standardized benchmark for PD script generation. Furthermore, full functional verification requires expensive commercial EDA tools, making scalable, reproducible evaluation nearly impossible. Existing methods rely on incomplete syntax checks or qualitative human review.

2. Methodology

The authors propose a comprehensive framework consisting of a data synthesis pipeline, a specialized model training strategy, and a novel evaluation system.

A. Multi-Stage Data Synthesis Pipeline

To overcome data scarcity, the authors constructed a 10,000-sample dataset of (Requirement, Chain-of-Thought, Script) tuples using a three-stage process:

1. Script Generation: Extracted core Tcl commands and parameters from manuals and community sources (CSDN, forums). A multi-agent system generated a vast corpus of script fragments via permutations.
2. Static Syntax Validation: A Tcl Linter filtered out syntactically invalid scripts, ensuring the corpus was structurally correct.
3. Requirement Back-Inference & CoT Generation: A powerful "teacher" model (GPT-4.1) was used to:
   • Reverse-engineer the natural language requirement from the valid script.
   • Generate a Chain-of-Thought (CoT) explanation detailing the reasoning behind the script's logic.
   • Result: A high-quality dataset suitable for Supervised Fine-Tuning (SFT).

B. iScript Model Training

The model is based on Qwen3-8B and trained using a two-stage strategy:

1. Domain-Adaptive Continued Pre-Training (CPT): The model is pre-trained on raw Tcl scripts to internalize the specific vocabulary, syntax, and patterns of Innovus Tcl. This improves "fluency" in the domain language.
2. Supervised Fine-Tuning (SFT) with CoT: The model is fine-tuned on the (Requirement, CoT, Script) dataset. The inclusion of CoT teaches the model not just what to generate, but why, enhancing its ability to handle complex, multi-step design intents.

C. Two-Step Verification Framework

To evaluate scripts without running full commercial toolflows, the authors propose:

1. Static Syntax Verification: Scripts are executed in a lightweight, physical-design-consistent sandbox. Logs are parsed for error patterns (e.g., ** ERROR:) to determine syntactic validity.
2. LLM-Based Functional Evaluation: Scripts passing syntax checks are evaluated by a knowledge-enhanced LLM (Gemini-2.5-pro). The evaluator is provided with the requirement, the "golden" reference script, and official command definitions to verify if the generated script fulfills the design intent.

3. Key Contributions

1. iScript: A domain-adapted LLM specifically trained for Physical Design Tcl generation, capable of producing syntactically valid and semantically meaningful scripts.
2. iScript-Bench: The first comprehensive benchmark for PD Tcl generation. It covers 5 task categories (Design Info Query, Netlist Analysis, Floorplan Adjustment, Placement Optimization, Timing Analysis) across 3 difficulty levels (L1-L3), totaling 116 test cases.
3. Evaluation Framework: A scalable, reproducible two-step verification strategy (Syntax + LLM Functional Evaluation) that avoids the cost of full EDA tool execution while maintaining high reliability.
4. Data Synthesis Pipeline: A novel method for generating high-quality, reasoning-rich training data from low-resource sources using back-inference and CoT.

4. Experimental Results

The authors compared iScript against four state-of-the-art general LLMs (GPT-4.1, Gemini-2.5-pro, Claude-sonnet-4.5, DeepSeek-V3.1) on iScript-Bench.

• Overall Performance: iScript significantly outperformed all baselines.
  • Pass@1 (Syntax): iScript achieved 59.48% vs. the next best (Gemini) at 31.03%.
  • Pass@1 (Function): iScript achieved 18.97% vs. Gemini at 14.66%.
  • Pass@5 (Function): iScript reached 46.55%, nearly double the performance of the second-best model.
• Difficulty Levels: iScript demonstrated superior robustness as task difficulty increased.
  • At Level 3 (Hardest), general models (Claude, DeepSeek) dropped to ~3.85% Pass@1 syntax accuracy, while iScript maintained 61.54%.
• Category Performance: iScript showed particular strength in Floorplan Adjustment (FA) and Design Information Query (DIQA), where other models often achieved 0% functional correctness.
• Validation: A study comparing LLM evaluation against human experts showed the LLM evaluator had near-perfect precision (no false positives), acting as a rigorous lower-bound estimator.

5. Significance

• Bridging the Gap: This work demonstrates that domain adaptation (CPT + SFT) is essential for applying LLMs to specialized engineering domains like EDA, where general models fail due to data scarcity and semantic complexity.
• Standardization: By introducing iScript-Bench, the paper provides the community with a standardized metric and dataset, enabling fair comparison and reproducible research in "EDA+LLM."
• Practical Impact: The proposed framework offers a pathway to automate time-consuming, expertise-dependent scripting tasks for physical design engineers, potentially lowering the barrier to entry for advanced-node design.
• Limitations & Future Work: The authors note that training data volume is still slightly insufficient for the most complex logic combinations, and a fully automated execution-based evaluation system (using real tools) remains the ultimate goal for future work.