ChipLingo: A Systematic Training Framework for Large Language Models in EDA

1. Problem Definition and Challenges

The paper addresses the application of Large Language Models (LLMs) to Electronic Design Automation (EDA), a highly knowledge-intensive and document-driven engineering domain. While general-purpose LLMs excel in broad tasks, their direct application to EDA faces three critical challenges:

• Insufficient Domain Expertise: General models lack specific EDA knowledge, leading to inaccurate or generic responses regarding tool commands, parameters, and design flows.
• Cross-Tool Knowledge Confusion: EDA involves diverse tools (e.g., logic synthesis, physical design) with distinct syntaxes and workflows. Models often confuse terminology and usage patterns between different tools.
• RAG Capability Degradation: A novel and critical observation is that after domain-specific training, models tend to rely heavily on parametric knowledge (internal weights) and ignore retrieved external knowledge. This "parametric bias" causes the performance of Retrieval-Augmented Generation (RAG) systems to degrade, even when correct documents are provided.

2. Methodology: The ChipLingo Framework

The authors propose ChipLingo, a systematic three-stage training pipeline designed to adapt LLMs for EDA scenarios while preserving general capabilities and RAG effectiveness.

Stage 1: Domain-Adaptive Pretraining

• Data Curation: The team constructed a corpus of over 200,000 pages covering 50+ EDA tools, including technical documentation, engineer QA records, and scripts.
• QA-Augmented Pretraining: Unlike traditional methods that use only raw text, ChipLingo injects Question-Answer (QA) format data during pretraining. This helps the model learn the relationship between knowledge and tasks earlier.
• Partial Parameter Training: To balance domain adaptation with general capability retention, the authors employ a Partial Fine-Tuning (Partial FT) strategy. They freeze the bottom layers of the model and update only a selected subset of remaining parameters. This was found to be superior to LoRA (which lacked expressiveness for complex EDA knowledge) and Full Fine-Tuning (which degraded general capabilities too much).

Stage 2: Instruction Alignment

• Data Construction: Raw engineer QA records were distilled and rewritten using strong LLMs to create high-quality, structured instruction data (Question, Reasoning Chain, Answer).
• Supervised Fine-Tuning (SFT): The model is trained on ~40,000 high-quality samples to improve instruction following and task execution capabilities.

Stage 3: RAG Scenario Training

• Addressing Degradation: To counteract the tendency of models to ignore retrieved context, the authors introduced a specialized training phase using diverse RAG scenarios:
  1. Correct Retrieval: Training the model to use relevant context.
  2. Irrelevant Retrieval: Training the model to ignore noise and rely on parametric knowledge.
  3. Incomplete Retrieval: Training the model to combine context with internal knowledge.
• Goal: This explicit training restores the model's ability to utilize external documents, ensuring stability for future EDA agents.

3. Key Contributions

1. Systematic Training Pipeline: A comprehensive framework covering data preparation, domain-adaptive pretraining (with QA augmentation), instruction alignment, and RAG scenario training specifically for EDA.
2. Empirical Analysis of Training Strategies: The paper systematically compares QA augmentation, LoRA, Full FT, and Partial FT. It finds that QA augmentation boosts domain performance, and Partial FT offers the best trade-off between domain adaptation and general capability retention compared to LoRA.
3. Identification and Mitigation of RAG Degradation: The authors quantify the phenomenon where domain training reduces RAG efficacy and demonstrate that explicit RAG scenario training can reverse this trend, restoring the model's ability to leverage external knowledge.
4. EDA-Bench: The creation of a new evaluation benchmark covering four major EDA tool categories (Logic Synthesis, Physical Implementation, Simulation Verification, and Design-for-Test) with thousands of real-world engineering questions.

4. Experimental Results

The models were evaluated on EDA-Bench using an LLM-as-a-Judge approach.

• Performance Gains:
  • ChipLingo-8B achieved 59.7% accuracy, significantly outperforming its base model (Qwen3-8B at 26.85%) and surpassing a much larger general-purpose model (DeepSeek-v3.2 at 56.28%).
  • ChipLingo-32B reached 70.02% accuracy, approaching the performance of leading closed-source commercial models (e.g., Claude-Sonnet-4.5 at 71.11%).
• Ablation Findings:
  • QA Augmentation: Improved domain learning speed and final accuracy.
  • Partial FT vs. LoRA: Partial FT achieved near Full FT performance on EDA tasks while maintaining much higher scores on general benchmarks (IFEval, SimpleQA, HumanEval) compared to LoRA.
  • RAG Recovery: Without RAG training, providing correct retrieval context actually decreased accuracy (negative gain). After RAG scenario training, the model regained a positive gain (+5.1) when correct context was provided.

5. Significance and Future Impact

• Foundation for EDA Agents: The paper argues that reliable retrieval utilization is a foundational capability for building higher-level EDA agents and harness systems. Without the ability to stably leverage external documents and tool feedback, autonomous engineering workflows cannot function.
• Domain Adaptation Insights: The study provides empirical evidence that in knowledge-intensive vertical domains, parameter-efficient methods like LoRA may be insufficient due to the complexity and density of the knowledge, and that explicit training on retrieval scenarios is necessary to prevent "knowledge overwriting" of retrieval capabilities.
• Open Contribution: The authors plan to publicly release EDA-Bench, fostering further research and standardization in the application of LLMs to semiconductor design.

In conclusion, ChipLingo demonstrates that a structured, multi-stage training approach can successfully adapt general LLMs to the complex, document-heavy EDA domain, solving critical issues regarding knowledge specificity and retrieval reliability.