Bridging the Sim-to-Real Gap in Semiconductor Visual Program Synthesis via Input Binarization

Technical Summary: Bridging the Sim-to-Real Gap in Semiconductor Visual Program Synthesis via Input Binarization

Problem Statement
Precise parametric control over circuit geometry is critical for semiconductor inspection and metrology. However, acquiring sufficient real-world training data is costly. While generative models like Diffusion Models and GANs can augment data, they lack the nanometer-scale geometric accuracy required for these tasks. Existing image-to-code approaches are often limited to environments with readily available image-code pairs (e.g., web data) or require specialized network designs for specific domains. Furthermore, a significant "sim-to-real" domain gap exists when models trained on synthetic data attempt to process real Scanning Electron Microscope (SEM) images, primarily due to SEM-specific noise, texture, and edge effects that obscure geometric structures.

Methodology
The authors propose a visual program synthesis framework that converts semiconductor inspection images into a reproducible, editable Domain-Specific Language (DSL). The methodology consists of four key components:

1. DSL Design: A lightweight DSL was developed specifically for Manhattan wiring architectures. Unlike raster images, this language uses vector-based geometric commands to define circuit patterns. It includes primitives for the canvas (CANVAS), conductive traces with parameterized "dogbone" shapes for stable via connections (WIRE), and interlayer connections (VIA). The routing paths are described as sequences of relative Manhattan movements (Horizontal/Vertical), allowing for flexible, one-stroke descriptions of complex topologies.
2. Synthetic Data Generation: To train the model, the authors generated 19,200 synthetic image-DSL pairs. Real SEM topologies were categorized into Vertical Stripes, Horizontal Lines, and Manhattan patterns. Structural parameters (e.g., grid divisions, wiring densities, bend probabilities) were randomized using broad uniform distributions to prevent overfitting to specific spatial frequencies. The synthetic images were rendered as binary patterns with optional salt impulse noise, matching the binary nature of the DSL.
3. VLM Training: A Vision-Language Model (specifically Qwen3-VL-8B) was fine-tuned via Supervised Fine-Tuning (SFT) to translate input images into the corresponding DSL sequences. The model was trained exclusively on the synthetic binary data.
4. Input Binarization Strategy: To address the domain gap during inference, real grayscale SEM images are preprocessed using global thresholding (fixed at a threshold of 100) to convert them into binary images before being fed to the VLM. This strategy strips away SEM-specific texture and noise, aligning the real input distribution with the binary synthetic training data and forcing the model to focus on geometric structure.

Key Contributions

• Visual Program Synthesis Framework: The introduction of a framework that translates semiconductor images into editable DSL code, enabling controlled generation of training data with exact parameter manipulation (e.g., linewidth, length).
• Input Binarization for Domain Adaptation: The demonstration that simple texture abstraction via input binarization effectively bridges the sim-to-real gap. The paper hypothesizes that texture and noise are the dominant factors hindering VLM inference in this domain, and their removal yields substantial gains.
• Reusability: The approach treats the language specification as a design variable, allowing the same VLM architecture to be reused across different inspection domains by simply changing the DSL.

Results
Experiments were conducted on the MIIC dataset (1,034 test images). The proposed method (binary input) was compared against a baseline using raw grayscale inputs.

• Quantitative Performance: Input binarization improved all evaluated metrics for executable outputs. The mean Dice coefficient increased from 0.4393 (raw input) to 0.5256 (binary input). Improvements were also observed in IoU (0.2865 to 0.3619), Boundary F1 score (0.4054 to 0.4412), and Skeleton F1 score (0.1276 to 0.1746).
• Upper Bound Analysis: While binary inputs improved performance on real data, the scores remained lower than those achieved on synthetic evaluation data (Dice: 0.6340), indicating that while texture gaps were reduced, other non-color domain gaps (e.g., subtle shape differences) persist.
• Complexity Trade-off: A positive correlation was found between structural complexity (measured by compressed DSL size) and reconstruction error. Accuracy declines as patterns become more intricate, with failures often manifesting as topology-level errors like missing or hallucinated wires.

Significance and Claims
The paper claims that input binarization is a highly effective, simple strategy for mitigating the sim-to-real gap in semiconductor visual program synthesis. By abstracting texture information, the method allows a model trained solely on synthetic data to robustly reconstruct editable DSL from real SEM images. This establishes a reliable foundation for procedural data augmentation, enabling the active generation of diverse training samples, including hard negatives and critical areas, which are difficult to collect from real samples. The authors note that future work will focus on optimizing the DSL for token efficiency and improving the model's long-context reasoning to address the observed complexity trade-offs.