🤖 AI

Foundation Models for Automatic CAD Generation

This paper introduces LLMForge, a multi-model framework for automatic parametric CAD generation that evaluates seven foundation models on a benchmark of 97 engineering design problems, demonstrating that compact instruction-tuned models can achieve high-quality results comparable to larger systems while revealing specific challenges in handling rotationally symmetric geometries through both analytic and vision-language model-based critique regimes.

Original authors: J de Curtò, Victoria Guillén, I. de Zarzà

Published 2026-07-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: J de Curtò, Victoria Guillén, I. de Zarzà

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master architect who speaks only in plain English. You want to build a complex mechanical part, like a metal plate with holes or a sturdy bracket, but you don't know how to speak the computer's language (code). You just say, "Make me a 150mm plate with six holes in a circle."

This paper is about teaching AI architects (Large Language Models) to listen to your plain English and instantly build the 3D blueprint for that part. The researchers built a testing ground called LLMForge to see which AI is the best at this job.

Here is a breakdown of their experiment using simple analogies:

1. The Challenge: The "Translation" Problem

Usually, turning a spoken idea into a perfect 3D machine part is hard. The AI has to translate your words into strict code. If the code has even one tiny error, the 3D shape might be broken, have holes where it shouldn't, or be the wrong size. It's like asking a translator to write a legal contract; if they miss one word, the whole deal falls apart.

2. The Test: A "Design Gym"

The researchers created a gym with 97 different exercises (design problems). These ranged from simple flat plates with holes to more complex shapes like cylinders and L-brackets. They asked seven different AI models to solve these problems.

3. The Two Coaches (Critique Regimes)

The AI doesn't just guess once; it gets to try, get feedback, and try again. The researchers tested two different types of "coaches" to give that feedback:

Coach A: The "Mathematical Inspector" (IterTracer)
- How it works: This coach is a super-fast calculator. It looks at the AI's drawing and checks the numbers. "Is the hole 5mm wide? Is the edge clear?" It uses strict math and ray-tracing (like a video game camera) to measure the shape.
- The Vibe: It's like a strict geometry teacher who checks your ruler work. It's fast, precise, and never gets tired.
- Result: The top AI models did almost perfectly here. They all scored about the same (around 89%), proving that for standard shapes, the best AIs are already very good at following the math.
Coach B: The "Art Critic" (IterVision)
- How it works: This coach is a Vision-Language Model (an AI that can see and think). Instead of just checking numbers, it looks at the 3D image and says, "This looks like a cylinder, but the holes are in the wrong place," or "The overall shape doesn't match your description." It uses "chain-of-thought" reasoning, like a human designer explaining why something looks wrong.
- The Vibe: It's like a senior engineer who looks at the blueprint and says, "This feels off," even if the numbers look okay.
- Result: This coach was harder to please. It lowered the scores slightly because it caught subtle mistakes the math coach missed. However, it helped one specific AI (Gemma-3-27B) achieve a perfect 100% success rate, creating flawless, watertight 3D shapes.

4. The "Cylinder" Problem

The researchers noticed something funny: The AIs struggled the most with cylinders (round shapes).

Why? The "Mathematical Inspector" relies on seeing edges and flat surfaces to measure things. A cylinder is round and smooth, so it's harder for the math to find the "edges" to check.
The Analogy: It's like trying to measure a ball with a ruler meant for a box. The AI gets confused because the visual clues it usually relies on are missing.

5. The "Iterative" Magic (Learning from Mistakes)

The study showed that letting the AI try, fail, and try again is crucial.

Round 1: The AI makes a first guess.
Round 2 & 3: The Coach points out errors ("You missed a hole," "The edge is too thin"). The AI fixes it.
The Finding: For the best AIs, the first guess was already great. But for the weaker AIs, the feedback loop was a lifesaver, turning broken designs into working ones.

6. The Big Takeaway

The "Big Four": Four of the AI models (DeepSeek, Qwen, Llama, and Gemma) are now so good at this that they are almost indistinguishable when using the strict math coach. They can turn your English words into perfect machine parts with 99% reliability.
The "Secret Sauce": Adding the "Art Critic" (the visual AI) helps catch the tiny, weird mistakes that the math misses, pushing the best models to perfection.
The Limit: The AI is great at simple, standard parts (plates, boxes, brackets). It still struggles a bit with complex round shapes (cylinders) and very fancy, free-form designs.

In short: We have reached a point where AI can reliably turn your spoken ideas into real, usable 3D machine parts, provided you give it a chance to check its work and correct its mistakes. The best AI models are now ready to be used as a "first draft" tool for engineers.

Technical Summary: Foundation Models for Automatic CAD Generation

Problem Statement

The automation of mechanical design remains a significant challenge in engineering informatics. Traditional Computer-Aided Design (CAD) workflows require specialized knowledge of constraint-based parametric modeling and scripting, creating a barrier for non-specialists and hindering integration into automated processing chains. While Large Language Models (LLMs) have demonstrated the ability to generate structured code from natural language, a critical gap exists between generating syntactically valid code and producing semantically correct, manufacturable 3D geometries. A script may parse without error yet yield a part with incorrect dimensions, missing features, or topological defects that prevent downstream meshing or simulation. This paper addresses the need for a robust evaluation framework to assess the capability of foundation models to translate natural-language specifications into valid parametric CAD designs, specifically for mechanical parts.

Methodology

The authors introduce LLMForge, a multi-model text-to-CAD framework designed to evaluate and refine the output of foundation models through a unified pipeline and a curated benchmark.

Benchmark Dataset

The study utilizes a benchmark of 97 engineering design problems drawn from four canonical geometry families:

Rectangular plates with holes and bolt circles.
Multi-feature boxes (hollow and solid).
Flanged cylinders.
L-brackets.

Each problem is paired with a ground-truth feature specification (hole count, dimensions, surface treatments) used to compute analytic adherence scores. Models are prompted to output a strictly structured JSON object representing the CAD parameters rather than free-form code.

Evaluation Pipelines

The framework evaluates seven state-of-the-art foundation models (DeepSeek-V3.2, Qwen3-235B-A22B, Llama-3.3-70B, Gemma-3-27B, GLM-4.5, MiniMax-M2.1, and INTELLECT) under two distinct critique regimes, both employing an iterative refinement loop (up to 4 rounds):

IterTracer (Analytic Regime):
- Mechanism: Uses a deterministic, differentiable geometric analysis of rendered bitmaps.
- Process: A Phong-shaded ray-trace renderer generates orthographic views. An analytic scorer computes five sub-signals: silhouette IoU, hole visibility, edge clearance, aspect-ratio conformance, and cross-sectional consistency.
- Feedback: These metrics are converted into a scalar critique and fed back into the model for refinement. This regime is lightweight, fast, and introduces no stochastic variance.
IterVision (Semantic Regime):
- Mechanism: Augments the analytic scorer with a Vision-Language Model (VLM) based semantic critic (Qwen2.5-VL-72B).
- Process: The VLM evaluates rendered views of candidate geometries using chain-of-thought reasoning to assess spatial coherence and design intent.
- Feedback: The VLM provides a semantic match score ( $s_{vlm}$ ) and structured textual feedback (missing features, geometry issues), which is incorporated into the refinement prompt alongside analytic metrics.

Scoring Axes

Generated parts are evaluated along independent axes:

Validation ( $s_{val}$ ): JSON schema conformance.
Mesh ( $s_{mesh}$ ): Binary indicator of successful watertight mesh generation.
Feature ( $s_{feat}$ ): Analytic comparison against ground-truth specifications.
Visual ( $s_{vis}$ ): Composite of the five analytic visual sub-signals.
VLM Semantic ( $s_{vlm}$ ): (IterVision only) Semantic match score from the VLM critic.

Key Contributions

IterTracer Pipeline: An iterative text-to-CAD generation system utilizing analytic visual metrics for lightweight, geometry-aware feedback without secondary neural models.
IterVision Pipeline: An extension integrating a VLM-based semantic critic to evaluate spatial coherence and design intent, introducing a new dimension to the evaluation record.
Comprehensive Benchmark: A dataset of 97 engineering problems across four geometry families, paired with ground-truth specifications for analytic scoring.
Empirical Evaluation: A comparative study of seven foundation models across both pipelines, providing per-round convergence trajectories and full mesh-quality statistics.

Experimental Results

Top-Tier Performance (IterTracer): Four models (DeepSeek-V3.2, Qwen3-235B-A22B, Llama-3.3-70B, Gemma-3-27B) formed a tight performance cluster with overall scores in the range $[0.885, 0.890]$ and a mesh success rate of 98.97%. This suggests the task has reached a saturation point for top-tier instruction-tuned models under analytic critique.
Impact of VLM Critique (IterVision): The addition of the VLM axis uniformly reduced overall scores by approximately 0.04 points for the top cluster, indicating a more demanding semantic assessment. However, Gemma-3-27B achieved a 100% mesh success rate (97/97) with all meshes being watertight and topologically valid.
Model Differentiation: The VLM axis revealed performance differences not visible in analytic metrics. For instance, DeepSeek-V3.2 led in semantic match ( $s_{vlm}$ ), while Gemma-3-27B scored lowest among the top four on this specific axis despite perfect mesh generation.
Failure Modes: Lower-tier models exhibited distinct failure patterns. GLM-4.5 showed high variance with intermittent schema errors, while MiniMax-M2.1 and INTELLECT suffered from fundamental geometric failures (e.g., malformed parameters causing mesh generation exceptions). Notably, INTELLECT displayed a "visual-geometric decoupling," where its rendered outputs received high VLM scores despite failing mesh validation.
Geometry Difficulty: Cylindrical geometries consistently scored 0.04–0.07 points lower than plates and L-brackets, attributed to the lack of planar hole features (reducing visual discriminators) and potential under-representation of rotationally symmetric parts in VLM training data.
Refinement Dynamics: Under IterTracer, top models peaked at round 1 and declined, suggesting analytic feedback corrects discrete errors in a single pass. Under IterVision, models continued to improve through rounds 2 and 3, indicating that semantic feedback provides a continuous quality gradient supporting deeper search.

Significance and Claims

The paper claims that compact instruction-tuned models can attain reliability competitive with substantially larger systems for parametric CAD generation of canonical geometries. The study demonstrates that:

Analytic vs. Semantic Feedback: Analytic feedback is highly effective for rapid constraint satisfaction (schema and mesh validity) but saturates quickly. Semantic visual feedback (VLM) is necessary to uncover residual variations in design intent and spatial coherence that analytic metrics miss.
Industrial Workflow Integration: The structured JSON representation and multi-round critique protocol are amenable to integration into engineering Product Data Management (PDM) workflows. The authors suggest a two-stage strategy: using analytic critique for rapid generation and selective VLM invocation for high-value or ambiguous parts to balance token expenditure and quality.
Benchmark Saturation: The near-identical performance of top models under analytic critique suggests that current benchmarks may be insufficient to differentiate state-of-the-art models, necessitating the inclusion of semantic evaluation axes.

The authors conclude that while the current scope is bounded to canonical geometries, the framework provides a foundation for future research into scalable, automated mechanical design, including freeform surfaces and manufacturing-specific constraints.