CADSmith: Multi-Agent CAD Generation with Programmatic Geometric Validation

CADSmith is a multi-agent system that generates CadQuery code from natural language and employs an iterative refinement process combining programmatic geometric validation and vision-language model assessment to significantly improve the accuracy and reliability of text-to-CAD generation.

The Gist

Imagine you want to build a custom piece of furniture, like a complex bookshelf with specific dimensions, curved shelves, and precise holes for bolts. You ask a very smart, but slightly scatterbrained, AI assistant to write the instructions for a robot to build it.

In the past, if you asked an AI to do this, it would try to write the instructions once, hand them to the robot, and hope for the best.

• The Problem: The AI often gets the math wrong (the shelf is 2 inches too short) or the logic wrong (the robot tries to glue a piece that doesn't exist). The robot might crash, or build something that looks okay from a distance but falls apart if you touch it.
• The Old Way: You'd look at the broken shelf, say "Oops," and ask the AI to try again from scratch. It's slow, and the AI keeps making the same mistakes because it doesn't "know" exactly why it failed.

CADSmith is like hiring a specialized construction team instead of a single assistant. They work together in a loop to ensure the final product is perfect. Here is how they do it, using a simple analogy:

The Team (The Multi-Agent Pipeline)

Instead of one person doing everything, CADSmith splits the job into five specialized roles:

1. The Architect (Planner): You tell the team, "I need a bookshelf." The Architect doesn't build anything; they just break your request down into a clear, step-by-step blueprint. They make sure everyone understands the exact measurements (e.g., "The shelf must be exactly 50mm wide").
2. The Draftsman (Coder): This person writes the actual code (the instructions) for the robot. But they don't just guess! They have a library of manuals (Retrieval-Augmented Generation) right next to them. If they need to know how to cut a specific curve, they look it up in the manual instead of making it up. This stops them from "hallucinating" fake tools.
3. The Foreman (Executor): This is the robot that tries to run the instructions.
   • Inner Loop: If the robot trips over its own feet (a coding error), the Foreman stops, reads the error message, and asks the Fixer to rewrite the instructions. They try this up to three times until the robot can actually start moving.
4. The Inspector (Validator): Once the robot builds the shelf, the Inspector checks it. This is the magic part. The Inspector has two superpowers:
   • The Laser Tape Measure (Programmatic Validation): It uses a super-precise digital ruler (OpenCASCADE) to measure the shelf. Is it exactly 50mm? Is the volume correct? Is it watertight (no holes)? This gives exact numbers.
   • The Human Eye (Vision Judge): It also looks at 3D photos of the shelf from three different angles. Sometimes, the numbers look right, but the shape looks weird (like a shelf that is the right size but has the wrong number of holes). The "Human Eye" catches these visual mistakes that a ruler might miss.
5. The Fixer (Refiner): If the Inspector finds a problem (e.g., "The shelf is 2mm too short" or "There are only 4 holes instead of 6"), they don't just say "Try again." They give the Draftsman a specific note: "The laser says you are 2mm short; the photo shows the holes are in the wrong place." The Draftsman then fixes the code, and the Foreman tries again.

The "Double-Loop" Safety Net

The system has two safety nets working together:

• The Inner Loop: Fixes "Can I run this code?" errors. (e.g., "You used a tool that doesn't exist.")
• The Outer Loop: Fixes "Is this the right object?" errors. (e.g., "You built a chair, but I asked for a table, even though the code ran fine.")

Why This Matters (The Results)

The researchers tested this team against a "solo" AI (the old way) using 100 different design challenges, from simple blocks to complex engine parts.

• The Solo AI: Failed to build anything 5% of the time. When it did build something, it was often slightly off (like a shelf that wobbles).
• The CADSmith Team:
  • 100% Success Rate: They never failed to produce a buildable object.
  • Precision: The difference between the requested size and the built size dropped from being "off by a lot" to being "off by less than a millimeter."
  • The "Eye" Factor: When they removed the "Human Eye" (the photos) and only used the "Laser Tape Measure," the team got confused on complex shapes. The numbers looked right, but the shape was wrong. The photos were essential to catch those subtle mistakes.

The Bottom Line

CADSmith is like upgrading from asking a single person to build a house in the dark, to hiring a team with a blueprint, a library of building codes, a robot foreman, and an inspector with both a laser measure and a keen eye.

By combining exact math (the laser) with visual common sense (the eye) and letting them argue with the builder until the job is perfect, CADSmith turns "maybe it works" into "it definitely works." This is a huge step forward for letting AI help engineers and makers create real, usable 3D parts without needing to be coding experts themselves.

Technical Summary

1. Problem Statement

Current methods for Text-to-CAD generation face a critical trade-off between generative fluency and geometric precision.

• Limitations of Existing Approaches:
  • Single-pass generation: Models like Text-to-CadQuery generate code in one shot without iterative correction, leading to unverified outputs.
  • Lossy Visual Feedback: Systems like Query2CAD use visual feedback (e.g., BLIP-2) for refinement, but encoded visual representations lack the millimeter-scale precision required to resolve dimensional errors.
  • Hallucinations: Large Language Models (LLMs) frequently hallucinate dimensions and misuse CAD APIs, producing parts that look visually plausible but fail strict engineering constraints (e.g., incorrect bounding boxes, missing faces, or non-manifold geometry).
• The Gap: There is a lack of systems that combine exact programmatic geometric measurements (kernel-level) with holistic visual assessment in a closed-loop refinement process to ensure both dimensional accuracy and topological validity.

2. Methodology: The CADSmith Pipeline

CADSmith is a multi-agent system designed to generate executable CadQuery code from natural language prompts. It employs a hierarchical decomposition of tasks across five specialized agents and utilizes two nested correction loops.

A. Agent Architecture

1. Planner: Converts natural language prompts into a structured design specification (JSON), extracting component lists, target dimensions, and geometric constraints.
2. Coder: Generates the CadQuery Python script. Instead of fine-tuning, it uses Retrieval-Augmented Generation (RAG) over two knowledge bases:
   • KB1 (API Docs): CadQuery method signatures, usage examples, and pitfalls.
   • KB2 (Error Patterns): Common failure modes (e.g., boolean errors, fillet violations) and their fixes.
3. Executor: Runs the code in a sandboxed Python subprocess. It extracts exact geometric metrics from the OpenCASCADE (OCCT) kernel (volume, bounding box, face/edge/vertex counts, solid validity) and exports STEP/STL files.
4. Validator (The Judge): An independent Vision-Language Model (VLM) using Claude Opus (stronger than the code generator, Claude Sonnet) to evaluate the output.
   • Inputs: Original prompt, generated code, OCCT kernel metrics, and three-view rendered images (Isometric, High-angle rear, Front profile).
   • Role: Cross-references visual evidence with numerical metrics to detect "false convergence" (where numbers look right but the shape is wrong).
5. Refiner: Receives feedback from the Validator and modifies the code to correct specific errors.

B. The Nested Correction Loops

• Inner Loop (Execution Errors): If the code fails to run (syntax/API errors), the Error Refiner uses RAG to diagnose the traceback and retry up to 3 times.
• Outer Loop (Geometric Refinement): If the code runs but the geometry is incorrect, the Validator provides structured feedback (numerical discrepancies + visual assessment). The Refiner iterates up to 5 times to converge on the correct geometry.

3. Key Contributions

1. Multi-Agent Architecture: Replaces monolithic single-model approaches with a decomposed pipeline (Planner, Coder, Executor, Validator, Refiner) for specialized task handling.
2. Dual-Loop Programmatic Validation:
   • Combines exact kernel measurements (OCCT) for dimensional precision with VLM-based visual inspection for global shape coherence.
   • This dual approach prevents the system from satisfying numerical tolerances while producing structurally malformed parts.
3. Independent VLM-as-Judge: Uses a stronger model (Claude Opus) as the Judge to avoid the self-confirmation bias inherent in single-model self-refinement.
4. RAG over API Documentation: Eliminates the need for fine-tuning on static datasets, allowing the system to adapt immediately as the CadQuery library evolves.
5. New Benchmark: A custom dataset of 100 prompts across three difficulty tiers (T1: Primitives, T2: Engineering Parts, T3: Complex Multi-feature parts) with hand-written reference scripts.

4. Results

The system was evaluated on 100 prompts using Chamfer Distance (CD), F1 Score, and Volumetric IoU in absolute millimeter space.

• Overall Performance:
  • Execution Rate: Achieved 100% (vs. 95% for zero-shot baseline).
  • Chamfer Distance: Reduced mean CD from 28.37 (zero-shot) to 0.74 (full pipeline).
  • F1 Score: Improved median F1 from 0.9707 to 0.9846.
  • IoU: Improved median IoU from 0.8085 to 0.9629.
• Ablation Studies:
  • No-Vision Ablation: Removing the three-view images caused a massive degradation in complex parts (T3), with mean CD jumping from 1.42 to 49.68. This proves that kernel metrics alone cannot detect structural errors (e.g., missing features that don't drastically change volume).
  • Zero-Shot Baseline: Failed significantly on complex geometries, often producing code that executed but generated fundamentally wrong shapes.
• Tier Breakdown:
  • T1 and T2 parts achieved near-perfect scores.
  • T3 (Complex) parts achieved a median F1 of 0.886, demonstrating the system's ability to handle multi-step constructions (lofts, sweeps, shells).

5. Significance and Future Work

• Reliability: CADSmith demonstrates that closed-loop refinement driven by programmatic geometric feedback is essential for reliable automated CAD generation. It bridges the gap between "plausible-looking" code and "manufacturable" geometry.
• Manufacturing Impact: By ensuring dimensional exactness and topological validity, the system lowers the barrier to entry for engineering design and supports the goal of bringing manufacturing capabilities back to the US.
• Limitations & Future Directions:
  • Visual Resolution: The system can miss subtle gaps (e.g., in a quadcopter frame) if they are not visible in fixed 3-view renders.
  • Future Work: Adaptive view selection, higher-resolution crops of joints, and scaling to multi-part assemblies.

In conclusion, CADSmith establishes a new paradigm for text-to-CAD generation, proving that combining LLM code synthesis with rigorous, multi-modal geometric validation yields significantly higher quality and reliability than previous single-pass or purely visual-feedback methods.