💻 computer science

PLLM: Pseudo-Labeling Large Language Models for CAD Program Synthesis

The paper introduces PLLM, a self-training framework that leverages pseudo-labeling to fine-tune large language models for synthesizing CAD programs from unlabeled 3D geometries, thereby overcoming the scarcity of paired shape-program data and improving both geometric fidelity and program diversity.

Original authors: Yuanbo Li, Dule Shu, Yanying Chen, Matt Klenk, Daniel Ritchie

Published 2026-02-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuanbo Li, Dule Shu, Yanying Chen, Matt Klenk, Daniel Ritchie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive library of 3D objects—chairs, cars, buildings—but you only have the finished products (the shapes). You don't have the blueprints (the computer code) that engineers used to build them.

In the world of Computer-Aided Design (CAD), having the blueprint is a superpower. It means you can easily resize a chair, change its material, or tweak a curve. Without the blueprint, you only have a static statue; you can't really "edit" it easily.

The problem? Blueprints are hard to find. Most 3D models in the wild don't come with their original code.

This paper introduces a clever solution called PLLM (Pseudo-Labeling Large Language Models). Think of it as a "Self-Taught Architect" that learns to reverse-engineer blueprints just by looking at the finished buildings.

Here is how it works, broken down into simple steps with some analogies:

1. The Starting Point: A Smart but Unpracticed Intern

Imagine you hire a very smart architecture intern (a pre-trained AI model). This intern has read millions of blueprints from a specific style of building (let's call it "DeepCAD"). They know the rules of construction well.

However, you now show them a pile of buildings from a completely different city ("ABC dataset") that they've never seen before. They try to guess the blueprints, but because the style is different, their first guesses are messy, inefficient, or slightly wrong.

2. The "Try, Test, and Keep" Loop

Instead of giving up, the intern starts a self-improvement cycle. Here is the process:

Step A: The Guessing Game (Sampling)
The intern looks at one 3D shape and tries to write ten different blueprints for it. Some are terrible, some are okay, and one might be close.
- Analogy: It's like a chef trying to guess a secret recipe by tasting the dish and writing down ten different ingredient lists.
Step B: The Reality Check (Execution)
The system takes those ten blueprints and actually tries to "build" them in the computer. It compares the new 3D model built from the blueprint against the original shape.
- Analogy: The chef cooks the dish based on their guessed recipe and compares it to the original. If the taste is off, the recipe is wrong.
Step C: Keeping the Best (Selection)
The system picks the blueprint that built the closest match. It throws away the bad ones. This "best guess" becomes a fake blueprint (a "pseudo-label") that is now treated as a correct answer for training.

3. The Secret Sauce: "Editing the Drafts" (Data Augmentation)

This is where the paper gets really creative. Just picking the best blueprint isn't enough; the AI might get lazy and only write short, simple blueprints.

So, the system takes the "winning" blueprint and manually edits it:

Expansion: It adds extra steps to make the blueprint more complex (e.g., "Add a second window here").
Shortening: It removes unnecessary steps to make it concise (e.g., "Combine these two cuts into one").
Analogy: Imagine the intern writes a draft story. A senior editor then takes that draft, writes a longer, more detailed version, and also writes a shorter, punchier version. Now, the intern has three different versions of the story to learn from, all based on the same original plot.

This creates a huge library of "fake but accurate" blueprints paired with the shapes they build.

4. The Final Result: A Master Architect

The AI is then retrained (fine-tuned) on this new, massive library of self-made blueprints.

Iteration 1: It's okay.
Iteration 2: It gets better because it learned from its own "best guesses."
Iteration 5: It becomes a master, capable of writing high-quality blueprints for shapes it has never seen before, even though it was never taught by a human with the correct answers.

Why is this a big deal?

Usually, to teach an AI to do this, you need a human to sit down and write the code for thousands of 3D shapes. That is expensive and slow.

PLLM says: "We don't need humans to write the code. We can use the AI to write its own homework, check its own work, and then learn from the results."

The Catch

The only downside is time and computing power. It takes a lot of computer energy to "build" these 3D models over and over again to check if the blueprints are right. But the authors argue that this cost is worth it to unlock the ability to learn from the millions of 3D shapes that currently have no blueprints.

In a nutshell: The paper teaches an AI to teach itself how to reverse-engineer 3D designs by generating its own practice problems, grading them, and then studying the "correct" answers it created for itself.

1. Problem Statement

The core challenge addressed is CAD program synthesis from 3D geometries (the inverse problem of reverse engineering). While Large Language Models (LLMs) have shown promise in generating CAD code (e.g., CadQuery), existing methods rely heavily on supervised training using paired datasets of 3D shapes and their corresponding ground-truth CAD programs.

The Bottleneck: High-quality paired data is expensive to create, domain-specific, and often unavailable for large-scale repositories (e.g., the ABC dataset contains shapes but lacks program annotations).
The Goal: Develop a method to adapt pre-trained CAD-capable LLMs to new, unlabeled shape distributions without requiring manual program annotations.

2. Methodology: The PLLM Framework

The authors propose PLLM (Pseudo-Labeling Large Language Models), a self-training framework that iteratively constructs synthetic supervision from unlabeled 3D shapes. The process consists of four key stages per iteration:

A. Program Sampling

Input: A pre-trained LLM (specifically CAD-Recode, trained on DeepCAD) and a target dataset of unlabeled shapes ( $S^*$ ).
Process: For each input shape $x^*$ , the model samples $k=10$ candidate CAD programs ( $z_i$ ) using stochastic decoding (nucleus sampling with $top-p=0.8$, $top-k=30$, temperature $1.2$).
Selection: Each candidate program is executed via a black-box executor (CadQuery) to generate a 3D geometry. The resulting geometry is compared to the input shape using Chamfer Distance (CD). The program with the lowest CD is selected as the "best" candidate. If multiple candidates yield similar geometry, the shorter program is preferred for conciseness.

B. Execution-Based Curation

The selected high-fidelity programs are treated as synthetic supervision.
The framework filters out low-quality generations, ensuring that only programs producing geometrically accurate reconstructions are retained for training.

C. Program-Level Data Augmentation

To prevent the model from overfitting to a narrow distribution of program structures and to handle varying procedural complexities, the framework applies two types of programmatic edits to the selected "best" programs:

Program Expansion: Appending additional operations or spawning new workspaces (capped at 5) to increase structural complexity.
Program Shortening: Removing top-level Boolean operations (union, cut, intersect) while maintaining syntactic validity to create concise variants.

Result: These edits generate diverse $(X, Z)$ pairs where $X$ is the execution result and $Z$ is the program. This ensures perfect consistency between the label (program) and the input (shape), eliminating label noise.

D. Iterative Self-Training

The LLM is fine-tuned using LoRA (Low-Rank Adaptation) on the expanded synthetic dataset.
The updated model is then used in the next iteration to sample new candidates, creating a self-reinforcing cycle where the model progressively aligns with the target distribution $S^*$ .

3. Key Contributions

PLLM Framework: A novel self-training pipeline that synthesizes and enriches CAD programs to create supervision for unlabeled 3D datasets, bypassing the need for ground-truth annotations.
Data Synthesis Strategy: A unique approach that not only selects high-quality generations but also expands and refines them through programmatic edits (expansion/shortening). This creates a diverse dataset covering a broader range of program lengths and structural complexities.
Empirical Validation: Demonstration that synthetic self-training significantly improves geometric fidelity and program diversity when adapting a model from the DeepCAD domain to the unlabeled ABC dataset.

4. Results and Evaluation

The authors evaluated PLLM by adapting CAD-Recode to the ABC dataset (unlabeled) over 6 iterations.

Geometric Fidelity (Chamfer Distance):
- The average Chamfer Distance decreased consistently over the first four iterations.
- Final Performance: PLLM achieved an average CD of 9.73, significantly outperforming the baseline CAD-Recode (26.12) and other pseudo-labeling baselines (e.g., 28.24 for training on generated-execution pairs without augmentation).
Program Diversity:
- The framework successfully increased the maximum program length from the baseline limit (768 tokens) to 1200+ tokens, allowing the model to represent more detailed geometries.
- The synthetic dataset covered a wider spectrum of program lengths and complexities compared to baselines.
Ablation Studies (Baseline Comparisons):
- Baseline 1 (Training on best sample + its execution) performed poorly (CD 28.24) because the data drifted from the target shape distribution.
- Baseline 2 (Training on best sample + original input shape) introduced noise (CD 10.28) because the generated program didn't perfectly reconstruct the input.
- PLLM outperformed all baselines by ensuring the training pairs were perfectly consistent (Program $\to$ Execution $\to$ Shape) while introducing structural variations.

5. Significance and Limitations

Significance:
- Scalability: PLLM enables the scaling of CAD program synthesis to massive, unlabeled repositories (like the ABC dataset) where manual annotation is impossible.
- Domain Adaptation: It demonstrates that pre-trained LLMs can be effectively adapted to new domains and languages (e.g., from DeepCAD to ABC) without ground-truth labels.
- Data-Centric AI: It highlights the potential of using execution feedback to curate and enrich synthetic data, a strategy applicable beyond CAD to other symbolic program synthesis tasks.
Limitations:
- Computational Cost: The iterative process of sampling, executing, and filtering programs is computationally expensive (approx. 150 hours for 6 iterations on 4 GPUs).
- Model Constraints: Improvements plateaued after iteration 4, attributed to the limited operation set of the base model (CAD-Recode), suggesting that the ceiling of performance is tied to the underlying model's capabilities.

In conclusion, PLLM represents a significant step forward in unsupervised CAD reverse engineering, proving that high-quality synthetic supervision can be generated automatically to train LLMs for complex program synthesis tasks.