Technical Summary: Computer-Aided Design Generation by Cascaded Discrete Diffusion Model

1. Problem Statement

Computer-Aided Design (CAD) is a cornerstone of modern engineering, yet its creation remains heavily reliant on labor-intensive, manual workflows prone to human error. While recent deep learning approaches have sought to automate CAD generation by representing models as sequences of discrete commands and parameters, existing methods face a fundamental limitation.

Current state-of-the-art methods, such as autoregressive models or continuous diffusion models (e.g., Diffusion-CAD), often embed discrete CAD tokens into a continuous Euclidean space. The authors argue that this approach is fundamentally misaligned with the nature of CAD data. CAD tokens are inherently discrete and heterogeneous; projecting them into a continuous domain fails to preserve semantic neighborhood relationships. Consequently, the isotropic Gaussian noise used in continuous diffusion perturbs representations into regions that map to semantically invalid or meaningless CAD symbols, undermining the geometric validity of the generated designs.

2. Methodology

To address these limitations, the paper proposes a Cascaded Discrete Diffusion Model (CDDM). This framework operates directly on categorical token distributions rather than continuous embeddings, ensuring semantic consistency throughout the generation process. The architecture is divided into two sequential stages:

A. Cascaded Architecture

The generation task is factorized into two conditional probabilities: $p(M) = p(\zeta) p(\theta | \zeta)$, where $\zeta$ represents the sequence of discrete CAD commands and $\theta$ represents the associated parameters.

1. Command Diffusion: Generates the sequence of discrete commands (e.g., Line, Circle, Extrude).
2. Parameter Diffusion: Generates the quantized parameters conditioned on the generated commands.

B. Forward Process: Symbol-Aware Transition Matrices

Unlike standard diffusion that applies uniform noise, the CDDM employs a Markov chain with carefully designed transition matrices ($Q_t$) tailored to the specific nature of the data:

• For Commands: An absorbing-state transition matrix is used. This progressively corrupts tokens toward a designated absorbing symbol, effectively erasing semantic content in a manner consistent with the sequential nature of commands.
• For Parameters: Recognizing the heterogeneity of CAD parameters, the authors design three specific transition kernels:
  • Coordinate Parameters: A Gaussian-based kernel is used to capture local spatial continuity (e.g., ensuring token 100 transitions more likely to 99 or 101 than to 1).
  • Dimensional Parameters: A scale-invariant kernel is employed. This measures perturbation relatively (e.g., $|i-j|/(i+j)$) rather than absolutely, ensuring that a 1% change in a large dimension is treated similarly to a 1% change in a small dimension.
  • Boolean Parameters: A prior-preserving kernel restricts transitions to valid categories based on empirical training statistics, preventing diffusion into invalid states.

C. Reverse Process: Denoising Networks

The recovery of clean data from corrupted states is handled by two distinct networks:

1. Command Denoiser: A Transformer encoder equipped with multi-head self-attention to model global dependencies among command tokens.
2. Parameter Denoiser: A specialized Transformer architecture that includes:
   • Global Self-Attention: For long-range dependencies.
   • Local Self-Attention: A core innovation where a command-level attention mask suppresses interactions between parameters belonging to different commands, ensuring parameters only correlate with their specific command.
   • Cross-Attention: Injects the generated command sequence as a condition to guide parameter generation.

D. Conditional Generation

The framework supports conditional generation (e.g., from point clouds or specific command lengths) by introducing additional cross-attention layers that fuse conditional features (from PointNet or length indicators) with the command and parameter features.

3. Key Contributions

The paper outlines three primary contributions:

1. First Cascaded Discrete Diffusion Framework for CAD: The authors introduce a model that directly operates on categorical tokens without continuous embedding, preserving semantic consistency and avoiding the invalid geometries common in continuous diffusion.
2. Symbol-Aware Transition Mechanisms: The design of specific transition matrices for heterogeneous parameters (Gaussian for coordinates, scale-invariant for dimensions, and prior-preserving for booleans) enables controllable, token-valid corruption processes impossible in continuous spaces.
3. Command-Aware Parameter Denoising Network: The proposal of a network architecture featuring local self-attention (to restrict parameter interactions within commands) and cross-conditioning, which explicitly models the structural relationship between commands and their parameters.

4. Experimental Results

Experiments were conducted on the DeepCAD dataset (over 178,000 models).

• Unconditional Generation: The CDDM outperformed existing autoregressive models (DeepCAD, SkexGen) and continuous diffusion baselines (Diffusion-CAD) across multiple metrics. Notably, it achieved a Coverage (COV) of 89.10% (vs. 79.11% for Diffusion-CAD) and an Invalidity rate of 4.0% (vs. 4.1% for Diffusion-CAD and significantly lower than B-rep methods which ranged from 20-30%).
• Ablation Studies:
  • Cascaded vs. Unified: A unified diffusion model (treating commands and parameters as a single sequence) resulted in an invalidity rate exceeding 90%, confirming the necessity of the cascaded approach.
  • Discrete vs. Continuous: Discrete diffusion significantly outperformed continuous diffusion for parameter generation, validating the claim that continuous noise disrupts discrete token semantics.
  • Transition Matrices: Using the proposed mixed transition matrices (Gaussian, scale-invariant, prior-preserving) yielded superior results compared to using a uniform transition matrix for all parameters.
  • Attention Mechanisms: The combination of global and local self-attention layers in the parameter network outperformed using either layer alone.
• Conditional Generation: Qualitative results demonstrated effective controllability, successfully generating CAD models with specific command lengths and reconstructing valid CAD sequences from point cloud inputs.

5. Significance and Claims

The paper claims that the proposed CDDM represents a significant advancement in automated CAD generation by respecting the inherent discrete and heterogeneous nature of design data. By moving away from continuous embeddings, the method avoids the semantic inconsistencies that plague previous diffusion approaches.

The authors position the work as the first to successfully apply a cascaded discrete diffusion framework to this domain. They assert that their symbol-aware transition mechanisms and command-aware attention structures provide a robust solution for generating geometrically valid and diverse CAD models, surpassing the state-of-the-art in both unconditional generation quality and conditional controllability. The source code is intended to be released to facilitate further research.