Text-to-CAD Retrieval: a Strong Baseline

Technical Summary: Text-to-CAD Retrieval: a Strong Baseline

Problem Definition
The paper addresses the critical yet underexplored challenge of retrieving legacy Computer-Aided Design (CAD) models using natural language queries. Traditional industrial CAD repositories rely on filenames, directory structures, or manual memory for search, which limits scalability and accuracy. The authors formalize text-to-CAD retrieval as a new cross-modal retrieval task. The objective is to map textual descriptions and CAD models into a shared embedding space where semantically relevant pairs are close, and mismatched pairs are distant. A primary challenge identified is representing CAD models in a format amenable to learning; while procedural sequences (commands and parameters) capture construction logic, they often lack explicit geometric characteristics crucial for semantic matching.

Methodology
The authors propose a unified framework that leverages multi-modal representations of CAD models, combining procedural sequences and geometric point clouds. The architecture consists of three encoders and a training-time auxiliary decoder:

1. Encoders:

   • Text Encoder ($\phi_T$): Utilizes a pretrained language tokenizer (e.g., CLIP or BERT) followed by a trainable Transformer encoder to generate semantic representations.
   • Sequence Encoder ($\phi_S$): A Transformer-based stack that processes ordered sequences of 2D geometric primitives and modeling commands.
   • Point Encoder ($\phi_P$): Adapts existing point-based networks (e.g., PointNet or PointTransformer) to extract explicit geometric features from point clouds sampled from CAD entities.

2. Training Strategy (Implicit Alignment):

   • Retrieval Loss: The model employs a bidirectional InfoNCE loss to supervise the text-sequence and text-point pairs independently, pushing non-matching pairs apart in the embedding space.
   • Feature Decoder ($\psi_S$): A novel auxiliary module introduced solely for training. It reconstructs masked sequence features ($\hat{F}^S$) using cross-attention, where the masked sequence acts as the query, and text and point features alternately serve as keys and values. This forces the text and point encoders to learn representations that are complementary and aligned with the sequence modality without requiring explicit positive-negative contrast between all three modalities.
   • Loss Function: The total objective combines the retrieval loss and a Mean Squared Error (MSE) reconstruction loss ($\mathcal{L} = \mathcal{L}_{ret} + \lambda \mathcal{L}_{rec}$).

3. Inference:

   • The feature decoder is discarded during inference.
   • The final CAD representation is formed by concatenating the pooled sequence and point features ($f^M = [f^S; f^P]$).
   • The text feature is duplicated to match the dimensionality of the CAD embedding ($f^T = [f^T; f^T]$).
   • Retrieval is performed by computing cosine similarity between the concatenated CAD features and the duplicated text features.

Key Contributions

1. Task Formalization: The paper formally introduces text-to-CAD retrieval as a distinct cross-modal retrieval task, establishing a practical benchmark using paired text-CAD annotations from existing generation datasets (Text2CAD and CadTranslator).
2. Strong Baseline Framework: The authors propose a multi-modal framework that learns from both procedural sequences and geometric point clouds. It incorporates a novel feature decoder to achieve implicit multi-modal alignment during training, serving as a strong baseline for the field.
3. Empirical Validation: Quantitative analyses demonstrate the approach's superiority over existing text-to-3D shape retrieval models. Qualitative results confirm the method's sensitivity to specific geometric descriptions in text queries.

Experimental Results
The model was evaluated on two datasets: Text2CAD and CadTranslator, both derived from the DeepCAD repository.

• Performance: On the Text2CAD dataset, the proposed method (IMMA) achieved a Rank-1 (R1) accuracy of 9.71%, a significant improvement over using sequence features alone (5.00%) and outperforming state-of-the-art methods like SCA3D (6.33%) and Parts2Words (4.81%). On CadTranslator, it achieved an R1 of 8.02%.
• Ablation Studies:
  • Multi-modal Fusion: Combining sequence and point features significantly outperformed single-modality variants.
  • Feature Decoder: The inclusion of the reconstruction decoder yielded substantial gains, particularly on Text2CAD (improving R1 from 6.99% to 9.71%).
  • Hyperparameters: The model showed robustness to the reconstruction loss weight ($\lambda$) and the masking ratio ($r$), with optimal performance observed at specific configurations (e.g., $r=0.5$ for Text2CAD).
• Comparison: The proposed method outperformed adapted versions of SOTA text-to-3D retrieval models (Parts2Words, TriCoLo, SCA3D, RoMa) and various multi-modal fusion strategies (Concat & Linear, Self-Attention, Cross-Attention, Modulation), suggesting that simple concatenation with implicit alignment is more effective than complex fusion for this specific modality gap.

Significance
The paper positions this work as a foundational step for industrial legacy design reuse, enabling engineers to query large-scale CAD repositories using natural language. Furthermore, the authors claim the framework lays the groundwork for downstream CAD generation paradigms, specifically Retrieval-Augmented Generation (RAG), by establishing a reliable mechanism for retrieving semantically relevant CAD models based on text. The source code is intended for release to facilitate further research.