Technical Summary: Garment Particles

Problem Statement

Garment design is a complex process spanning two distinct modes: intuitive, high-level creation (e.g., from text or images) and precise, low-level editing of 2D sewing patterns and 3D draped geometries. Existing frameworks typically address only one end of this spectrum. Industry-standard tools (e.g., CLO3D) allow for precise low-level editing but require expert knowledge of the causal relationship between 2D patterns and 3D draping. Conversely, recent generative models can create patterns from high-level inputs but often rely on modality-specific training and lack the ability to perform symmetric, iterative editing across both 2D and 3D domains. Furthermore, existing representations, such as geometry images, exhibit asymmetry where the 3D appearance depends on the 2D panel shape (pixel opacity), making it difficult to optimize 3D objectives directly on the representation. The core challenge is to develop a unified representation and framework that supports both multimodal generation and versatile, training-free editing across 2D patterns and 3D geometries.

Methodology

1. Garment Particles Representation

The authors propose Garment Particles, a novel 5D point-cloud representation that symmetrically encodes both the 2D sewing pattern and its 3D draped geometry.

• Mathematical Formulation: A garment is modeled as a parametric surface $\mathbf{r}: U \to \mathbb{R}^3$, where $U$ is the 2D sewing pattern domain and $\mathbf{r}(U)$ is the 3D geometry. The representation is the graph $\Gamma(\mathbf{r}) = \{(\mathbf{x}, \mathbf{r}(\mathbf{x})) \mid \mathbf{x} \in U\}$.
• Discretization: This graph is discretized into a 5D point cloud $\mathbf{X}_{\mathbf{r}} = \{(\mathbf{x}_{\mathbf{r}}, f_{\mathbf{x}_{\mathbf{r}}})\}$, where each point contains 2D coordinates, 3D coordinates, and a boundary flag ($f$) indicating if the point lies on the panel edge.
• Properties: This representation allows for differentiable projection to either the 2D domain ($\pi_D$) or the 3D image space ($\pi_I$), enabling symmetric interaction between the pattern and the drape.

2. Garment Particles Flow (GPF)

To generate and edit these particles, the authors introduce Garment Particles Flow, a rectified flow-based generative model.

• Architecture: The model uses a Diffusion Transformer (DiT) architecture. It learns a probability flow mapping noise $\mathbf{X}_0$ to garment particles $\mathbf{X}_1$.
• Conditioning: GPF supports multimodal conditioning (text, images, sketches) via cross-attention mechanisms. Text is encoded using CLIP, and images are encoded using a frozen DINOv2 encoder.
• Training: The model is trained on the GarmentCodeData v2 (GCDv2) dataset, learning a prior space over the joint 2D-3D garment space.

3. Particles-to-Pattern Flow (PPF)

Since generated particles are unordered point clouds, a mechanism is needed to recover simulation-ready, vectorized sewing patterns.

• Function: PPF is a conditional flow model that maps generated garment particles $\mathbf{X}$ to a structured sewing pattern $P$.
• Representation: The output pattern is represented as a tensor of ordered panels and edges (cubic Bézier curves or arcs), including stitching flags and boundary conditions.
• Robustness: Unlike deterministic regression, PPF treats pattern recovery as a conditional generation task, making it robust to noise in the input particles.

4. Editing via Diffusion Posterior Sampling (DPS)

A key feature of the framework is the ability to perform training-free editing using Diffusion Posterior Sampling (DPS).

• Mechanism: Instead of retraining the model for specific edits, the generation process is guided by an objective function $\mathcal{L}$ during the sampling steps. The model optimizes the posterior mean to minimize the loss between the generated sample's projection and a user observation.
• Applications:
  • Point-Cloud-Conditioned Generation: Guiding generation to match a specific 3D point cloud (using Earth Mover Distance).
  • Garment Completion: Completing partial 3D geometries.
  • Sewing Pattern Editing: Editing the 2D pattern directly and regenerating the 3D drape (using $\pi_D$ projection).
  • Silhouette-Conditioned Generation: Generating garments that match a 2D silhouette from arbitrary viewpoints.

Key Contributions

1. Garment Particles: A symmetric, 5D point-cloud representation that jointly encodes 2D sewing patterns and 3D draped geometry, enabling a fully differentiable pipeline for objective-guided sampling.
2. Garment Particles Flow (GPF): A rectified flow model that generates garment particles from multimodal inputs and achieves state-of-the-art (SOTA) performance in garment generation.
3. Particles-to-Pattern Flow (PPF): A model that converts generated particles into simulation-ready, vectorized sewing patterns.
4. Versatile Editing via DPS: A framework that enables diverse editing scenarios (pattern editing, 3D completion, silhouette conditioning) without retraining, leveraging the symmetric nature of the representation.

Results and Evaluation

Generation Performance

The model was evaluated on the GCDv2 dataset against baselines including AIpparel, ChatGarment, SewingLDM, and Design2GarmentCode.

• Text-Conditioned Generation: GPF achieved SOTA scores in distribution metrics (Coverage, MMD, 1-NNA, p-FID) and CLIP alignment. It demonstrated superior diversity and fidelity compared to baselines, which often produced artifacts like incorrect panel counts or styles.
• Image-Conditioned Generation: Evaluated on GCDv2 renderings, garment sketches, and in-the-wild images. GPF outperformed baselines in panel accuracy, intersection-over-union (IOU), and stitch accuracy, recovering patterns that closely matched ground truth.
• Runtime: The generation time (approx. 4 seconds) is comparable to other diffusion-based methods and significantly faster than LLM-based approaches that require external queries.

Editing Capabilities

The paper demonstrates several editing applications enabled by DPS:

• Interpolation: Smooth transitions between different garment styles (e.g., asymmetric to symmetric sleeves) in the latent space, avoiding the abrupt changes seen in vector-based or geometry-image baselines.
• 3D Editing: Users can add, delete, or translate points in a 3D point cloud to guide the generation of a matching sewing pattern.
• 2D Pattern Editing: Users can modify the 2D sewing pattern (e.g., widening a skirt), and the model regenerates the corresponding 3D drape.
• Silhouette Editing: Generating garments that match user-drawn silhouettes from various camera angles.

Physical Validation

• Fabrication: Two garments (an asymmetric pencil skirt and a turtleneck shirt) generated unconditionally by the pipeline were successfully fabricated by a tailor, validating the physical plausibility of the output.
• Simulation Success Rate (SSR): The model achieved high SSR (approx. 90%), indicating that the generated geometries are physically valid for draping simulation.

Significance and Claims

The paper claims that Garment Particles bridges the gap between intuitive high-level design and professional low-level editing. By unifying 2D patterns and 3D geometry into a single, symmetric representation, the framework allows for:

• Symmetric Editing: The ability to edit in either the 2D or 3D domain and have the changes propagate correctly to the other, a capability lacking in previous modality-specific or asymmetric representations.
• Training-Free Flexibility: The use of DPS allows for a wide range of editing tasks without the need for collecting specific datasets or retraining models for each new editing objective.
• Practical Utility: The pipeline produces simulation-ready sewing patterns, moving beyond purely visual generation to assets usable in downstream physical simulation and manufacturing.

The authors acknowledge limitations, such as the discrete nature of the point cloud making fine-grained adjustments (e.g., dart sizing) difficult, and the current reliance on a single body type and pose. Future work is suggested to extend the model to handle multi-body inputs and enforce hard manufacturing constraints.