AutoSew: A Geometric Approach to Stitching Prediction with Graph Neural Networks

AutoSew is a fully automatic, geometry-based framework that utilizes Graph Neural Networks and optimal transport to predict stitch correspondences directly from 2D pattern contours, achieving high accuracy in assembling garments without relying on manual annotations or semantic cues.

The Gist

Imagine you have a pile of flat, 2D paper cutouts. These are the "sewing patterns" used to make clothes. If you were a human tailor, you would look at these shapes, use your experience, and know exactly which edges need to be sewn together to turn that pile of paper into a 3D jacket or dress. You'd know that the sleeve needs to go into the armhole, and that sometimes one long edge (like a sleeve) needs to be sewn to two different pieces at once (the front and back of the shirt).

Now, imagine trying to teach a computer to do this. The problem is that most computers are terrible at guessing. They usually need a human to label every piece ("This is a sleeve," "This is the front") or rely on perfect, standardized instructions that real-world factories rarely have.

Enter AutoSew.

Think of AutoSew as a super-smart, geometry-obsessed puzzle master that doesn't need labels. It doesn't care what the piece is called; it only cares about the shape of the edges.

Here is how it works, broken down into simple concepts:

1. The "Shape Detective" (Geometric Features)

Instead of reading a label, AutoSew looks at every edge of the paper pattern like a detective examining a fingerprint. It measures:

• How long is the edge?
• Is it curved or straight?
• What angle does it make with the next piece?
• How many other edges are attached to this piece?

It turns all these measurements into a "mathematical ID card" for every single edge.

2. The "Social Network" (Graph Neural Networks)

This is the brain of the operation. Imagine all the edges of the clothing pattern are people at a party.

• Old methods tried to match people by just looking at them individually (e.g., "This edge is 5cm long, so it must match that 5cm edge").
• AutoSew uses a Graph Neural Network (GNN). Think of this as a social network where everyone talks to their neighbors.
  • An edge doesn't just look at itself; it asks its neighbors, "Hey, what does the edge next to me look like?"
  • By passing messages around the whole "party" (the entire garment pattern), the system builds a global understanding. It realizes, "Ah, this edge is part of a sleeve, and it needs to connect to two other edges to make a 3D shape," even if no one told it that explicitly.

3. The "Flexible Matchmaker" (Differentiable Optimal Transport)

Once the system has learned about all the edges, it has to decide who gets sewn to whom.

• The Problem: In real life, one edge might need to be sewn to two other edges at once (like a sleeve connecting to both the front and back of a shirt). Old computer methods usually force a "one-to-one" match, like a strict dating app that says, "You can only pick one partner."
• The Solution: AutoSew uses a mathematical tool called Optimal Transport. Think of this as a super-flexible matchmaker. It can say, "Okay, this sleeve edge is a 'polygamist'—it needs to connect to both the front and back panels simultaneously." It calculates the best possible arrangement for the whole group, allowing for these complex, multi-edge connections.

4. The "New Rulebook" (The Dataset)

To teach this system, the researchers realized the old training data was too simple. It was like teaching someone to drive only on empty, straight highways. Real life has curves, intersections, and complex traffic.

• They created a new dataset called M-E.GARMENTCODEDATA.
• They took over 18,000 existing patterns and manually "fixed" them to reflect real industrial sewing, specifically adding those tricky "one-to-many" connections (like sleeves). This gave the AI the real-world experience it needed to learn.

The Result

When you feed AutoSew a raw 2D pattern (with no labels, no 3D models, just the lines), it:

1. Analyzes the shapes.
2. Understands the relationships between all the pieces.
3. Predicts exactly which edges to sew, including the complex ones.
4. Outputs a perfect 3D assembled garment.

In a nutshell: AutoSew is the first computer program that can look at a pile of flat fabric shapes and figure out how to sew them into a 3D outfit just by looking at the geometry, handling complex "one-to-many" connections that previous AI couldn't understand. It's like teaching a robot to sew by showing it the shapes, rather than giving it a manual.

Technical Summary

1. Problem Statement

The paper addresses the challenge of automating garment assembly from 2D sewing patterns. While digital fashion tools can vectorize patterns, they lack the semantic cues (e.g., panel names, manual notches) required to determine which edges should be stitched together.

• Current Limitations: Existing methods rely heavily on semantic annotations or handcrafted heuristics, limiting their applicability to real-world, non-standardized industrial patterns.
• The Multi-Edge Gap: Most existing datasets and methods assume one-to-one stitching. However, real-world industrial assembly often involves multi-edge connections (e.g., a single sleeve edge stitched simultaneously to both front and back torso panels), which current models fail to handle.
• Goal: Develop a fully automatic, geometry-based system that predicts stitching correspondences (including multi-edge connections) using only 2D contour data, without relying on semantic labels or 3D information.

2. Methodology: AutoSew Architecture

AutoSew formulates the stitching prediction task as a partial graph matching problem solved end-to-end using a Graph Neural Network (GNN) and differentiable optimal transport.

A. Input and Graph Construction

• Input: 2D sewing pattern contours consisting of multiple panels.
• Graph Representation: Each stitching edge in the pattern is treated as a node in a graph. Edges between nodes represent the topological adjacency of panels.
• Geometric Features (24 dimensions): The model relies solely on geometric data, avoiding semantic dependencies. Features include:
  • Local Shape Descriptors (18 dims): Start/end vertices, length, orientation, curvature, control points.
  • Topological Properties (4 dims): Interior angles with adjacent edges, total edge count per panel, and a unique panel identifier.
  • Preprocessing: Features are normalized, rotated to ensure invariance, and edges are sorted anticlockwise.

B. Graph Neural Network (GNN)

• Architecture: The model uses GraphSAGE to learn node embeddings.
• Mechanism: It employs a sampling and aggregation strategy over $L$ hidden layers (5 layers in the final model).
• Advantage: Unlike Transformers which require positional encoding and are sensitive to input size/order, GNNs are permutation and size invariant. This allows the model to propagate information across the graph structure, capturing both local edge geometry and global garment context, regardless of the number of panels.

C. Differentiable Optimal Transport (OT)

To solve the assignment problem (matching edges to edges), the authors use a differentiable Sinkhorn algorithm instead of non-differentiable solvers like the Hungarian algorithm.

• Partial Assignment: The problem is formulated to allow edges to remain unstitched. This is achieved by introducing a "dustbin" node (dummy assignment) in the cost matrix.
• Cost Matrix: Pairwise costs are calculated via the inner product of learned edge embeddings.
• Multi-Edge Handling: The soft assignment matrix allows a single edge to be probabilistically assigned to multiple other edges. A hard assignment is derived by sorting probabilities and applying a confidence threshold ($\tau_{multi}$) to retain valid multi-edge connections.

3. Key Contributions

1. M-E.GARMENTCODEDATA Dataset:

   • The authors updated the existing GarmentCodeData dataset by modifying over 18,000 patterns to include realistic multi-edge stitching annotations.
   • This involved merging split sleeves and torso panels to reflect industrial "one-to-many" assembly logic, creating a new benchmark for realistic garment construction.

2. Geometry-Only Framework:

   • AutoSew is the first method to predict stitching purely from geometric features without requiring semantic labels (e.g., "sleeve," "collar") or 3D data.

3. Novel Problem Formulation:

   • Combines GNNs with differentiable optimal transport to handle variable pattern sizes and partial assignments (unmatched edges) in an end-to-end trainable framework.

4. Comprehensive Validation:

   • Includes extensive ablation studies on feature importance, GNN depth, and the multi-edge threshold.

4. Results and Performance

The model was evaluated on the new M-E.GARMENTCODEDATA test set using seven metrics, including Total F1-score (TF1) and Garment Sewn Percentage (GSP).

• Overall Accuracy:
  • Total F1-score: 96.2%
  • Garment Sewn Percentage (GSP): 73.3% (Successfully assembling a full garment without error).
• Multi-Edge Performance:
  • Multi-edge F1-score (MEF1): 84.7%
  • Multi-edge Recall: 89.5%
• Comparison:
  • On the original one-to-one dataset, AutoSew achieved 97.06% F1 and 80.6% GSP.
  • The drop in performance on the multi-edge dataset is attributed to the increased complexity of real-world scenarios (e.g., symmetric panels causing ambiguity).
• Efficiency:
  • Inference time is approximately 7ms per garment pattern on an NVIDIA RTX 4090.
  • Reducing Sinkhorn iterations from 500 to 100 reduced runtime by 3x with negligible accuracy loss.

Ablation Insights:

• Topological Properties: Removing these caused a massive drop in GSP (from 73.3% to 30.5%), proving global structure is critical.
• GNN vs. MLP: Replacing the GNN with a standard MLP dropped GSP by ~15%, highlighting the necessity of message passing for global context.
• Depth: 5 layers yielded the best results; 7 layers caused over-smoothing and performance degradation.

5. Significance and Impact

• Industrial Applicability: By removing the need for manual semantic annotation, AutoSew enables the digitization of vast legacy archives of sewing patterns, bridging the gap between 2D vectorization and 3D simulation.
• Robustness: The geometry-only approach makes the system robust to diverse, non-standardized patterns found in real manufacturing, where semantic data is often missing or inconsistent.
• Future Directions: The authors plan to expand the dataset to include more complex $N$-to-1 connections and explore adaptive learning for the multi-edge threshold to further improve generalization.

In conclusion, AutoSew demonstrates that geometric context alone is sufficient to robustly solve the complex, partial assignment problem of garment assembly, offering a scalable solution for the digital transformation of the fashion industry.