Technical Summary: GarmentZoom

Problem Statement

Online garment listings typically present a fragmented viewing experience: a standard-resolution full-view image provides context, while separate close-up images reveal fine texture details. Users must constantly toggle between these views, breaking browsing continuity. While Reference-Based Super-Resolution (RefSR) offers a potential solution, existing methods face significant limitations in this specific domain:

1. Spatial Misalignment: Standard RefSR assumes the reference image and the low-resolution (LR) input share spatial correspondence. In garment listings, close-ups often cover only a small, non-overlapping region of the full garment, and may differ in viewpoint, lighting, or fabric deformation, making pixel-level alignment infeasible.
2. Variable Scale Factors: Traditional RefSR methods assume fixed, discrete upsampling factors (e.g., 4×). Real-world garment listings exhibit continuous scale factors ranging from 3× to 20× between the full view and the close-up.
3. Computational Cost: Prior approaches capable of handling variable scales and misalignment (e.g., UltraZoom) rely on per-instance fine-tuning, which is computationally prohibitive for large-scale e-commerce deployment.

Methodology

GarmentZoom addresses these challenges by synthesizing a single, ultra-high-resolution garment image from a full-view photo and an unaligned close-up reference. The system is built upon Flux.1-dev, a state-of-the-art flow-matching model, and incorporates three key components:

1. Dataset Construction

Since real-world product photos lack reliable pixel-level alignment, the authors construct a synthetic supervised dataset from high-resolution garment close-ups.

• Triplet Generation: For each high-resolution close-up, two spatially separated crops are sampled: one as the reference ($I_{Ref}$) and one as the ground truth ($I_{GT}$).
• Synthetic LR Input: The ground truth is downsampled by a random continuous scale factor $s \in [3, 20]$ to create the low-quality input ($I_{LQ}$).
• Constraint: Spatial separation between crops prevents the model from learning trivial copy-paste solutions, forcing it to learn texture transfer based on appearance cues rather than direct correspondence.
• Text Conditioning: A vision-language model generates detailed text descriptions for each garment to serve as auxiliary conditioning.

2. Model Architecture

The system adapts the Flux.1-dev transformer architecture with the following modifications:

• Unified-Sequence Conditioning: The reference image ($I_{Ref}$) and the noisy target latent ($x_t$) are encoded into latent sequences and concatenated. This combined sequence is processed by the transformer, allowing the target tokens to access reference texture information via cross-attention mechanisms.
• ControlNet Integration: A frozen, pretrained 4× Single-Image Super-Resolution (SISR) ControlNet is conditioned on the low-quality input ($x_{LQ}$) to preserve the global structure and fabric deformation of the original full-view image.
• Training Objective: The model is trained using a flow-matching objective to predict the velocity field, minimizing the difference between the predicted and target velocities.
• Parameter Efficiency: The model is fine-tuned using LoRA (Low-Rank Adaptation) with a rank of 512, keeping the base weights frozen to ensure efficiency.

3. Inference Strategy

To handle full-resolution generation within GPU memory constraints, the authors employ a sliding-window inference strategy:

• The full-view image is divided into overlapping windows.
• Inference is performed on each patch independently.
• To mitigate artifacts caused by repeated window boundaries, the position of the sliding windows is varied at each inference timestep.
• Overlapping latent regions are averaged on a unified canvas to produce a seamless, artifact-free output.

Key Contributions

1. A Novel System: GarmentZoom synthesizes a single high-detail garment image from full-view and unaligned close-up photos, enabling seamless zoom-and-pan exploration.
2. Robust Reference-Guided SR: The model supports spatially unaligned references and continuous scale factors (3–20×), overcoming the limitations of fixed-scale and alignment-dependent prior methods.
3. Efficient Training Pipeline: The approach trains a single generalizable model using synthetic triplets derived from real e-commerce listings, avoiding the high computational cost of per-instance fine-tuning.

Experimental Results

The method was evaluated on a dataset of 8,547 garment products, comparing against general RefSR baselines (TTSR, DATSR, AdaRefSR, ReFIR) and a continuous-scale SISR baseline (ContinuousSR).

• Quantitative Performance: GarmentZoom achieved state-of-the-art results across perceptual metrics (LPIPS, DISTS) and frequency-domain metrics (LSD) at both 4× and 10× scales. Notably, it maintained consistent performance up to 16×, whereas baseline methods degraded significantly as scale factors increased.
• User Study: In a study with 13 participants, GarmentZoom received top-1 preference in 97.7% of cases for 10× scaling, significantly outperforming fine-tuned baselines.
• Comparison with Per-Instance Methods: When compared qualitatively and quantitatively against UltraZoom (which requires ~1.5 hours of fine-tuning per product), GarmentZoom produced results of comparable quality without instance-specific training. It demonstrated superior color fidelity and avoided the "color drift" issues observed in UltraZoom.
• Ablation Studies:
  • Scale Robustness: The model generalized effectively across the 3–20× range.
  • Reference Similarity: Performance decreased as the spatial displacement between the reference and target increased, confirming the model relies on reference similarity but remains robust to moderate misalignment.
  • Architecture: The unified-sequence conditioning with LoRA outperformed alternative cross-attention injection methods (inspired by IP-Adapter), suggesting that co-evolving reference and input latents within the same attention space is more effective for texture transfer.

Significance and Claims

The paper positions GarmentZoom as a practical step toward seamless virtual product experiences. By decoupling the need for spatial alignment and per-instance training, the system enables the generation of high-fidelity, zoomable garment images at a fraction of the training cost of existing per-instance methods. The authors claim the system successfully generalizes across diverse garments and scale factors, bridging the gap between overview context and detail-level fidelity in online shopping.

Limitations: The authors acknowledge that, like other RefSR methods, if the required texture is entirely absent from the reference image, the model must hallucinate plausible details. They suggest future work could focus on automatically selecting optimal reference crops to cover necessary textures.