PhotoQuilt: Training-Free Arbitrary-Resolution Photomosaics via Bootstrapped Tiled Denoising

Technical Summary: PhotoQuilt

Problem Statement
Photomosaics are composite images where local regions function as independent, self-contained photographs, yet their collective arrangement reconstructs a coherent global scene. Generating high-resolution photomosaics presents a dual challenge: the system must maintain global structural coherence (the target image must be recognizable from a distance) while ensuring local tile autonomy (each tile must be a convincing, detailed image). Existing approaches struggle to satisfy both scales simultaneously. Retrieval-based methods are constrained by finite image pools, leading to repetitive tiles and limited color correction. Generative methods, particularly those using diffusion models, face a trade-off: direct high-resolution generation is computationally expensive and tends to produce a single smooth image rather than a mosaic, while sequential tile generation or patch-based methods often lose global structure or require costly coordination procedures to align independently generated tiles.

Methodology: Bootstrapped Tiled Denoising
PhotoQuilt is a training-free framework that decouples global composition from local tile generation. It operates without architectural changes to the underlying diffusion model (compatible with both U-Net and Diffusion Transformer backbones). The method proceeds in three stages:

1. Globally-Coherent Initialization: A base latent representation ($z_{base}$) is generated at a low resolution (e.g., 768×768) either from a text prompt or by encoding a reference image. This base is upsampled to the target latent grid size using a fixed upsampler.
2. Partial Renoising: The upsampled latent is partially corrupted with Gaussian noise at a specific strength $s$. This step is critical: it retains the coarse, low-frequency layout of the global structure while destroying high-frequency details. This renoised latent ($\tilde{z}_s$) serves as a shared initialization for all tiles, acting as a single global control that fixes the overall scene composition.
3. Independent Per-Tile Denoising: The target latent space is partitioned into $K$ non-overlapping tiles. Each tile is denoised independently, starting from its specific region within the shared renoised latent $\tilde{z}_s$. Crucially, each tile follows its own denoising trajectory conditioned on a specific tile-level prompt or reference image. Because the tiles are denoised separately within fixed regions, they develop unique high-frequency details (autonomy) while remaining anchored to the shared low-frequency structure (global coherence).

The process concludes by decoding the independently denoised tiles into the final photomosaic image. The method avoids quadratic attention costs by confining attention mechanisms within each tile, allowing the generation of arbitrarily large canvases.

Key Contributions

• Training-Free Framework: PhotoQuilt requires no additional training, fine-tuning, or specialized modules (such as ControlNet or adapters). It leverages the internal capabilities of pre-trained text-to-image and image-to-image models.
• Arbitrary Resolution: By decoupling global layout from local detail generation and confining attention to individual tiles, the method scales to high resolutions without the computational bottlenecks associated with global attention mechanisms.
• Dual-Scale Optimization: The "bootstrapped" approach explicitly resolves the tension between global structure and local realism. The shared renoised latent enforces the global layout, while the independent denoising trajectories allow each tile to diverge into a distinct, high-quality image.
• Model Agnosticism: The framework is compatible with various diffusion backbones, including Stable Diffusion 2.1, FLUX.1, and FLUX.2.

Experimental Results
Experiments were conducted on high-resolution mosaics (6144×6144 pixels) using 8×8 tile grids. PhotoQuilt was evaluated against six baselines, including retrieval-based methods (Match & Tone), style transfer approaches (AdaIN), and other generative strategies (Phomosaic, StreamDiff).

• Quantitative Performance: PhotoQuilt outperformed existing baselines on both global structure preservation (measured by PSNR, SSIM, LPIPS, and HPSv2 on downsampled images) and local tile quality (measured by CLIP score, BLIP, and Image Reward). Notably, while methods like StreamDiff achieved high global fidelity, they failed to produce diverse tiles (collapsing local scores). Conversely, methods focusing on tile autonomy often sacrificed global coherence. PhotoQuilt achieved the best balance, with the FLUX.1 variant achieving the highest CLIP and Image Reward scores alongside strong global metrics.
• Ablation Studies:
  • Global Guidance: Removing the bootstrapped initialization (replacing the shared latent with independent noise) resulted in high-quality individual tiles but a complete collapse of global structure (PSNR dropped to 13.10), confirming the shared latent is essential for layout coherence.
  • Renoising Strength ($s$): The strength parameter $s$ acts as a dial between global fidelity and tile independence. Lower $s$ values preserve global structure but restrict tile diversity; higher $s$ values increase tile freedom but degrade global coherence. A value of $s=0.6$ was identified as the optimal trade-off.
  • Bootstrap Resolution: Higher resolution base images (768×768) provided richer structural guidance compared to lower resolutions (256×256), directly improving global fidelity.

Significance and Claims
The paper claims that PhotoQuilt offers a minimal yet effective solution to high-resolution photomosaic generation. By resolving the dual-scale problem through a bootstrapped tiled denoising procedure, it enables the creation of large-scale mosaics where the target image is clearly recognizable as a whole, while every tile remains a meaningful, self-contained photograph. The authors emphasize that this approach avoids the computational expense of direct high-resolution generation and the coordination overhead of sequential tile assembly, providing a stable and efficient method that works across a wide range of output resolutions and model backbones.