InfScene-SR: Arbitrary-Size Image Super-Resolution via Iterative Joint-Denoising

InfScene-VF proposes a diffusion-based framework for arbitrary-size image super-resolution that eliminates boundary artifacts and enables memory-efficient, parallelized inference on gigapixel imagery by introducing Variance-Corrected Fusion and Spatially-Decoupled Variance Correction to achieve spatially continuous joint-denoising.

The Gist

Imagine you have a giant, blurry photograph of a coastline, maybe the size of a whole city block. You want to zoom in and see every single leaf on a tree and every crack in the pavement. This is the job of Image Super-Resolution (SR): turning a low-quality, small image into a high-quality, huge one.

For a long time, computers struggled with this, especially when the image was massive (like a satellite photo). Here is how the authors of this paper, InfScene-SR, solved the problem using a clever mix of magic and math.

The Problem: The "Jigsaw Puzzle" Disaster

Imagine you have a giant mural you want to paint, but your brush is tiny. The old way to do this was to cut the mural into small squares, paint each square separately, and then tape them back together.

• The Issue: When you tape them back, the edges don't match perfectly. One square might have a slightly different shade of blue than its neighbor. In computer terms, this creates ugly "seams" or "grid lines" where the pieces meet.
• The Diffusion Model Problem: Modern AI (called Diffusion Models) is great at painting realistic textures, like grass or clouds. But if you ask it to paint 100 separate squares and tape them together, the AI gets confused. Because it adds a little bit of "random noise" (like static on an old TV) to make the image look real, taping the pieces together accidentally cancels out that noise. The result? The image becomes fuzzy and over-smoothed, losing all the cool details. It's like trying to listen to a symphony by plugging in 100 separate speakers that are slightly out of sync; the music turns into a muddy mess.

The Solution: InfScene-SR

The authors created InfScene-SR, a new way to paint the mural that keeps the edges seamless and the details sharp. They did this in two main steps:

1. The "Variance-Corrected Fusion" (Fixing the Fuzziness)

Think of the AI's "random noise" as the secret ingredient that makes a photo look crisp and real. When the old method glued the pieces together, it accidentally washed away this secret ingredient.

The authors invented a special "glue" called Variance-Corrected Fusion (VCF).

• The Analogy: Imagine you are mixing a batch of cookies. If you take 10 bowls of cookie dough and just dump them into one big bowl, the texture might get weird. But if you use a special recipe that tells you exactly how much "crunch" (noise) to add back in after mixing, you get the perfect cookie every time.
• The Result: This technique ensures that when the AI stitches the pieces together, it doesn't lose the "crunch." The image stays sharp and full of detail, not blurry.

2. The "Spatially-Decoupled" Trick (The Parallel Superpower)

Even with the perfect glue, painting a massive mural is slow if you have to wait for one painter to finish a square before the next one starts. The old method required all the computers to talk to each other constantly to make sure the math was right, which is slow and requires huge amounts of memory (like trying to hold a library of books in your head at once).

The authors introduced Spatially-Decoupled Variance Correction (SDVC).

• The Analogy: Imagine a team of 100 painters. Instead of standing in a circle discussing every brushstroke, they are given a map with a grid. Each painter works on their own square independently. They don't need to talk to anyone else because they have a special "instruction sheet" (the math formula) that tells them exactly how their piece will fit with the neighbors.
• The Result: This allows the computer to process the image in parallel. You can use many small, cheap computers (or even a regular gaming PC) to super-resolve a massive image that used to require a supercomputer. It turns a slow, heavy process into a fast, lightweight one.

Why Does This Matter? (The Real-World Impact)

The authors tested this on satellite images of California.

• Before: If you tried to zoom in on a satellite photo to count invasive plants (like Iceplant) or track a disaster, the "seams" between the image patches would confuse the computer. It might think a road ended abruptly or miss a patch of plants entirely.
• With InfScene-SR: The image is seamless. The computer can now see the whole picture clearly.
  • Better Accuracy: In their tests, the AI could identify invasive plants almost as well as if it were looking at the original, high-resolution photo.
  • No More Blurry Edges: The "grid lines" disappeared, making the map look like a real, continuous landscape.

Summary

InfScene-SR is like a master artist who can take a blurry, low-res photo of the entire world and turn it into a crystal-clear, high-definition masterpiece without leaving any visible seams or making it look fuzzy. They did this by:

1. Fixing the math so the AI doesn't lose its "randomness" (which creates detail) when stitching pieces together.
2. Reorganizing the workflow so many computers can work at the same time without getting in each other's way.

This means we can now analyze huge satellite images, medical scans, or microscopic photos with incredible detail, using standard computers, opening the door for better disaster response, farming, and scientific discovery.

Technical Summary

1. Problem Statement

While diffusion models (specifically Denoising Diffusion Probabilistic Models like SR3) have achieved state-of-the-art results in Single Image Super-Resolution (SISR), they face two critical limitations when applied to real-world, large-scale imagery (e.g., gigapixel satellite remote sensing or medical whole-slide imaging):

• Fixed Input Constraints: Standard diffusion models are restricted to fixed, small input sizes (e.g., 512×512) due to the memory demands of attention mechanisms.
• The Patch-Stitching Dilemma: The standard engineering workaround involves partitioning large images into independent patches, super-resolving them, and stitching them together. This approach fails for diffusion models because:
  • Boundary Artifacts: Independent generation of contiguous patches leads to semantic discontinuities and visible seams.
  • Variance Erosion (The Core Issue): When applying joint-denoising (fusing overlapping patches at every denoising step) to Stochastic Differential Equation (SDE) based solvers, the naive averaging of overlapping regions artificially attenuates the stochastic noise variance. Over successive iterations, this "variance erosion" causes the model to lose its high-frequency generative capability, resulting in blurry, over-smoothed outputs.

2. Methodology: InfScene-SR

The authors propose InfScene-SR, a diffusion-based framework designed to perform spatially continuous, arbitrary-size super-resolution. The method builds upon the SR3 architecture and introduces two key innovations to solve the variance erosion and scalability issues.

A. Variance-Corrected Fusion (VCF)

To address the blurring caused by naive joint-denoising, the authors adapt the Variance-Corrected Fusion strategy.

• Mechanism: In standard joint-denoising, overlapping patches are averaged, which reduces the variance of the stochastic noise term ($\sigma_t \epsilon_t$). VCF mathematically reformulates the fusion step to restore the exact target variance ($\sigma_t^2$) while preserving the guided mean.
• Formula: Instead of a simple weighted average, the fused pixel is calculated by scaling the weighted sum of sampled states and adding a correction term based on the weighted sum of predicted means. This ensures the fused distribution matches the statistical properties of the original diffusion process, preserving high-fidelity textures.

B. Spatially-Decoupled Variance Correction (SDVC)

While VCF solves the quality issue, the standard implementation requires gathering all patch data into a centralized memory for global normalization, creating a massive synchronization bottleneck that prevents distributed inference.

• Mechanism: The authors reformulate the VCF equation into Spatially-Decoupled Variance Correction (SDVC). They pre-calculate global normalization maps ($W$ and $S$) representing the sum of weights and the square root of the sum of squared weights for every pixel coordinate.
• Decoupling: By substituting these global maps into the fusion equation, the computation is transformed into independent, atomic patch operations. Each GPU or node can compute its local contribution tensor ($C_t^{(i)}$) without needing to communicate with other nodes during the denoising steps.
• Result: The final image is reconstructed by asynchronously accumulating these local tensors. This reduces memory complexity to $O(1)$ (independent of the total image resolution) and enables fully parallelized, distributed inference on gigapixel images.

3. Key Contributions

1. Identification of Variance Erosion: The paper formally identifies and mathematically explains why naive joint-denoising fails in SDE-based diffusion models, leading to texture degradation.
2. Adaptation of VCF for SR: The authors successfully adapt Variance-Corrected Fusion to the conditional super-resolution setting, enabling diffusion models to generate continuous, high-fidelity textures on arbitrary-sized inputs.
3. SDVC for Scalability: The derivation of Spatially-Decoupled Variance Correction allows for fully distributed, parallelized inference with constant memory complexity, making gigapixel super-resolution feasible on consumer-grade hardware.
4. Downstream Utility Validation: Unlike many SR papers that focus solely on pixel metrics, this work emphasizes the utility of the output for downstream tasks (semantic segmentation), proving that the generated details are semantically meaningful.

4. Experimental Results

The method was evaluated on a 5× super-resolution task using NAIP (National Agriculture Imagery Program) remote sensing data (0.6m resolution) and a downstream Iceplant (invasive species) segmentation task.

• Quantitative Performance:

  • Perceptual Quality: InfScene-SR achieved the best perceptual scores with the lowest FID (33.09) and KID (0.0117), significantly outperforming both Bicubic interpolation and standard patch-based SR3.
  • Reconstruction Fidelity: It reduced the RMSE from 37.05 (standard SR3) to 24.89, while maintaining a competitive PSNR (20.21 dB).
  • Comparison: Bicubic interpolation had the highest PSNR (28.77 dB) but the worst perceptual quality (FID 90.13), confirming the perception-distortion trade-off.

• Qualitative Results:

  • Visual comparisons show that InfScene-SR eliminates the grid-like artifacts and seams visible in standard SR3 outputs.
  • It recovers fine, high-frequency details (e.g., vegetation texture) that are lost in Bicubic interpolation.

• Downstream Task (Semantic Segmentation):

  • Using the super-resolved images for Iceplant detection, InfScene-SR achieved an IoU of 0.7461 and F1 Score of 0.8546, nearly matching the performance of the original High-Resolution ground truth (IoU 0.7577).
  • Standard SR3 performed poorly (IoU 0.6797) due to boundary artifacts confusing the segmentation network.
  • Bicubic interpolation suffered from over-segmentation due to blurred boundaries.
  • Conclusion: The high-frequency details synthesized by InfScene-SR are not "hallucinations" but contain semantically valid cues that improve land-cover classification.

5. Significance

InfScene-SR represents a significant step forward in applying generative AI to large-scale scientific imaging.

• Scalability: It removes the memory barrier that previously restricted diffusion models to small crops, enabling the processing of gigapixel scenes (e.g., entire counties or medical slides) on standard hardware.
• Scientific Impact: By providing seamless, high-fidelity reconstructions, it bridges the gap between coarse, frequent satellite data (e.g., Planet 3m) and high-resolution, infrequent data (e.g., NAIP 0.6m). This enables time-sensitive applications like agricultural phenology tracking, disaster response, and invasive species monitoring.
• Generalizability: The approach is applicable beyond remote sensing to any domain requiring large-content image generation, such as medical pathology and electron microscopy.