💻 computer science

Match-and-Fuse: Consistent Generation from Unstructured Image Sets

Match-and-Fuse is a zero-shot, training-free framework that generates consistent image sets from unstructured collections by modeling the task as a graph to fuse pairwise features, thereby preserving shared visual elements across varying viewpoints and contexts without manual supervision.

Original authors: Kate Feingold, Omri Kaduri, Tali Dekel

Published 2026-03-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kate Feingold, Omri Kaduri, Tali Dekel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a photo album of your favorite vintage bag. You have one picture of it on a sunny beach, another in a cozy coffee shop, and a third hanging on a rack in a boutique. They are all the same bag, but the backgrounds and angles are totally different.

Now, imagine you want to use AI to turn this bag into a shiny, futuristic robot-bag for a sci-fi movie poster. You want to keep the exact same robot-bag in all three photos, but change the backgrounds to look like a cyberpunk city, a space station, and a neon alley.

The Problem:
Current AI tools are like a clumsy artist who gets confused when asked to draw the same character in three different places.

If you ask the AI to draw the robot-bag in the coffee shop, it might make it look like a toaster.
If you ask it to draw the robot-bag in the space station, it might make it look like a toaster with a different handle.
The result is a set of images where the "hero" object looks like three different characters. It breaks the magic.

The Solution: "Match-and-Fuse"
The paper introduces a new method called Match-and-Fuse. Think of it as a super-smart art director who manages a team of painters.

Here is how it works, using a simple analogy:

1. The "Graph" (The Team Huddle)

Instead of asking the AI to paint each picture one by one (which leads to mistakes), Match-and-Fuse treats the whole photo album as a team huddle.

Imagine every photo is a person in a circle.
The AI draws invisible strings (edges) connecting every person to every other person.
Now, instead of working alone, the AI looks at pairs of photos at the same time. It asks: "Okay, if I'm painting the bag in the coffee shop AND the bag in the space station simultaneously, how do I make sure they look like the exact same robot?"

2. The "Fusion" (The Shared Canvas)

This is the magic trick. The AI uses a technique called Multiview Feature Fusion.

Imagine the AI has a special pair of glasses that can see the "skeleton" of the bag in the original photos.
When it starts painting the new robot-bag, it constantly checks its glasses. If it paints a red bolt on the left side of the bag in the coffee shop photo, it instantly paints that same red bolt in the exact same spot on the bag in the space station photo.
It does this for every pair of photos in the album, fusing their details together. It's like if three people were drawing the same picture on three different pieces of paper, but they were all holding their hands together so their pencils moved in perfect sync.

3. The "Guidance" (The Final Polish)

Sometimes, even with the team huddle, tiny details might get slightly out of sync (like a button being slightly higher in one photo).

The method adds a final step called Feature Guidance. This is like a strict editor who walks around the room, checking every single detail.
If the editor sees a mismatch, they gently nudge the AI's "latent space" (its internal thought process) to fix it, ensuring the robot-bag looks identical in every single photo, down to the texture of the metal.

Why is this a big deal?

No Training Needed: You don't need to teach the AI anything new. It uses the AI it already knows, just with a smarter workflow.
It Handles Chaos: It works even if your photos are messy—some are close-ups, some are far away, some are sideways. It doesn't need perfect 3D models or video; it just needs a collection of photos.
The Result: You get a set of images where the "hero" object is perfectly consistent (it's the same robot-bag), but the world around it changes exactly how you asked (from a coffee shop to a space station).

In a nutshell:
Match-and-Fuse stops the AI from getting confused by forcing it to "think together" across all your photos at once. It ensures that if you change a bag into a robot, that robot stays the same robot in every single picture, no matter how crazy the background gets. It turns a chaotic pile of photos into a perfectly consistent visual story.

1. Problem Statement

The paper addresses the challenge of consistent set-to-set generation from unstructured image collections. Unlike video generation (which benefits from temporal continuity) or single-image editing, existing generative AI methods struggle to maintain cross-image consistency when processing a set of images that share a common object or subject but differ in:

Viewpoint and Pose: The object may be rotated, scaled, or articulated differently.
Environment: Backgrounds and lighting vary significantly.
Structure: The sets are often unordered and lack a continuous 3D representation.

Current methods typically operate on individual images or require dense temporal sampling (videos). When applied to image sets, they often fail to preserve the identity, geometry, and texture of shared elements across the collection, leading to "identity drift" or inconsistent appearances.

2. Methodology: Match-and-Fuse

The authors propose Match-and-Fuse, a zero-shot, training-free framework that leverages pre-trained text-to-image (T2I) diffusion models (specifically FLUX with depth conditioning). The core innovation is modeling the image set as a graph to enforce consistency through pairwise interactions and feature fusion.

Key Components:

Pairwise Consistency Graph:
- The input set of $N$ images is modeled as a graph $G=(V, E)$ , where nodes are images and edges represent all possible image pairs.
- Instead of generating images independently or in a fixed grid, the method generates joint two-image grids for every edge in the graph.
- This formulation allows the model to exploit the "grid prior" (the ability of T2I models to generate coherent multi-image layouts) while overcoming the resolution and consistency limits of generating a single large canvas.
Multiview Feature Fusion (MFF):
- To ensure that shared regions (e.g., a specific bag or character) look identical across different generated pairs, the method fuses internal features.
- Dense Correspondences: It uses off-the-shelf feature matching (RoMA) on the source images to identify pixel-level correspondences between shared objects.
- Feature Averaging: During the denoising process, the latent features of matched coordinates across different image pairs are averaged. This forces the model to align the appearance of shared elements globally, not just locally between two images.
Feature Guidance:
- As a refinement step, the method applies a feature-matching objective via gradient descent (Feature Guidance).
- This minimizes the distance between matched features in the generated output, correcting fine-grained inconsistencies that MFF might miss.
- This step is applied lightly to avoid pushing the latents out of distribution.
Prompt Composition:
- A Vision-Language Model (VLM) automatically generates detailed per-image captions.
- These captions separate the shared content (controlled by a user prompt $P_{shared}$ ) from the unique background/pose (controlled by a theme prompt $P_{theme}$ ), ensuring the edit applies consistently to the object while allowing backgrounds to vary.

3. Key Contributions

First Unstructured Set-to-Set Method: It is the first framework to perform consistent generation across entire collections of unstructured images, moving beyond pairwise editing or fixed grids.
Training-Free & Zero-Shot: The method requires no fine-tuning of the base diffusion model, relying instead on architectural modifications (graph denoising, feature fusion) and prompt engineering.
Graph-Based Consistency: It introduces a novel graph formulation that consolidates pairwise generations into a unified framework, enforcing local consistency (between pairs) and global coherence (across the whole set).
New Evaluation Metric: The authors introduce DINO-MatchSim, a metric that measures fine-grained cross-image consistency by computing cosine similarity of DINOv3 features at matched pixel locations, which correlates better with human judgment than global similarity metrics.

4. Results

Quantitative Performance:
- Match-and-Fuse achieves state-of-the-art consistency scores (DINO-MatchSim: 0.80) compared to baselines like FLUX, IC-LoRA, and Edicho (which score between 0.57 and 0.72).
- It maintains high prompt adherence (CLIP Score) while significantly outperforming others in preserving object identity and texture.
- User studies and VLM evaluations show a strong preference (88–92% agreement) for Match-and-Fuse over all baselines.
Qualitative Performance:
- The method successfully handles rigid and non-rigid objects, varying poses, and complex backgrounds.
- It preserves fine-grained details (textures, typography, small patterns) across sets of up to 15 images.
- It outperforms methods that rely on a single reference image (like Edicho), which often degrade in consistency for views far from the anchor.
Scalability: By limiting graph connectivity (node degree of 4) for large sets, the method maintains linear runtime complexity while preserving consistency.

5. Significance and Applications

Creative Workflows: The method enables transformative workflows for product advertising, character concept art, and film set design, where a single object must be visualized consistently across multiple distinct scenes or angles.
Beyond Single-Image Limits: It unlocks the ability to edit "collections" rather than just single images, addressing a fundamental gap in Generative AI that has previously been limited to single-image or video domains.
Future Directions: The paper lays the groundwork for future set-to-set foundation models and applications in video editing (by treating video frames as an unstructured set) and 3D reconstruction.

In summary, Match-and-Fuse solves the difficult problem of maintaining visual identity across a diverse set of images by treating the generation task as a graph problem, leveraging dense correspondences to fuse features, and utilizing the emergent capabilities of pre-trained diffusion models without requiring additional training.