Rethinking Few-Shot Image Fusion: Granular Ball Priors Enable General-Purpose Deep Fusion

This paper proposes a few-shot image fusion framework that leverages a Granular Ball Pixel Computation algorithm to generate adaptive, confidence-aware incomplete priors, enabling a lightweight neural network to learn effective fusion rules from minimal data without requiring real fused images for supervision.

The Gist

Imagine you are trying to bake the perfect cake, but you've never seen a finished one before. You only have two ingredients: a bowl of flour (Image A) and a bowl of sugar (Image B). You don't have a recipe, and you don't have a picture of the final cake to copy.

Most AI researchers try to solve this by feeding the computer millions of examples of "flour + sugar = cake" so it can memorize the pattern. But what if you only have ten examples? That's the problem this paper tackles: How do you teach an AI to fuse images when you barely have any data to teach it with?

Here is the simple breakdown of their solution, using some everyday analogies.

1. The Problem: The "Perfect Recipe" Trap

Usually, to teach an AI to combine images (like merging a night-vision photo with a regular photo), researchers use "priors." Think of a prior as a pre-written recipe or a guidebook.

• Old Way: They used "Complete Priors." This is like giving the AI a recipe that says, "Mix exactly 50% flour and 50% sugar." The AI just copies this. The problem? If the recipe is slightly wrong for a specific cake, the AI blindly follows it and ruins the cake. It also needs millions of cakes to learn that the recipe isn't perfect.
• The New Idea: The authors say, "Let's stop giving the AI a perfect recipe. Let's give it a rough sketch and let the AI finish the painting."

2. The Solution: The "Granular Ball" Detective

The authors introduce a new tool called Granular Ball Pixel Computing (GBPC). Imagine you are a detective trying to figure out which parts of a crime scene photo are important.

Instead of looking at every single pixel (every single grain of sand on the beach), the AI groups pixels into "Granular Balls."

• The Analogy: Imagine you are looking at a crowd of people. Instead of analyzing every single face, you group them into "balls" of people standing close together.
• The Magic: The AI looks at these "balls" and asks two questions:
  1. Are these two images similar here? (e.g., Both are dark). If yes, it's a "Boundary" area. It's fuzzy, and the AI isn't sure what to do.
  2. Are they totally different? (e.g., One is bright, one is dark). If yes, it's a "Positive" area. The AI knows, "Ah, this part is important! I can trust my guess here."

3. The "Incomplete Prior": The Sketch vs. The Masterpiece

This is the core genius of the paper.

• The GBPC algorithm creates a "Rough Sketch" (the Incomplete Prior). It fills in the easy parts (the "Positive" areas) where the images clearly agree or disagree.
• But for the fuzzy, confusing parts (the "Boundary" areas), the sketch is left blank or blurry.
• Why is this good? Because the AI isn't forced to copy a potentially wrong recipe. Instead, it sees the sketch and thinks, "Okay, the AI knows the sky should be blue, but it's unsure about the edges of the building. I will use my own brain to figure out the building edges based on the original photos."

4. Few-Shot Learning: Learning from Ten Examples

Because the AI is doing the heavy lifting of "reasoning" on the blurry parts, it doesn't need to memorize millions of examples.

• The Analogy: Imagine teaching a child to draw a cat.
  • Old Way: Show them 10,000 cat drawings so they memorize every whisker.
  • New Way: Show them 10 drawings, but tell them, "Here is the outline of the head and body (the Prior). You figure out the whiskers and tail (the Reasoning)."
  • The child learns the logic of drawing a cat, not just the specific picture.

5. The Result: A Lightweight, Smart Fusion

The authors tested this on various tasks:

• Night Vision + Day Vision: Merging thermal and regular cameras.
• Medical Scans: Combining PET and MRI scans.
• Multi-Exposure: Fixing photos that are too bright in some spots and too dark in others.

The Outcome:
Even though they only trained the AI on 10 image pairs (a tiny amount!), the AI produced results that were better than massive, complex AI models trained on thousands of images.

• Efficiency: The AI model is tiny (like a smartwatch app) compared to the "supercomputers" usually needed for this.
• Quality: The images look sharper, with better details and fewer weird artifacts.

Summary

This paper is about trusting the AI to think rather than just memorize.
By giving the AI a "rough sketch" (Incomplete Prior) that highlights what is known and what is uncertain, the AI learns to fill in the gaps itself. This allows it to become an expert at combining images after seeing only a handful of examples, making it fast, cheap, and incredibly smart.

In one sentence: They taught the AI to be a detective that solves the puzzle itself, rather than a robot that just copies a manual.

Technical Summary

Here is a detailed technical summary of the paper "Rethinking Few-Shot Image Fusion: Granular Ball Priors Enable General-Purpose Deep Fusion."

1. Problem Statement

Image fusion aims to combine information from multiple sensors (e.g., infrared/visible, multi-exposure, multi-focus, medical) into a single, comprehensive image. However, current deep learning approaches face two critical challenges:

• Lack of Ground Truth: Real fused images are rarely available as supervision signals, making supervised learning difficult.
• Data Dependency: Existing methods typically rely on large-scale datasets to learn fusion rules or use fixed, handcrafted priors (like wavelet transforms) that lack adaptability. This makes few-shot learning (training on very few samples) difficult, as models tend to overfit or fail to generalize to complex real-world scenarios.
• Rigidity of Hybrid Methods: Traditional hybrid approaches often use fixed loss functions based on complete priors, creating a rigid coupling between the algorithm and the network that prevents adaptive learning.

2. Methodology

The authors propose a novel framework that integrates Granular Computing with deep learning to enable few-shot image fusion. The core innovation is the concept of "Incomplete Priors" and the Granular Ball Pixel Computation (GBPC) algorithm.

A. Granular Ball Pixel Computation (GBPC)

Instead of treating pixels as isolated points, GBPC models fused-image pixels as information units using "meta-granular balls."

• Meta-Granular Balls: Represent paired pixel attributes from two input images at the same coordinate.
• Multi-Granularity Analysis:
  • Fine-Grained Level: Uses adaptive granular balls to calculate pixel-level weights based on feature similarity, performing initial fusion.
  • Coarse-Grained Level: Uses Rough Set Theory to evaluate the reliability of the generated prior. It classifies regions into:
    • Positive Domain (POS): Regions with high confidence where the prior is reliable (e.g., clear structural edges).
    • Boundary Domain (BND): Regions with uncertainty where the prior is incomplete (e.g., ambiguous edges or over-exposed areas).
• Modality Perception: The algorithm statistically analyzes the ratio of POS regions. If the ratio exceeds a threshold (indicating extreme intensity discrepancies, common in multi-exposure fusion), it resets weights to enforce equal contribution from both modalities, preventing over-exposure dominance.

B. Incomplete Priors & Adaptive Learning

Unlike traditional methods that provide a "complete" fused image as a target, GBPC generates an incomplete prior.

• Concept: The prior acts as a degraded image with regional confidence labels.
• Mechanism: The neural network is not trained to simply mimic the prior. Instead, it is tasked with re-inference:
  • In POS regions, the network trusts the structural cues of the prior.
  • In BND regions, the network must infer missing details (edges, textures) directly from the source images.
• Sample-Level Adaptive Loss: A custom loss function dynamically adjusts based on the confidence ($r_{POS}$ and $r_{BND}$) of the prior:
  • $L_{SSIM}$: Transfers structural features from the prior.
  • $L_{POS}$: Enforces edge fidelity in reliable regions.
  • $L_{BND}$: Guides the network to extract edge features from source images in uncertain regions using Sobel and Laplacian operators.

C. Few-Shot Strategy

The framework simulates complex real-world environments by randomly cropping input images into patches. Even with only 10 image pairs (approx. 100+ patches), the adaptive nature of GBPC allows the generation of diverse priors, enabling the lightweight CNN to learn robust fusion rules without massive datasets.

3. Key Contributions

1. First Application of Granular Computing to General Fusion: Introduces granular ball computing to a unified framework covering infrared-visible, multi-exposure, multi-focus, and medical image fusion.
2. Incomplete Prior Concept: Proposes a theoretical shift from "complete priors" (which cause overfitting and bias) to "incomplete priors" that explicitly model uncertainty, allowing the network to perform re-reasoning.
3. GBPC Algorithm: Develops an algorithm that performs fine-grained pixel weighting and coarse-grained reliability estimation without explicit spatial partitioning, creating a unified information granulation standard.
4. Sample-Level Adaptive Coupling: Establishes a deep coupling mechanism where the loss function adapts to the specific characteristics of each sample's prior, transforming the learning objective from distribution modeling to sample-specific re-reasoning.
5. State-of-the-Art Few-Shot Performance: Demonstrates that a lightweight network trained on only 10 image pairs can outperform or match heavy, data-hungry models (including Diffusion and Transformer-based methods) in both visual quality and computational efficiency.

4. Experimental Results

The method was evaluated across four major fusion tasks using standard datasets (MEFB, Lytro, MFI-WHU, M3FD, MSRS, TNO, Harvard PET-MRI).

• Performance: The proposed method achieved best or second-best performance in quantitative metrics (MI, PSNR, VIF, SCD, $Q_g$, etc.) across all tasks.
• Visual Quality: Qualitative results showed superior preservation of details, better handling of over-exposure in multi-exposure fusion, and clearer edge retention in infrared/visible fusion compared to SOTA methods like U2Fusion, UltraFusion, and DDcGAN.
• Efficiency:
  • Parameters: The model is extremely lightweight (0.015M parameters), significantly smaller than competitors (e.g., 556M for UltraFusion).
  • Speed: Inference time is 0.333ms, orders of magnitude faster than diffusion-based models (which take seconds to minutes).
  • FLOPs: Drastically lower computational cost compared to existing deep learning approaches.
• Ablation Studies: Confirmed that removing the "incomplete" nature (using complete priors like Curvelet or GFF) degraded performance. The modality perception mechanism was proven essential for suppressing over-exposure artifacts.

5. Significance

This paper fundamentally rethinks the paradigm of deep image fusion:

• Paradigm Shift: It moves away from the dependency on large-scale datasets and fixed priors. By treating the prior as a "guide" rather than a "target," it enables neural networks to learn fusion rules from minimal data.
• Practical Deployment: The combination of high performance, extreme lightweight architecture, and few-shot capability makes this approach highly suitable for edge devices and real-time applications where data collection is limited.
• Theoretical Insight: It bridges the gap between traditional algorithmic priors and deep learning by formalizing "uncertainty" and "confidence" within the learning process, offering a new theoretical perspective for designing fusion algorithms.

In summary, the authors demonstrate that Granular Ball Priors enable a general-purpose, highly efficient, and data-efficient deep fusion framework, solving the long-standing challenge of few-shot image fusion.