🤖 AI

Additive Causal Construction for Transferable and Reconfigurable Cross-System Learning in Multi-Source Image Fusion

This paper proposes the Additive Causal Construction (ACC) framework and its learnable instantiation, ACC-CRL, to address cross-system discrepancy and entanglement in multi-source image fusion by establishing transferable causal anchors and reconfigurable fusion paths via uncertainty quantification, thereby significantly improving out-of-distribution generalization in tasks like medical imaging.

Original authors: Zhizhong Fu, Wei Zhou, Zhaoyang Jiang, Yulong Lin, Yifu Hou, Xiaorong Ding, Qiang Yan, Yifan Chen

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhizhong Fu, Wei Zhou, Zhaoyang Jiang, Yulong Lin, Yifu Hou, Xiaorong Ding, Qiang Yan, Yifan Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to bake the perfect cake by combining recipes from three different chefs. Chef A uses a gas oven, Chef B uses an electric one, and Chef C uses a wood-fired stove. They all claim to make "chocolate cake," but the way heat affects the batter is totally different for each.

If you just dump all three recipes into one bowl and hope for the best, you might end up with a disaster. You might accidentally rely on a trick Chef A used (like "add more sugar because the gas oven burns faster") that doesn't work at all in Chef B's electric oven. When you try to bake this "super-cake" in a new kitchen (a new environment), it falls apart.

This paper is about solving that exact problem, but instead of cakes, it's about combining medical images (like MRI scans) from different hospitals to predict a disease called Microvascular Invasion (MVI) in liver cancer.

Here is the simple breakdown of their solution, called ACC-CRL:

1. The Problem: "Fake Connections"

When doctors combine images from different sources (different machines, different hospitals, different patients), the computer often gets confused. It starts looking for fake shortcuts.

The Analogy: Imagine the computer learns, "Oh, if the image is slightly blue, it's a tumor!" because Hospital A's machines happen to make tumors look blue. But Hospital B's machines make tumors look gray. When the computer sees a gray tumor at Hospital B, it gets confused and fails.
The Paper's Terms: They call this Cross-System Entanglement (CSE). It's when the computer gets tangled up in unreliable, accidental patterns that only exist in one specific place.

2. The Solution: Building a "Universal Translator"

The authors propose a new way to mix these images. They don't just mash them together; they build a causal construction (a logical structure) first.

Step A: Finding the "Anchors" (CGT)

Before mixing, they ask: "What is the real thing we are looking for, regardless of the machine?"

The Analogy: They create a Universal Translator. Even though Chef A says "sugar" and Chef B says "sweetener," the translator knows they both mean the same causal ingredient for sweetness.
How it works: The system strips away the "machine noise" (like the blue tint) and finds the shared causal content (the actual tumor shape). It forces the computer to agree that "this shape" in Hospital A is the same "shape" in Hospital B, even if they look different. This solves Cross-System Discrepancy (CSD).

Step B: The "Trust Meter" (CGR)

Once the system has translated the images, it needs to decide: "Should I trust this new connection between the two images?"

The Analogy: Imagine you are mixing two soups. You have a Trust Meter. If the two soups taste similar and the ingredients make sense together, the meter goes up, and you mix them fully. But if one soup tastes weird or the ingredients clash (like adding chocolate to tomato soup), the meter goes down.
How it works: The system calculates Uncertainty. If the connection between the two images feels shaky or unstable (like a shortcut), the system says, "No thanks, I won't use that part." It dynamically decides how much of the second image to trust. If it's unreliable, it falls back to just using the first image safely. This prevents the "fake shortcuts" from ruining the prediction.

3. The Result: A Robust Doctor

They tested this on two things:

Fake Data (ColorMNIST): They created a game where the computer had to guess a number based on its color. The "easy" way was to just look at the color (a shortcut). Their method forced the computer to ignore the color and actually look at the shape of the number. When they changed the rules (new colors), the computer didn't crash; it kept working.
Real Medical Data (Liver Cancer): They took MRI scans from two different hospitals. One hospital's data was used to train the model, and the other (which had slightly different machines and patients) was used to test it.
- Old methods: Worked great on the training hospital but failed miserably on the new one.
- Their method (ACC-CRL): Worked well on the training hospital and stayed strong and stable on the new hospital. It didn't get confused by the differences in the machines.

Summary

Think of this paper as a recipe for mixing information without getting confused.

Old way: Throw everything in a blender. If the ingredients are slightly different, the smoothie tastes bad.
Their way: First, translate everything into a common language (Anchors). Second, use a Trust Meter to only mix the ingredients that actually belong together. If something looks suspicious, leave it out.

The result is a system that can learn from one hospital and still work perfectly in a completely different hospital, because it learned the real cause of the disease, not just the accidental tricks of the camera.

Technical Summary: Additive Causal Construction for Transferable and Reconfigurable Cross-System Learning in Multi-Source Image Fusion

1. Problem Statement

The paper addresses the challenge of multi-source image fusion under out-of-distribution (OOD) scenarios, particularly in medical imaging. The core issue arises when heterogeneous inputs (e.g., images from different centers, devices, or anatomical regions) are driven by distinct generative mechanisms. The authors identify two primary failure modes in existing fusion approaches:

Cross-System Discrepancy (CSD): The absence of a unified causal reference across systems makes semantic alignment difficult, leading to misaligned representations.
Cross-System Entanglement (CSE): The fusion process often introduces unstable dependencies and spurious causal pathways due to cross-system interference. These pathways are not robust to distribution shifts, causing significant performance degradation when models encounter data from different domains (e.g., different hospitals or scanners).

Existing methods typically rely on observational-level joint modeling with static constraints, treating causal structures as pre-defined and fixed. They fail to explicitly model the evolution of causal structures or the differences in generative mechanisms, rendering them inadequate for handling the dynamic nature of CSD and CSE.

2. Methodology: The ACC Framework and ACC-CRL

The authors propose the Additive Causal Construction (ACC) framework, which reframes multi-source fusion not as simple feature aggregation, but as a dynamic process of constructing a unified causal structure from heterogeneous systems. This framework is instantiated as a learnable model called ACC-CRL (Additive Causal Construction - Causal Representation Learning).

A. The ACC Framework Principles

The framework operates on two key pillars:

Causal Graph Transferability (CGT):
- Goal: Mitigate CSD by establishing shared causal "anchors" across systems.
- Mechanism: It defines an anchor-equivalent pair as nodes from different systems that map to a shared abstract causal content reference ( $X$ ) and induce consistent interventional responses.
- Requirements: This requires semantic comparability (mapping to a common content space) and interventional response consistency (ensuring that interventions on the shared anchor produce consistent downstream responses across systems).
Causal Graph Reconfigurability (CGR):
- Goal: Suppress CSE by treating fusion as a topological reorganization rather than a static merge.
- Mechanism: It introduces structural uncertainty to evaluate the credibility of candidate cross-system edges.
- Process: Candidate edges are selected, weakened, or suppressed based on their credibility (derived from mechanism compatibility and response consistency). This allows the system to adaptively regulate the fusion process, preserving stable causal relations while discarding unstable shortcuts.

B. The ACC-CRL Implementation

ACC-CRL realizes the ACC framework in a representation space through three core components:

Learnable CGT (Content-Mechanism Disentanglement):
- Observations are decomposed into shared causal content ( $X$ ) and system-specific mechanisms ( $U$ ).
- Contrastive Learning: A contrastive loss ( $L_{con}$ ) aligns semantically corresponding samples from different systems in the shared content space.
- Disentanglement: A disentanglement constraint ( $L_{dis}$ , using HSIC) prevents mechanism information from leaking into the shared content, ensuring $X$ captures stable causal semantics while $U$ preserves heterogeneity.
- Response Alignment: Bidirectional response mappings ( $\Psi$ ) are learned to reconstruct one system's output using the other's content and mechanisms. The residual error ( $e_{str}$ ) serves as a signal for structural mismatch.
Learnable CGR (Uncertainty-Aware Fusion):
- Uncertainty Estimation: The model computes structural uncertainty ( $u_{str}$ ) based on response alignment residuals and predictive uncertainty ( $u_{unc}$ ) based on mechanism representations and content discrepancies.
- Adaptive Gating: A fusion confidence weight ( $\gamma$ $γ$ ) is derived from the total uncertainty.
  - High $\gamma$ (low uncertainty): The model fuses representations from both systems.
  - Low $\gamma$ (high uncertainty): The model suppresses cross-system information, falling back to a conservative single-system representation to avoid unstable dependencies.
- Observation-Aware Reconfiguration: To prevent the loss of fine-grained visual cues (e.g., texture, boundaries) during latent abstraction, an observation-supplementary branch is added. The final representation dynamically balances the latent causal fusion and raw observation features based on the confidence weight $\gamma$ .
Joint Optimization:
The model is trained with a combined objective including supervised loss, contrastive loss, disentanglement loss, response alignment loss, and a regularizer ( $L_{uaf}$ ) that penalizes high fusion weights when structural mismatch is severe.

3. Key Contributions

Theoretical Formulation: The paper introduces the ACC framework, conceptualizing multi-source fusion as a dynamic causal construction process that explicitly addresses CSD and CSE through CGT and CGR.
Learnable Instantiation: It proposes ACC-CRL, a practical, end-to-end trainable architecture that implements mechanism disentanglement, interventional consistency alignment, and uncertainty-aware structural regulation.
Novel Mechanism: The method uniquely combines interventional response consistency (to define shared anchors) with structural uncertainty modeling (to regulate fusion reliability), moving beyond static observational constraints.
Observation Supplementation: It introduces a mechanism to dynamically balance latent causal abstraction with raw visual features, ensuring robustness in tasks where fine-grained details are critical.

4. Experimental Results

The authors validate the method on two datasets:

Synthetic Data (ColorMNIST):
- Setup: A controlled environment with modality bias (color vs. grayscale) to simulate spurious correlations (CSE).
- Findings: ACC-CRL significantly reduced directional bias and improved OOD generalization compared to baselines (Concatenation, Intervention, etc.). It demonstrated the ability to suppress shortcut pathways and learn separable, stable clusters in the representation space.
Real-World Medical Imaging (Microvascular Invasion - MVI Prediction):
- Setup: A cross-center prediction task using arterial-phase MRI from two hospitals (Sichuan Provincial People's Hospital and Huzhou Central Hospital). The task involves fusing intra-tumoral and peri-tumoral regions.
- Findings:
  - OOD Generalization: ACC-CRL maintained competitive In-Distribution (ID) performance while significantly outperforming baselines (MultiOOD, IRM, VREx, etc.) on the external OOD test set.
  - Robustness: The method achieved a better trade-off between ID and OOD performance (measured by BACC), showing smaller generalization gaps.
  - Ablation Studies: Confirmed that mechanism alignment (CGT) improves representation stability, while Uncertainty-Aware Fusion (CGR) is crucial for suppressing unstable cross-system dependencies under distribution shifts.
  - Analysis: Visualizations (t-SNE) showed that mechanism alignment led to better class separation and domain disentanglement. Uncertainty analysis confirmed that the model correctly assigned higher uncertainty to OOD samples and that uncertainty correlated with prediction errors.

5. Significance and Claims

The paper claims that the proposed ACC-CRL strategy offers a robust solution for multi-source fusion in open environments where distribution shifts are inevitable. By shifting from a static, observational perspective to a dynamic, causal construction perspective, the method:

Effectively mitigates Cross-System Discrepancy by establishing transferable causal anchors.
Suppresses Cross-System Entanglement by adaptively regulating fusion based on structural and predictive uncertainty.
Demonstrates that explicitly modeling the generation and evolution of causal structures leads to superior generalization compared to methods that assume fixed causal graphs.

The authors conclude that this approach is particularly valuable for medical applications (like MVI prediction) where data heterogeneity across centers is high, and reliable, interpretable decision-making is critical. They suggest future work will explore more complex multi-causal-system modeling and advanced structure learning techniques to further enhance clinical applicability.