🤖 machine learning

TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification

The paper introduces TIER-MoE, a trust-informed mixture-of-experts model that optimizes multimodal biomedical classification by routing samples to specialized experts based on learned modality-specific prediction risks and subspace compatibility, thereby achieving superior predictive performance and calibration across diverse disease datasets.

Original authors: Yu Chang, Anzhe Cheng, Chenwei Wu, Zhuoran Wang, Jiahao Chen, Tamoghna Chattopadhyay, Sophia I. Thomopoulos, Paul M. Thompson, Liyue Shen, Paul Bogdan

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yu Chang, Anzhe Cheng, Chenwei Wu, Zhuoran Wang, Jiahao Chen, Tamoghna Chattopadhyay, Sophia I. Thomopoulos, Paul M. Thompson, Liyue Shen, Paul Bogdan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of just one witness, you have a whole panel of them. Some witnesses are experts on the weather, others know everything about the local traffic, and a few are great at reading people's faces. In the world of medical science, doctors often face a similar puzzle: a patient's health isn't just one thing. It's a mix of brain scans, blood flow videos, and clinical notes. The goal of "multimodal fusion" is to combine all these different clues to get a better answer than any single clue could give alone.

However, there's a catch. Just because you have more witnesses doesn't mean you'll get a better story. Sometimes, a witness might be confused, or their expertise might not fit the specific crime you're investigating. If you blindly trust everyone equally, you might end up with a confused verdict. The big question scientists are asking is: How do we know which clue to trust for this specific patient, without throwing away the other clues that might still be useful? It's like trying to build the perfect team for a specific game, where you need to know who is in the best shape right now, rather than just picking the team with the most famous players.

This is exactly what the researchers behind a new method called TIER-MoE are tackling. They realized that current computer models often make a mistake: they assume that if a type of medical data (like an MRI scan) is generally good, it's good for every patient. But in reality, for one person, an MRI might be crystal clear, while for another, it might be blurry or misleading, even if the machine is the same.

To fix this, the team created a smart system that acts like a "trust meter" for every single piece of evidence. Instead of just guessing which clue is best, TIER-MoE asks a very specific question: "If we used only this one clue to predict the answer for this specific person, how likely would we be to make a mistake?" They call this the "conditional modality risk." It's like checking a weather forecast for a specific hour before deciding whether to bring an umbrella, rather than just checking if it rains often in that city.

Here is how their system works, using a fun analogy: Imagine a high-stakes quiz show where the host has a panel of specialized experts (like a brain specialist, a heart specialist, and a skin specialist). In the past, the host would just ask everyone to shout out their answers and mix them together. Sometimes, the "skin specialist" would give a wrong answer about a brain problem, dragging the whole team's score down.

TIER-MoE changes the rules. Before the experts speak, the host runs a quick, private test for each expert on this specific question. If the test shows the skin specialist is likely to get it wrong, the host doesn't just ignore them; instead, the host listens to them less and listens to the brain specialist more. But here's the clever part: even if the skin specialist is shaky, their answer might still have a tiny bit of useful info that helps the brain specialist. So, TIER-MoE keeps a "shared path" where all experts whisper their thoughts to each other, while a "specialist path" only lets the most reliable experts give their big, loud answers.

The researchers tested this idea on real medical data involving Alzheimer's disease, skin cancer, and eye conditions. They found that their method was much better at predicting the right diagnosis than previous methods. For example, on the Alzheimer's dataset, they improved the accuracy score (Macro-F1) to 0.648, beating the next best method by a noticeable margin. They also showed that their system was much better at knowing when it was unsure, which is crucial for medical safety.

Perhaps the most impressive part is that this "trust meter" worked even when they tested it on a completely new group of patients it had never seen before (a "zero-shot" test). The system didn't need to relearn anything; it just applied its logic of checking the risk of each clue and worked just as well.

The paper argues against the old way of thinking where we just average all the data together or rely on a single "best" type of data for everyone. They showed that simply trusting the "strongest" data source isn't enough because what is strong for one person might be weak for another. By combining a "risk check" with a flexible team of experts, TIER-MoE suggests we can build medical AI that is not only smarter but also safer, knowing exactly when to trust a clue and when to be careful.

Technical Summary: TIER-MoE

Problem Statement

Multimodal biomedical classification aims to improve disease diagnosis and prognosis by combining complementary evidence from diverse sources (e.g., MRI, DTI, clinical metadata). However, existing fusion methods face a fundamental trade-off between reliability and complementarity.

Current approaches often fail to distinguish between two distinct scenarios:

A modality that is genuinely unreliable for a specific subject.
A modality that is weak in isolation but provides unique, non-redundant information when fused.

Standard fusion architectures (e.g., gated models, attention mechanisms) tend to over-rely on globally dominant modalities, potentially discarding weaker but complementary evidence. Conversely, adaptive fusion methods that use uncertainty, quality, or confidence scores to weight modalities do not directly predict the expected prediction loss a specific modality will incur on a given sample. Furthermore, existing Mixture-of-Experts (MoE) models often decouple modality reliability from expert specialization, failing to route evidence to experts whose subspaces are best suited to process it.

Methodology: TIER-MoE

The authors propose TIER-MoE (Trust-Informed Expert Routing via Conditional Modality Risk), a hierarchical framework that integrates subject-specific modality risk estimation with expert-subspace affinity for sparse routing. The architecture consists of three primary components:

1. Cross-Fitted Conditional Modality Risk

The core innovation is defining sample-specific modality reliability not as a static quality score, but as the conditional expected prediction loss of a unimodal predictor for that specific subject.

Risk Definition: $R_m(V_i) = E[L_{i,m} | V_i]$ , where $L_{i,m}$ is the loss of the unimodal predictor for modality $m$ on subject $i$ .
Label-Honest Targets: To avoid data leakage, the risk targets are derived from Out-of-Fold (OOF) predictions. Models are trained on all subjects except the target subject, and their calibrated predictions generate the realized Brier loss used as the ground truth for risk estimation.
Estimator: A shared mask-aware neural network estimates this risk based on local evidence (calibrated distributions, uncertainty, quality indicators) and pooled context from all available modalities.
Recalibration: A low-capacity monotone map is fitted on the validation set to correct the shift between OOF estimates and the final refitted model's predictions.

2. Reliability Anchor

The estimated risks form a "reliability anchor" that generates a risk-weighted unimodal prediction ( $p^{rel}_i$ ). This branch ensures that dependable unimodal evidence is preserved and not discarded, acting as a baseline that prevents the fusion from performing worse than the best single modality.

3. Risk-Guided Subspace Expert Routing

Parallel to the reliability anchor, the model employs a sparse MoE mechanism to capture cross-modal interactions:

Shared Path: An always-active shared representation ( $h^{shared}_i$ ) is computed via masked self-attention, maintaining cross-modal interaction independent of routing decisions.
Expert Subspaces: A bank of $E$ experts is associated with low-rank subspaces ( $B_e$ ).
Compatibility Scoring: A compatibility score ( $\psi_{i,m,e}$ ) measures the affinity between a modality token and an expert's subspace.
Joint Routing Criterion: The routing decision combines compatibility with the conditional risk:
$G_{i,m,e} = \psi_{i,m,e} - \eta \hat{R}_{i,m}$
where $\eta$ controls the risk penalty. Routes are selected only if the score is positive, and the top- $K$ routes are chosen. This ensures that a moderately risky modality can still be routed if it aligns strongly with a suitable expert, while low-risk modalities are not forced into unsuitable experts.
Sparse Correction: Selected experts process their assigned modalities through low-rank residual transformations. These residuals are added to the shared representation to form the interaction prediction ( $p^{int}_i$ ).

4. Final Prediction

The final output is a convex combination of the reliability anchor and the interaction branch:
$p^{final}_i = (1-\gamma)p^{rel}_i + \gamma p^{int}_i$
where $\gamma$ is a validation-locked coefficient.

Key Contributions

Task-Grounded Reliability Formulation: The authors formulate modality reliability as the conditional expected prediction loss of a unimodal predictor, learned directly from subject-level OOF predictions. This provides a direct, evaluable measure of reliability against held-out errors.
Integrated Routing Mechanism: TIER-MoE introduces a routing strategy that jointly couples conditional modality risk with expert-subspace affinity. This design preserves reliable unimodal evidence while retaining complementary value from weaker modalities via a shared path and sparse expert residuals.
Comprehensive Evaluation: The framework is evaluated across four biomedical cohorts (ADNI, OASIS-3, PAD-UFES-20, FPRM) covering Alzheimer's disease, skin lesion malignancy, and retinal classification.

Results

Experiments demonstrate TIER-MoE's superiority over state-of-the-art baselines (including GMU, QMF, PDF, Flex-MoE, and I2MoE) across multiple metrics:

In-Domain Performance: TIER-MoE achieved the best Macro-F1 and Brier scores on all three in-domain datasets. On ADNI, it improved Macro-F1 by 3.4 percentage points over the strongest published comparator.
Risk Calibration: The estimated conditional risk showed superior agreement with realized unimodal Brier losses (lower ERCE, higher Spearman correlation) and better discrimination of high-loss cases compared to uncertainty or quality-based baselines.
Robustness: Under conditions of modality degradation, missingness, and cross-modal conflict, TIER-MoE maintained higher performance retention and significantly reduced "Harm Rate" (the proportion of incorrect fused predictions when at least one unimodal predictor was correct).
Zero-Shot Generalization: Without adaptation, TIER-MoE transferred effectively to the external OASIS-3 cohort, achieving the best balanced accuracy and probability calibration. Crucially, the learned risk semantics (calibration and discrimination) remained valid under this cohort shift.
Ablation Studies: Removing either the risk-guided routing or the OOF supervision resulted in measurable performance drops, confirming that both the risk-conditioned routing and the label-honest OOF supervision are necessary for the model's success.

Significance

The paper argues that modality reliability, cross-modal complementarity, and expert compatibility are distinct yet coordinated factors in multimodal fusion. By decoupling these factors and explicitly modeling them—using OOF-derived risk for reliability, subspace affinity for expert compatibility, and a shared path for complementarity—TIER-MoE addresses the limitations of existing fusion methods that conflate these concepts. The results suggest that defining reliability as expected prediction loss, rather than uncertainty or quality, provides a more robust foundation for adaptive fusion in heterogeneous biomedical data.