Hospital Trust Aggregation for Multi-Hospital Medical Imaging under Non-IID Data and Malicious Clients: A Benchmark and Empirical Assessment

Technical Summary: Hospital Trust Aggregation for Multi-Hospital Medical Imaging

Problem Statement
Federated learning (FL) enables hospitals to train shared diagnostic models without exchanging patient records, addressing data privacy and regulatory constraints. However, real-world clinical consortia face two compounding challenges that undermine standard aggregation protocols like FedAvg:

1. Severe Non-IID Heterogeneity: Hospital cohorts differ significantly in referral patterns, case mixes, and cohort sizes. This creates a scenario where an honest hospital with a rare or unusual case mix produces model updates that are statistical outliers, not because they are malicious, but because their data distribution is unique.
2. Malicious Clients: A compromised hospital can steer the global model via data poisoning, backdoor attacks, or update manipulation (e.g., model replacement).

The core tension identified is that robust aggregation rules, which typically discard "unusual" updates to defend against attacks, struggle to distinguish between a legitimate outlier (an honest specialist center with rare cases) and a hostile outlier (a compromised node). Under severe heterogeneity, aggressive defense mechanisms risk excluding the very sites that hold the rare conditions the federation aims to diagnose.

Methodology: Hospital Trust Aggregation (HTA)
The authors propose Hospital Trust Aggregation (HTA), a multi-signal trust rule designed to evaluate hospitals over multiple rounds rather than in isolation. HTA computes a reputation score for each hospital based on three bounded signals (range [0, 1]), fused multiplicatively:

1. Directional Agreement ($s_h$): The rectified cosine similarity between a hospital's update and the coordinate-wise median of all updates (a robust consensus direction). Updates pointing away from the consensus receive zero credit.
2. Magnitude Plausibility ($m_h$): A symmetric penalty based on the logarithmic ratio of a hospital's update norm to the median update norm. This penalizes both inflated updates (model replacement) and shrunken updates (free-riding).
3. Reference-Cohort Validation ($v_h$): The server evaluates the hospital's local model on a small, curated, disjoint reference cohort ($D_{root}$). The signal is the balanced accuracy on this cohort, normalized against the round's median. This detects data-level attacks (e.g., label flipping) that preserve update direction/magnitude but degrade clinical utility.

Reputation and Exclusion Logic:

• Fusion: The signals are combined as $\tau_h = s_h^a m_h^b v_h^c$. The multiplicative nature ensures that a near-zero score in any single dimension suppresses the hospital's influence, preventing an adversary from compensating for a failure in one signal by excelling in another.
• Reputation Smoothing: The score is smoothed via an Exponential Moving Average (EMA) and penalized for its own variance. This discourages "alternating" strategies where an attacker behaves benignly in some rounds and maliciously in others to mask their mean score.
• Exclusion Criteria: A hospital is excluded only if it satisfies two conditions simultaneously:
  1. It is a statistical outlier (score $< \mu - \lambda \cdot \text{MAD}$).
  2. It is materially below the consensus (score $< \kappa \cdot \mu$).
     This dual condition is designed to prevent the exclusion of honest but atypical hospitals that are merely in the lower tail of a heterogeneous distribution.

Experimental Setup and Benchmark
The paper introduces a reproducible benchmark to evaluate HTA against seven baselines (FedAvg, FedProx, Coordinate-wise Median, Trimmed Mean, Krum, Multi-Krum, FLTrust).

• Datasets: DermaMNIST (7-class dermatoscopic diagnosis with severe class imbalance) and PneumoniaMNIST (binary chest radiography).
• Heterogeneity: Five regimes ranging from IID to severe label skew ($\alpha=0.1$) and quantity skew.
• Attacks: Six attacks including data-level (label flipping, backdoor) and update-level (sign flipping, scaling, ALIE, IPM), with adversary fractions from 5% to 30%.
• Metrics: Clinical (balanced accuracy, worst-class recall, per-hospital accuracy), Security (attack success rate), and Fairness.

Key Results

1. Heterogeneity vs. Adversaries: Heterogeneity is the dominant challenge. Moving from IID to severe label skew ($\alpha=0.1$) caused a 16.4-point drop in balanced accuracy, whereas adversaries caused a 7.2-point drop.
2. Performance of HTA:
   • Data-Level Attacks: HTA performs competitively, often leading or tying with the best baselines on label flipping and backdoor attacks.
   • Update-Level Attacks: HTA is significantly outperformed by Multi-Krum on sign flipping and scaling attacks (gaps of 4.9 and 6.7 points, respectively). Multi-Krum's geometric selection of a coherent subset proves more effective than HTA's reweighting strategy when update geometry is manipulated.
   • Scale: At 20 hospitals, HTA loses to Multi-Krum by 5.7 points.
   • Second Cohort: On PneumoniaMNIST, HTA underperforms FLTrust by 12–19 points across all conditions, attributed to the binary nature of the task reducing the discriminative power of the validation signal.
3. Clean Accuracy: Contrary to the common belief that robustness trades off clean accuracy, most robust rules (including HTA) achieved higher clean accuracy than FedAvg (e.g., HTA +4.6 points, Multi-Krum +5.9 points). This is attributed to variance reduction: robust rules dampen the noise from extreme, honest outliers in heterogeneous settings.
4. Ablation Findings:
   • Magnitude Signal: The update-magnitude signal ($m_h$) contributed no measurable benefit under either attack type. Removing it did not degrade performance. The authors conclude that under Dirichlet skew, an unusual norm is a reliable indicator of a rare honest case mix as often as it is of an attack, making this signal uninformative and potentially harmful.
   • Reputation & Direction: The reputation memory and directional agreement were the primary drivers of performance.
5. Detection Failure: Under severe heterogeneity and label flipping, the reputation scores for honest and malicious hospitals converged and overlapped completely. The rule did not successfully separate the two populations; its success was due to damping variance rather than detecting adversaries.

Significance and Claims
The paper positions itself not as a new state-of-the-art solution, but as a benchmark and a negative result.

• The Benchmark: It provides a rigorous, reproducible evaluation framework that integrates clinical metrics (balanced accuracy, fairness) with security metrics, revealing that defenses sacrificing minority-class recall are clinically useless.
• The Negative Result: The authors conclude that a trust signal based on update magnitude does not survive medically realistic heterogeneity. Furthermore, a reputation system built on these signals fails to distinguish honest from malicious hospitals under data-level attacks in heterogeneous settings.
• Mechanism Clarification: The observed benefits of HTA and other robust rules stem from variance damping (suppressing the noise of honest outliers) rather than adversary detection.
• Future Direction: The authors suggest that the effective component of their method is a reputation system driven by directional agreement with norm clipping, while the magnitude signal should be dropped. They emphasize the need for future work against adaptive adversaries who know the trust signals and for addressing the tension between per-update inspection and privacy-preserving techniques like secure aggregation.