LF2L: Loss Fusion Horizontal Federated Learning Across Heterogeneous Feature Spaces Using External Datasets Effectively: A Case Study in Second Primary Cancer Prediction

This paper proposes a Loss Fusion Horizontal Federated Learning (LF2L) framework that effectively integrates heterogeneous external SEER data with local Taiwanese hospital records to significantly improve the prediction accuracy of second primary lung cancer while preserving patient privacy and addressing feature inconsistencies.

The Gist

Imagine you are a detective trying to solve a very tricky case: predicting when a cancer survivor might develop a completely new, second type of cancer.

This is a life-or-death puzzle. The more data you have, the better your detective work becomes. However, in this story, the detective (the researchers) faces two major problems:

1. The "Small Notebook" Problem: The local hospital in Taiwan has a notebook with only about 10,000 patient stories. It's a good start, but not enough to see the big picture.
2. The "Foreign Language" Problem: There is a massive library of patient stories in the US (called SEER) with over 85,000 records. But, the US library uses different categories and labels than the Taiwan notebook. If you try to glue the two notebooks together, the pages don't match up, and the information gets messy or lost.

The Old Ways (And Why They Failed)

The researchers tried a few standard approaches, but they hit dead ends:

• The "Local Only" Approach: They just used the Taiwan notebook.
  • Analogy: It's like trying to learn to cook a complex dish by only tasting one spoonful of soup. You might get the basic flavor, but you'll miss the subtle spices that make it perfect. The model wasn't smart enough because it didn't have enough examples.
• The "Naive Merge" Approach: They tried to force the US and Taiwan data into one giant pile, filling in the missing blanks with "unknown."
  • Analogy: Imagine trying to build a house by smashing two different blueprints together. One blueprint says "put a window here," and the other says "put a door there." If you just smash them, you end up with a wall that has a window and a door in the same spot, or a hole where nothing should be. The data gets confused, and the house (the model) becomes unstable.
• The "Standard Teamwork" Approach (Federated Learning): They tried to let the two hospitals "talk" to each other without sharing the actual patient files.
  • Analogy: This is like two chefs trying to cook a meal together over a video call, but they can only agree on ingredients they both have. If the Taiwan chef has a special spice (like a specific gene mutation) and the US chef doesn't, they have to throw that spice away. They end up cooking a bland meal because they ignored the unique, powerful ingredients.

The New Solution: LF2L (The "Loss Fusion" Framework)

The researchers invented a clever new method called LF2L. Think of it as a Master Chef and an Apprentice working together in a way that respects their different kitchens.

Here is how it works, step-by-step:

1. The "Common Ground" Chat (Federated Learning):
   First, the two hospitals look at the features they both have (like age, gender, basic blood work). They use these common features to train a "Global Brain."

   • Analogy: The two chefs agree on the basic recipe steps they both know. They create a shared "flavor profile" of the soup.

2. The "Secret Sauce" (Local Features):
   Instead of throwing away the unique ingredients (like the special Taiwan gene mutations), the local hospital keeps them in its own private kitchen. It trains a "Local Brain" using its own unique data plus the "flavor profile" it got from the Global Brain.

   • Analogy: The Taiwan chef takes the shared flavor profile and adds their secret, special spice. They don't need to tell the US chef what the spice is; they just use it to improve their own dish.

3. The "Loss Fusion" (The Magic Glue):
   This is the secret sauce of the method. The system uses a special "scorecard" (called Loss) to measure how good the predictions are. It combines the score from the Global Brain and the Local Brain.

   • Analogy: Imagine a judge tasting the soup. The judge gives a score based on how well the soup tastes overall. If the Global Brain says, "This needs more salt," and the Local Brain says, "But I added my special spice, so it needs less," the system learns how to balance them perfectly. It doesn't force them to be the same; it teaches them how to work together to get the best result.

The Result: A Better Prediction

By using this method, the researchers didn't have to throw away any data. They got the size of the US dataset (more examples) and the specificity of the Taiwan dataset (unique medical details).

• The Outcome: The new model was significantly better at predicting second cancers than any of the old methods. It was more accurate, more reliable, and didn't violate patient privacy (because the actual patient data never left the local hospitals).

The Big Takeaway

This paper teaches us that in the world of medical AI, you don't have to choose between having a lot of data or having the right data.

Instead of forcing everyone to speak the same language (which loses information), we can build a system where different languages are translated into a shared "feeling" or "understanding," allowing everyone to contribute their unique strengths to solve the problem together. It's like a global orchestra where every musician plays a different instrument, but they all follow the same conductor to create a beautiful symphony.

Technical Summary

1. Problem Statement

The study addresses the challenge of predicting Second Primary Cancer (SPC) in lung cancer survivors. While improved survival rates have increased the population of cancer survivors at risk for SPC, early prediction remains difficult due to data limitations in local clinical settings.

• Data Scarcity & Generalizability: Local datasets (e.g., from Taiwanese hospitals) are often limited in sample size and geographic diversity, leading to models with poor generalizability and low statistical power.
• Feature Heterogeneity: Integrating external, large-scale datasets (like the US-based SEER program) is desirable to increase sample diversity. However, direct data merging is hindered by:
  • Privacy Constraints: Regulations prevent sharing raw patient data across institutions/countries.
  • Feature Inconsistency: Different sources collect different attributes (e.g., specific gene mutations available in one dataset but not the other). Naive merging requires imputation, creating sparse feature spaces and degrading model performance.
• Limitations of Standard Federated Learning (FL): Traditional Horizontal Federated Learning (HFL) requires all participating clients to share the exact same feature space, which is not the case here.

2. Methodology: The LF2L Framework

The authors propose Loss Fusion Horizontal Federated Learning (LF2L), a framework designed to collaborate across heterogeneous feature spaces without sharing raw data or forcing feature alignment.

Core Workflow:

1. Feature Grouping:

   • Common Features: Attributes shared across all datasets (e.g., age, gender, basic tumor stage).
   • Unique Features: Client-specific attributes (e.g., specific gene mutations like EGFR/ALK in the Taiwanese data, or specific demographic details in SEER).
   • Strategy: Each client retains both common and unique features for local training.

2. Stage 1: Global Federated Learning (HFL on Common Features):

   • Clients perform standard HFL using only the common feature set.
   • A centralized server aggregates model parameters to create a global model.
   • Embedding Extraction: The output embeddings from the last hidden layer of this global model are extracted. These embeddings serve as a compact, high-dimensional representation of the shared global context.

3. Stage 2: Localized Learning with Prune Net Guidance:

   • Main Net: Each client trains a local model using its full feature set (common + unique features).
   • Prune Net: A lightweight, single-layer neural network receives the global embeddings (from Stage 1) as input.
   • Loss Fusion: The total loss function is a weighted sum of two components:
     $$L_{total} = L_{local} + \beta \times L_{prune}$$
     • $L_{local}$: Loss from the Main Net (trained on local unique features).
     • $L_{prune}$: Loss from the Prune Net (guided by global embeddings).
     • $\beta$: A learnable parameter that dynamically balances the influence of the global context against the local features during backpropagation.

4. Prediction: The final prediction is derived solely from the Main Net, which has been optimized using both local specificities and global patterns.

3. Key Contributions

• Handling Heterogeneous Features: Unlike traditional HFL, LF2L does not require feature space alignment. It allows clients to utilize unique, domain-specific features that would otherwise be discarded or require harmful imputation.
• Privacy-Preserving Data Augmentation: The framework enables the integration of massive external datasets (SEER) to augment local models (Taiwanese hospitals) without exchanging raw patient data, satisfying strict privacy regulations.
• Loss Fusion Mechanism: The introduction of the "Prune Net" and the learnable parameter $\beta$ allows the model to effectively fuse global context (from common features) with local specificity (from unique features) in a unified optimization process.
• Superior Baseline Comparison: The study rigorously compares LF2L against localized learning, standard HFL, and centralized learning (naive merging), demonstrating that LF2L outperforms all baselines in handling multi-source medical data.

4. Experimental Results

The study utilized a Taiwanese dataset (10,545 lung cancer records) and the US SEER dataset (85,290 records). Experiments were repeated with 30 random seeds.

Performance Metrics (AUROC and AUPRC):

• Taiwanese Dataset Augmented with SEER:

  • vs. Localized Learning: LF2L significantly improved AUPRC ($0.1187$ vs. $0.1004$, $p < 0.001$).
  • vs. Standard HFL: LF2L significantly outperformed HFL in both AUROC ($0.7326$ vs. $0.7157$, $p < 0.05$) and AUPRC ($0.1187$ vs. $0.0953$, $p < 0.001$). This proves that utilizing unique features (via LF2L) is superior to restricting training to only common features (HFL).
  • vs. Centralized Learning: LF2L achieved a significantly higher AUROC ($0.7326$ vs. $0.6890$, $p < 0.05$) compared to naively merging datasets, highlighting the failure of imputation in centralized approaches.

• SEER Dataset Augmented with Taiwanese Data:

  • LF2L achieved the highest AUROC ($0.7337$) and AUPRC ($0.1373$), significantly outperforming both localized and standard federated baselines.

5. Significance and Conclusion

The paper demonstrates that LF2L is a robust solution for real-world clinical model development where data is siloed and heterogeneous.

• Clinical Impact: By effectively leveraging external data without compromising privacy or discarding critical local features (like specific genetic mutations), the model provides more accurate predictions for SPC, enabling timely clinical interventions.
• Methodological Impact: The study challenges the assumption that feature alignment is necessary for federated learning. It proves that a loss-fusion approach can successfully bridge the gap between diverse data sources, offering a scalable path for multi-institutional medical AI research.
• Generalizability: The framework is not limited to cancer prediction; it offers a blueprint for any domain requiring collaboration across institutions with disparate data schemas and strict privacy requirements.