💻 computer science

PrivFusion: A Privacy-preserving Multi-Agent Framework for Harmonizing Distributed Datasets

PrivFusion is a privacy-preserving multi-agent framework that automates the harmonization of heterogeneous distributed clinical datasets through iterative feature alignment, thereby overcoming a critical barrier to effective Federated Learning.

Original authors: Anisa Halimi, Liubov Nedoshivina, Kieran Fraser, Stefano Braghin

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anisa Halimi, Liubov Nedoshivina, Kieran Fraser, Stefano Braghin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to organize a massive potluck dinner where everyone brings a dish, but no one is allowed to leave their own kitchen. The goal is to create a single, delicious menu that everyone can enjoy together. However, there's a catch: every cook uses different measuring cups, speaks different languages, and calls the same ingredients by different names. One person calls it "flour," another calls it "wheat powder," and a third lists it as "white dust."

This is exactly the problem hospitals and research institutions face when they want to combine their medical data to train Artificial Intelligence (AI) models. They can't just send their patient records to a central computer because of privacy laws. This is where Federated Learning comes in: it lets them train AI together without moving the data. But, as the paper explains, this system often fails because the data is too messy and inconsistent.

Enter PrivFusion, a new "digital matchmaker" designed to fix this mess before the cooking (training) even begins.

The Problem: The "Tower of Babel" of Data

The authors point out that while Federated Learning is great in theory, it hits a wall in reality. One hospital might record a patient's age as a number (e.g., "45"), while another writes it as text ("forty-five"). One might list a location as a city name, another as a GPS coordinate, and a third as a country code.

Traditionally, fixing this requires a team of humans to sit down and manually rewrite thousands of rows of data. It's slow, expensive, and often impossible to do without breaking privacy rules.

The Solution: A Team of Digital Agents

PrivFusion solves this by using a team of AI agents (think of them as specialized digital assistants) that work together in a privacy-safe way. Here is how the process works, step-by-step:

The Local Check-Up (The Analysts):
Instead of sending their actual patient data to a central server, each hospital sends a "summary report" generated by their own local AI. This report includes:
- What kind of data they have (numbers, dates, text).
- What the data means (e.g., "This column represents a date").
- A few synthetic samples. Imagine these as "dummy dishes" created in the kitchen. They look and taste like the real food (same format, same structure) but contain no real ingredients (no real patient names or secrets). These help the server understand the shape of the data without seeing the actual data.
The Central Matchmaker (The Server):
A central server receives these summary reports and the dummy samples. It doesn't see the real data; it only sees the "menu descriptions."
- Clustering: The server groups similar items together. It realizes that "Total Cases," "TotalPositiveCases," and "cases" all mean the same thing.
- Recommendations: The server acts like a head chef, sending back a list of instructions to each kitchen: "Change your column name to 'cases', convert your dates to this specific format, and ignore this one column that doesn't match anyone else."
The Transformation (The Cooks):
Each hospital takes these instructions and applies them to their own data locally. They update their own "menu" to match the new standard.
The Loop:
They send the updated summary back to the server. If the server sees they are still not quite aligned, it sends new instructions. This happens in a loop (usually just 2 or 3 times) until everyone is speaking the same language.

The Results: Speed and Privacy

The researchers tested this system using four real-world COVID-19 datasets from different countries (Afghanistan, Indonesia, Italy, and the US). These datasets were a mess of different formats and names.

Efficiency: The system managed to harmonize the data in just 2 to 3 rounds of communication.
Accuracy: It successfully aligned features like dates and location codes, turning a chaotic mix of formats into a clean, standardized set.
Privacy: Crucially, the paper claims that at no point did the server see the actual patient data. It only saw the "dummy" samples and metadata. The server couldn't figure out who the patients were, nor could it steal specific details about individuals.

The "Secret Sauce" (Large Language Models)

The system relies on advanced AI models (like GPT and Llama) to understand the meaning behind the words, not just the spelling. For example, it knows that "Date" and "yyyy-mm-dd" are the same concept, even if they look different. The paper found that different AI models had different personalities: some were very strict and followed rules perfectly, while others were more creative but sometimes made unnecessary changes.

The Bottom Line

PrivFusion is a tool that automates the tedious, privacy-breaking work of cleaning up medical data. It allows different institutions to "speak the same language" without ever having to share their secret recipes. The paper concludes that this makes it much easier to run large-scale, collaborative medical studies without compromising patient privacy, provided the AI agents are guided by the right rules.

Technical Summary: PrivFusion

Problem Statement

The rapid growth of clinical and biomedical data has driven the adoption of Machine Learning (ML) for tasks like risk prediction and cohort discovery. However, traditional centralized data aggregation is often infeasible for sensitive information (e.g., electronic health records) due to privacy, security, and governance constraints. While Federated Learning (FL) offers a distributed alternative by keeping data on-premises, its real-world deployment is hindered by data heterogeneity across institutions.

Clinical datasets frequently differ in feature definitions, coding systems, granularity, and measurement practices. Before distributed analysis can occur, these datasets must be harmonized—a labor-intensive process involving variable alignment and semantic resolution. Current FL methodologies often unrealistically assume harmonization is pre-completed, creating a major bottleneck. Existing ontology-driven solutions often require centralization or predefined mappings, which are difficult to establish across institutions with differing workflows. There is a critical need for automated, scalable, and privacy-preserving harmonization methods that operate in distributed environments without centralizing raw sensitive data.

Methodology: The PrivFusion Framework

PrivFusion is a privacy-preserving multi-agent framework designed to automate the harmonization of structured datasets prior to federated training. It operates in an honest-but-curious threat model where a central server follows the protocol but may attempt to infer sensitive information. The framework involves two parties: researchers (holding local datasets) and a server (aggregator).

The process is iterative and consists of four main stages:

Local Metadata Generation (Dataset Analyzer):
Each researcher locally analyzes their dataset ( $D_i$ ) using a suite of agents to generate metadata ( $M_i$ ) without sharing raw data. This includes:
- Feature Extraction: Identifying data types (e.g., string, numeric) and semantic types using a general-purpose ontology (DBPedia).
- Description Generation: Agents ( $A_D$ , $A_F$ ) generate dataset descriptions and semantic feature descriptions.
- Relationship Inference: An agent ( $A_R$ ) infers relationships between features, while another ( $A_T$ ) extracts relevant topics.
- Synthetic Data: Researchers generate $n$ low-utility synthetic samples using Differential Privacy (DP) approaches to guide the server on feature formats and domains.
Feature Clustering (Server Side):
The server receives metadata ( $M_i$ ) from all sites. An agent ( $A_C$ ) clusters features across datasets based on semantic similarity (names, data types, and DBPedia URIs). Features within the same cluster are identified as candidates for consolidation or alignment.
Transformation Recommendation (Server Side):
A recommendation agent ( $A_H$ ) analyzes the clusters and metadata to determine necessary transformations. It compares DBPedia URIs and semantic descriptions to converge on a representative URI and granularity level. The server outputs specific instructions for each site, detailing which features to combine, remove, or modify, and the target representation.
Local Transformation and Iteration:
Researchers apply the recommended transformations locally using Python scripts or a code-generation LLM ( $M_C$ ). The updated metadata is resent to the server. This loop continues until no further transformations are required or a maximum iteration count ( $T$ ) is reached.

Key Contributions

Automated Privacy-Preserving Harmonization: PrivFusion introduces a framework that automates the alignment of heterogeneous datasets across institutions without centralizing raw data, addressing a critical prerequisite for FL often overlooked in literature.
Multi-Agent Architecture: The system leverages specialized agents for distinct tasks (analysis, clustering, recommendation, code generation) to handle the complexity of semantic alignment and transformation logic.
Ontology-Driven Semantic Alignment: By utilizing DBPedia URIs and semantic descriptions, the framework moves beyond simple string matching to reason about the semantic meaning of features, enabling the alignment of conceptually equivalent but structurally different columns.
Synthetic Data Guidance: The use of low-utility synthetic samples allows the server to understand feature domains and formats while maintaining privacy guarantees, avoiding the leakage of individual-level statistics.

Evaluation and Results

The framework was evaluated on four real-world, heterogeneous COVID-19 datasets from Indonesia (IDN), Afghanistan (AFG), Italy (IT), and the US. These datasets exhibited significant differences in date formats, location granularity (e.g., county vs. province vs. country), and feature naming conventions.

Experimental Setup:

Models: The authors tested various combinations of Large Language Models (LLMs), including GPT-OSS-120b (GPT-120b), Llama-3-70b (Llama-70b), and DeepSeek-Coder-33b (DSC-33b).
Metrics: Convergence speed (number of iterations), total transformations recommended, and the number of common features (sharing name and DBPedia URI) post-harmonization.

Key Findings:

Efficiency: PrivFusion typically achieved harmonization within 2–3 iterations for pairwise dataset combinations. For example, the IDN-AFG pair converged in 2 iterations using GPT-120b.
Feature Alignment: The framework successfully standardized feature names (e.g., to snake_case), harmonized date formats, and aligned location granularities.
LLM Impact:
- GPT-120b demonstrated superior instruction adherence, often recognizing when further iterations were unnecessary and favoring incremental updates.
- Llama-70b showed higher creativity but occasionally misapplied normalization steps or introduced unnecessary changes (e.g., renaming features arbitrarily), leading to higher iteration counts in some configurations.
- DeepSeek-Coder produced correct code but occasionally generated incomplete responses requiring manual intervention.
Scalability: Complexity increased non-linearly with the number of parties. While pairwise harmonization required few transformations, multi-party scenarios (e.g., AFG-IDN-IT) required significantly more transformations (up to 49 in some configurations) and iterations.
Convergence: Jaccard Similarity scores between feature names generally peaked within 3–6 iterations, indicating rapid stabilization of the schema.

Significance and Limitations

Significance:
PrivFusion demonstrates that automated, privacy-preserving harmonization is feasible and effective. By reducing the manual burden of aligning datasets, it broadens access to collaborative analytics and improves the generalizability of resulting models. The framework successfully bridges the gap between the theoretical potential of Federated Learning and the practical reality of heterogeneous, siloed clinical data.

Limitations:

LLM Dependence: Performance relies on the reasoning consistency of the underlying LLM, which can vary. LLMs may occasionally over-align or under-align features.
Human-in-the-Loop: In high-stakes clinical settings, the authors acknowledge that human review of recommendations is likely necessary to ensure semantic accuracy and reliability.
Theoretical Guarantees: While the framework converged in experiments, it lacks theoretical guarantees regarding the number of iterations required for convergence.
Computational Cost: The multi-agent workflow can be computationally intensive, potentially posing challenges for resource-constrained institutions.

The paper concludes that while PrivFusion effectively harmonizes datasets with minimal transformations, future work will explore its application to additional domains and the evaluation of alternative LLMs to further enhance scalability.