🤖 machine learning

Disjoint Generation of Synthetic Data

This paper proposes a novel framework for generating tabular synthetic data by partitioning datasets into disjoint subsets processed by separate generative models and recombined without common identifiers, a method that simultaneously enhances privacy, improves computational feasibility, and enables the use of mixed model types to achieve competitive utility while significantly reducing re-identification risks.

Original authors: Anton Danholt Lautrup, Muhammad Rajabinasab, Tobias Hyrup, Arthur Zimek, Peter Schneider-Kamp

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anton Danholt Lautrup, Muhammad Rajabinasab, Tobias Hyrup, Arthur Zimek, Peter Schneider-Kamp

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: "Divide and Conquer" for Data

Imagine you are trying to create a perfect, fake map of a bustling city. Usually, you would hire one giant, super-smart architect to look at the entire city—every street, every building, every traffic light—and draw the whole map at once.

The authors of this paper say: "What if we don't do that?"

Instead of hiring one giant architect, they propose hiring several smaller specialists.

Specialist A only looks at the roads.
Specialist B only looks at the buildings.
Specialist C only looks at the parks.

Each specialist draws their own part of the map in isolation. Then, a "glue master" (called a Joining Validator) tries to snap these separate pieces back together to form one complete city map.

This is the core of their new framework: Disjoint Generative Models (DGMs).

Why do this? (The Three Big Wins)

The paper argues that splitting the work up offers three specific advantages:

1. Better Privacy (The "Blindfold" Effect)
When one giant model sees everything (roads, buildings, and parks all at once), it might accidentally memorize specific details about a real person, like "The blue house on 5th Street has a red door." If someone steals the fake map, they might find that specific person.
But when you split the data, Specialist A only sees roads, and Specialist B only sees houses. Neither of them knows the full story. When you put the pieces back together, it's much harder to accidentally recreate a specific real person's exact address. It's like trying to guess a secret password when you only have half the letters; the risk of guessing the whole thing drops significantly.

2. Faster and Easier Computing (The "Assembly Line")
Some computer models are like heavy, slow trucks. If you try to drive a truck through a narrow alley (a dataset with hundreds of columns/variables), it gets stuck.
By splitting the data into smaller chunks, you can use smaller, faster cars (lighter models) for each part. You can even run these cars on different tracks at the same time (parallel processing). The paper found that for certain types of complex models, this made the whole process much faster and less likely to crash.

3. Mixing and Matching Tools (The "Best Tool for the Job")
Sometimes, one tool isn't good at everything. Maybe a "Road Specialist" is great at drawing straight lines but bad at drawing curves. Maybe a "Building Specialist" is great at curves but bad at straight lines.
In the old way, you had to pick one tool and hope it was good at everything. With this new method, you can use a "Road Specialist" for the roads and a "Building Specialist" for the buildings. You get the best of both worlds without forcing one model to be a jack-of-all-trades.

How Do They Put It Back Together? (The "Glue Master")

This is the trickiest part. If you just randomly glue the road map to the building map, you might end up with a skyscraper floating in the middle of a highway. That's a useless fake map.

The paper introduces a Joining Validator. Think of this as a strict quality control inspector.

The specialists send their fake pieces to the inspector.
The inspector tries to glue them together.
The inspector asks: "Does this combination look real? Do the roads actually connect to the buildings?"
If the answer is Yes, the inspector keeps that piece.
If the answer is No, the inspector throws it away and tries a different combination.

The paper shows that this "quality control" step is crucial. Without it, the fake data is either too messy (low utility) or too risky (low privacy). With it, they found a "sweet spot" where the data is useful for testing software but safe for people's privacy.

What Did They Actually Test?

The authors didn't just talk about this; they tested it on real-world data tables (like medical records for heart disease, cancer, and diabetes).

The Results: They found that as they split the data into more pieces, the fake data became safer (harder to identify real people) but slightly less accurate (a bit harder to use for predictions).
The Fix: However, when they used their "Glue Master" (the Validator) to carefully select the best combinations, they recovered most of that lost accuracy.
The Winner: The best results came from Mixed-Model Generation. This means using a high-privacy model for the sensitive parts of the data (like a patient's name or ID) and a high-accuracy model for the less sensitive parts (like age or blood pressure). This combination gave them the best balance: data that was very useful for scientists but very hard to hack.

The Bottom Line

This paper proposes a new way to make fake data. Instead of trying to build a perfect fake world in one giant, risky step, they suggest building it in small, safe pieces and then carefully assembling them.

They proved that this "divide and conquer" approach:

Makes the data safer from privacy leaks.
Makes the process faster for computers.
Allows you to mix different AI tools to get the best results.

The authors emphasize that while this makes the data safer, the process of assembling it still needs careful tuning to ensure the final result is both useful and secure. They have made their code available for others to try this "divide and conquer" method on their own data.

Technical Summary: Disjoint Generation of Synthetic Data

Problem Statement

The generation of high-quality synthetic tabular data faces a persistent trade-off between utility (fidelity to the original distribution and performance on downstream tasks) and privacy (protection against re-identification). Traditional centralized workflows, which aggregate data to train a single large generative model, often struggle to capture high-dimensional structures without overfitting, or they fail to provide sufficient empirical privacy guarantees. While Differential Privacy (DP) offers formal guarantees, it frequently introduces noise that degrades utility. Furthermore, existing federated or distributed approaches generally achieve performance comparable to, but rarely exceeding, centralized counterparts.

A specific challenge arises when data is naturally partitioned (e.g., vertical slices with no shared identifiers) or when computational constraints prevent training a single model on all features. Current methods often lack a robust framework for generating synthetic data from disjoint subsets of variables and reassembling them without losing the underlying cross-variable dependencies.

Methodology: Disjoint Generative Models (DGMs)

The authors propose Disjoint Generative Models (DGMs), a framework that applies a "divide and conquer" paradigm to synthetic data generation. The core procedure involves three stages:

Partitioning: The training dataset $X$ (an $n \times k$ matrix) is partitioned column-wise into $n_p$ disjoint subsets of variables ( $d_1, d_2, \dots, d_{n_p}$ ). These partitions contain no shared variables.
Disjoint Generation: Each partition is fed into a separate generative model instance ( $G_p$ ). Crucially, this allows for mixed-model generation, where different model architectures (e.g., a Bayesian Network for one partition and a GAN for another) can be used simultaneously based on the specific strengths of the data subsets.
Joining Operation: The independent synthetic partitions ( $s_1, s_2, \dots, s_{n_p}$ $s_{1}, s_{2}, \dots, s_{n_{p}}$ ) are combined post-hoc. The paper focuses on a specific joining strategy using a Validator Model ( $V$ $V$ ):
- Training the Validator: A binary classifier (e.g., Random Forest) is trained on the original data. The positive class consists of authentic rows, while the negative class consists of "fake" rows created by randomly shuffling and concatenating the disjoint partitions of the real data.
- Inference: Synthetic partitions are randomly concatenated to form candidate rows ( $q$ ). The validator assigns a score ( $z$ ) to each candidate.
- Selection: Candidates scoring above a threshold $\theta$ are accepted into the final synthetic dataset $S$ . Accepted rows are removed from the pool, and the remaining partitions are reshuffled to generate new candidates. This loop continues until the target dataset size is reached or iteration limits are hit.

Key Contributions

Framework Proposal: Introduction of the DGM framework, which challenges the assumption that a single monolithic model is always superior, demonstrating that partitioning can improve computational feasibility and privacy.
Mixed-Model Synthesis: A novel capability to combine different generative model types (e.g., high-utility models like TabDiff or synthpop with high-privacy models like DPGAN) within a single synthetic dataset, leveraging the strengths of each for specific variable subsets.
Joining Validator: A mechanism to reassemble disjoint synthetic data by learning cross-partition dependencies, rather than relying on simple random concatenation or shared identifiers.
Empirical Evaluation: A comprehensive benchmark across seven standard tabular datasets (ranging from 126 to 3,927 records) and one high-dimensional dataset (98 features), evaluating utility (PCA, correlation, F1, AUROC) and privacy (identifiability risk, Membership Inference Attack recall).

Results

The experiments yield several key findings:

Privacy-Utility Trade-off: Increasing the number of partitions generally improves empirical privacy metrics (lower identifiability risk and MIA recall) but degrades utility metrics (higher correlation matrix difference, lower F1/AUROC).
Role of the Validator: Using the joining validator significantly mitigates the utility loss associated with partitioning compared to random concatenation. While random concatenation preserves marginal distributions well, it fails to capture cross-partition dependencies, leading to poor downstream task performance. The validator restores some of this utility at a slight cost to the maximum privacy gain.
Mixed-Model Performance: Mixed-model DGMs (e.g., combining synthpop and DPGAN) achieve a superior balance. They often outperform single-model baselines in utility while maintaining privacy levels comparable to or better than high-privacy single models. For instance, a synthpop-DPGAN combination achieved competitive AUROC and F1 scores while reducing re-identification risk significantly compared to high-utility baselines.
Partitioning Strategy: The effectiveness of the joining process depends heavily on the correlation structure between partitions. High inter-partition correlation makes the validator's job easier and improves utility, whereas low correlation favors simple concatenation but offers less privacy benefit from the joining process.
Computational Efficiency: Partitioning reduces training time for models that scale poorly with dimensionality (e.g., Bayesian Networks), making them viable for high-dimensional datasets where a single model would be computationally prohibitive.

Significance and Claims

The paper positions DGMs as a practical alternative for scenarios where data cannot be pooled or where a single model cannot effectively balance privacy and utility. The authors claim that:

Empirical Privacy Gains: The framework offers a tangible increase in empirical privacy (measured by re-identification risk) without the severe utility degradation often seen in formal DP approaches.
Flexibility: The ability to mix models allows practitioners to tailor the generation process to specific data characteristics (e.g., using DP models for sensitive columns and high-fidelity models for others).
Modesty on Guarantees: The authors explicitly state that the privacy improvements are empirical, not formal. The framework does not currently provide end-to-end Differential Privacy guarantees, as the joining validator is exposed to real data and the constituent models' DP guarantees may be voided by the reassembly process.
Future Directions: The work opens avenues for exploring alternative joining mechanisms (e.g., Bayesian methods, hashing), optimizing partitioning strategies, and extending the framework to multi-modal data (images, text, time-series).

The paper concludes that while DGMs do not solve all privacy challenges, they offer a viable, flexible, and computationally efficient route to generating synthetic tabular data that better balances the competing demands of utility and privacy.