🤖 machine learning

TabKDE: Simple and Scalable Tabular Data Generation with Kernel Density Estimates

TabKDE introduces a highly scalable and efficient method for generating synthetic tabular data by combining copula transformations with kernel density estimates, achieving accuracy comparable to complex deep learning models while requiring negligible training time and storage space.

Original authors: Meysam Alishahi, Yan Zheng, Junpeng Wang, Chin-Chia Michael Yeh, Jeff M. Phillips

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Meysam Alishahi, Yan Zheng, Junpeng Wang, Chin-Chia Michael Yeh, Jeff M. Phillips

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, sensitive spreadsheet containing real customer data—things like their age, salary, education level, and whether they own a home. You want to share this data with researchers or developers so they can build better software, but you can't share the real data because it contains private information.

You need to create a "fake" version of this spreadsheet that looks and acts exactly like the real one, but where every single row is a new, made-up person who never actually existed. This is called Tabular Data Generation.

For the last few years, the best tools to do this have been like trying to paint a masterpiece using a super-complex, slow-moving robot arm (think Diffusion Models or VAEs). They produce great art, but they take hours to learn, require massive supercomputers, and often run out of memory if the spreadsheet has too many different categories (like thousands of different zip codes).

Enter TabKDE, a new method described in this paper. The authors propose a much simpler, faster, and lighter approach. Here is how it works, using everyday analogies:

1. The "Universal Translator" (Encoding)

First, the paper notes that spreadsheets are messy. Some columns are numbers (age), some are categories (education: High School, College), and some are ordered ranks (Grade: A, B, C).

The Old Way: Many methods try to turn every category into a long list of zeros and ones (like turning "High School" into 001, "College" into 010). If you have 10,000 categories, your list becomes 10,000 numbers long. This is like trying to carry a library in your pocket; it's too heavy and slows everything down.
TabKDE's Way: Instead of making a huge list, TabKDE uses a clever trick called Principal-Guided Encoding. Imagine you have a ruler made from the "vibe" of the numerical data (like salary). It places every category (like "High School") at a specific spot on that ruler based on how it usually relates to the numbers. Now, "High School" isn't a list of 10,000 zeros; it's just a single number on a line. This keeps the data compact and prevents the computer from running out of memory.

2. The "Sticky Note Map" (Copula Transformation)

Once everything is turned into numbers, the data is still in its original, messy shapes.

The Analogy: Imagine you have a pile of clay with different shapes. You want to flatten them all into perfect, identical squares so you can work with them easily, but you don't want to lose the relationships between the pieces (e.g., if two pieces were stuck together, they should stay stuck).
TabKDE's Way: It uses a Copula Transform. This is like a magical flattening machine. It squishes every column of data into a neat, standard range (from 0 to 1) while keeping the "stickiness" (correlations) between columns intact. Now, the data lives in a clean, uniform "unit square" where it's easy to measure distances.

3. The "Neighborly Wanderer" (Kernel Density Estimation)

Now comes the magic of creating new data.

The Old Way (Diffusion): Imagine trying to sculpt a new statue by starting with a block of noise and slowly chipping away at it for hours until it looks like a person. It's precise but incredibly slow.
TabKDE's Way: Imagine you have a map of where all the real people live (the training data). To create a new person, you don't sculpt from scratch. Instead, you:
1. Pick a random real person from your map.
2. Ask, "How far away is their closest neighbor?" (This is the Distance to Closest Record or DCR).
3. Take a step in a random direction, but make the step size match that typical "neighbor distance."
4. If you step outside the valid map boundaries (like a negative age), you just take a small step back inside.
This is Kernel Density Estimation (KDE). It's like saying, "New people usually live near old people, but not on top of them." It's incredibly fast because it doesn't need to "train" a complex neural network; it just learns the average distance between neighbors.

4. The "Pocket-Sized Model" (Coresets)

Usually, to remember a dataset, you need to store the whole thing.

TabKDE's Innovation: The paper introduces Coresets. Imagine you have a huge library of books, but you only need to remember the story of the library, not every single page. A coreset is a tiny, weighted selection of points that perfectly represents the whole library.
TabKDE can shrink its model down to a tiny fraction of the original data size (like storing a summary instead of the whole book) without losing much accuracy. This means you can run this on a simple laptop, even with massive datasets that would crash other systems.

The Results: Fast, Accurate, and Private

The paper compares TabKDE to the heavy-hitters (like TABSYN and TabDDPM):

Speed: While other methods take hours to train (and sometimes crash on large data), TabKDE trains in seconds or minutes. It can run on a standard laptop.
Accuracy: It produces fake data that is statistically almost identical to the real data. If you tried to train a machine learning model on the fake data, it would perform just as well as if it were trained on the real data.
Privacy: The goal is to make fake data that doesn't accidentally leak real people's info. The paper measures this by checking if the fake data is too close to the real data. TabKDE keeps a safe distance, ensuring it's creating new patterns rather than just copying and pasting real rows (a problem with older methods like SMOTE).

In short: TabKDE is a "simple and scalable" tool that turns messy, private spreadsheets into clean, fake, but statistically perfect versions, using clever math tricks to avoid the need for expensive supercomputers or hours of waiting.

Technical Summary: TabKDE

Problem Statement

Tabular data generation aims to produce synthetic rows that replicate the statistical distribution of an original dataset—including numerical, categorical, and ordinal features—without simply copying existing records. While recent advances in generative modeling (e.g., VAEs, GANs, and Diffusion models) have shown promise, they often suffer from significant limitations:

Scalability: Methods relying on one-hot encoding for categorical features explode in dimensionality, leading to memory exhaustion on datasets with many categories (e.g., the IBM dataset with ~37K categories).
Efficiency: Training complex models like Diffusion models or VAEs is computationally expensive, often requiring hours or days and specialized GPU hardware.
Privacy: Some non-parametric methods like SMOTE can inadvertently replicate training data points too closely, posing data leakage risks.
Interpretability: Latent space transformations in deep learning models often obscure the relationship between original features and generated samples.

Methodology: TabKDE

The authors propose TabKDE, a scalable, non-parametric approach that avoids deep learning training entirely. The method follows a three-step paradigm applied to tabular data:

1. Encoding ( $T \to E$ )

The method converts all features into a unified numerical format while maintaining a one-to-one mapping between original columns and latent dimensions (avoiding dimensionality explosion).

Numerical & Ordinal: Numerical features are z-score normalized; ordinal features are mapped to integers preserving rank order.
Categorical (Principal-Guided Encoding - PGE): Instead of one-hot encoding, PGE maps each categorical category to a single scalar value. This is achieved by:
1. Computing the first principal component (PC1) of the numerical features.
2. Projecting all data onto this PC1 direction.
3. Assigning each category a value equal to the average PC1 score of the rows containing that category.
  This preserves the dependency between categorical and numerical features without increasing dimensionality.

2. Copula Transformation ( $E \to Z$ )

The encoded data is mapped into a continuous latent space $Z \subset [0, 1]^d$ using a Copula transform.

For each feature $j$ , an empirical Cumulative Distribution Function (CDF) $F_j$ is constructed.
Each data point's value is transformed to $Z_{i,j} = F_j(E_{i,j})$ , resulting in a uniform distribution over $[0, 1]$ for each marginal.
This step decouples marginal distributions from the dependency structure and normalizes the data into a unit hypercube.
The sample covariance $\Sigma$ of $Z$ is computed to capture feature correlations.

3. Generative Modeling via KDE

Synthetic samples are generated using Kernel Density Estimation (KDE) in the latent space, guided by two key learned distributions:

Distance to Closest Record (DCR): The method learns the distribution of distances between data points and their nearest neighbors in the training set. This distribution, modeled as a mixture of Gaussians, determines the "radius" of perturbation for new samples, ensuring they reflect the natural spacing of the data without memorizing specific points.
Sampling Process:
1. Select a random seed point $z_i$ from the training data in $Z$ .
2. Sample a radius $r$ from the learned DCR distribution.
3. Sample a direction vector $u$ proportional to the covariance $\Sigma$ .
4. Generate a candidate point $z' = z_i + r \cdot u$ .
5. Boundary Correction: If $z'$ falls outside the unit hypercube $[0, 1]^d$ , an iterative rejection-sampling heuristic adjusts the out-of-bounds coordinates to ensure validity.
Decoding: The valid latent point is inverted through the Copula transform and the encoding map to return to the original tabular format.

Coresets for Efficiency

The authors introduce CORETABKDE and RANDCORETABKDE, which replace the full dataset with a small, weighted subset (coreset) of points to approximate the KDE. This significantly reduces storage requirements and model size while maintaining accuracy.

Key Contributions

Scalability: TabKDE handles datasets with high-cardinality categorical features (e.g., 37K categories) on standard consumer laptops, whereas state-of-the-art diffusion methods (like TABSYN) fail due to memory constraints from one-hot encoding.
Efficiency: The method requires almost no training time (seconds to minutes) compared to the hours required for training diffusion or VAE-based models. It achieves this by using classic statistical tools (Copulas, KDE) rather than iterative neural network optimization.
Privacy: By calibrating the perturbation radius using the DCR distribution and avoiding direct replication of training points, TabKDE mitigates data leakage. It performs significantly better than SMOTE in preventing the generation of near-duplicates of training data.
Interpretability: The one-dimensional-per-feature encoding ensures that each original column corresponds to a specific coordinate throughout the process, enhancing model interpretability.
Coreset Integration: The paper presents the first application of coresets specifically for generative tabular data modeling, allowing for compact model storage.

Experimental Results

The authors evaluated TabKDE on six UCI datasets and a large IBM credit card fraud dataset, comparing it against baselines including SMOTE, GReaT, CoDi, TabDDPM, TABSYN, and TABDIFF.

Accuracy:
- Marginal Alignment: TabKDE achieves a marginal alignment error (average across datasets) of 1.70, nearly matching the state-of-the-art TABSYN (1.42) and significantly outperforming SMOTE (2.49) and TabDDPM (14.52).
- Pairwise Correlation: TabKDE shows strong correlation alignment (average error 4.67), comparable to SMOTE and significantly better than most diffusion models, though slightly higher than TABSYN (2.21).
- Global Distribution (C2ST): TabKDE achieves a Classifier Two-Sample Test (C2ST) score of 0.938, indicating that classifiers struggle to distinguish synthetic data from real data, a result comparable to SMOTE (0.933) and slightly lower than TABSYN (0.970).
Efficiency:
- On a standard laptop CPU, TabKDE trains in ~43 seconds and samples in ~3 seconds on average. In contrast, TABSYN requires ~8.5 hours of training.
- On the IBM dataset, TabKDE completes in under 20 minutes (including training and sampling), while TABSYN and SMOTE run out of memory (OOM).
Privacy:
- TabKDE achieves a Distance to Closest Record (DCR) score of 58.6% (closer to the ideal 50% than SMOTE's 95.7%), indicating that synthetic points are not systematically closer to training data than to held-out data.
- The CORETABKDE variant further improves privacy with a DCR score of 53%, as it does not store exact training points.

Significance and Claims

The paper claims that TabKDE offers a new paradigm for tabular data generation that prioritizes scalability and efficiency without sacrificing accuracy or privacy. By leveraging copula transformations and kernel density estimates, the authors demonstrate that complex deep learning architectures are not strictly necessary for high-fidelity tabular synthesis.

The authors emphasize that TabKDE is particularly significant for scenarios requiring rapid iteration on sensitive data (e.g., finance and healthcare), where the computational cost of training diffusion models is prohibitive, and the risk of data leakage from methods like SMOTE is unacceptable. The ability to run on a simple laptop and handle massive categorical spaces makes the method accessible and practical for real-world deployment. Furthermore, the introduction of coresets for generative modeling provides a novel mechanism for reducing storage and computational overhead in future applications.