🤖 machine learning

FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching

This paper introduces FUSE, a novel framework for generating mixed-type tabular data that explicitly separates feature-specific processing from cross-column interactions via adaptive mixture modules and joint attention, thereby improving distributional fidelity and downstream utility while providing theoretical bounds on generation error.

Original authors: Suman Cha, Seongchan Lee, Dohyun Ko, Hyunjoong Kim

Published 2026-08-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Suman Cha, Seongchan Lee, Dohyun Ko, Hyunjoong Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master chef trying to recreate a complex, multi-course meal based only on a blurry photo of the final dish. You have to figure out not just the ingredients, but how they taste together, how the texture of the steak affects the perception of the sauce, and how the spices in the soup influence the overall vibe of the dinner. This is the daily challenge for computers trying to generate "synthetic data"—fake but realistic information that looks and acts just like the real stuff. In the world of machine learning, this is crucial because real data is often private (like medical records) or just too scarce to go around.

The specific type of data this paper tackles is called "mixed-type tabular data." Think of a spreadsheet. Some columns are numbers (like age or income), some are categories (like "red," "blue," or "green"), and some are counts. The tricky part isn't just making a fake number that looks right; it's making sure the fake numbers and fake categories still have the right relationships. If the real data shows that people with high income tend to buy luxury cars, your fake data must respect that rule. If you get the relationships wrong, the fake data is useless for training AI or testing policies. For a long time, computers struggled to handle this mix without either getting the individual numbers wrong or messing up the connections between them.

Enter FUSE, a new method introduced by researchers Suman Cha, Seongchan Lee, Dohyun Ko, and Hyunjoong Kim. Their goal was to build a better "chef" for this specific kind of data. They noticed that previous methods were like a kitchen where every ingredient was forced through the same single blender, or where the ingredients were processed in completely separate rooms with no one talking to each other. Both approaches failed to capture the unique personality of each ingredient while keeping the group dynamic intact.

FUSE solves this with a clever two-part strategy, which the authors call "Feature-Wise Unified Specialization with Cross-Column Exchange." Imagine a team of specialized chefs. First, there's a Specialization Station. Here, the "number" ingredients (like age) go to a team of chefs who only know how to handle numbers, and the "category" ingredients (like color) go to a different team of chefs who only know categories. But here's the twist: instead of each ingredient getting its own private chef, they share a pool of expert sub-chefs. Each ingredient gets to pick and choose which experts it wants to work with, mixing and matching their skills in a unique way. This ensures that a skewed number gets treated differently than a normal one, and a rare category gets handled with care.

However, a kitchen where chefs never talk is a disaster. That's why FUSE adds a Cross-Column Exchange step. After the specialized teams do their work, everyone gathers around a big round table for a "joint attention" meeting. Here, the number-chefs and category-chefs swap notes. They look at the whole picture, ensuring that the relationship between a person's age and their favorite color is preserved. This allows the system to learn complex dependencies without losing the unique identity of each data column.

The researchers tested FUSE on eight different real-world datasets, ranging from customer shopping habits to medical records. They compared it against seven other top-tier methods, including some that use complex diffusion models and others that use flow-based techniques. The results were impressive. In terms of how closely the fake data matched the real data's shape and relationships, FUSE consistently ranked at or near the top. For instance, on a dataset called "Adult," FUSE achieved a shape score of 0.993, beating out competitors like TabbyFlow (0.993, tied) and TabSyn (0.979). On the "News" dataset, it scored 0.988, significantly outperforming TabDDPM, which struggled with a score of 0.187.

The paper also dug into why this worked so well. They ran experiments where they removed parts of the FUSE system to see what happened. When they took away the "joint attention" (the group meeting), the data lost its ability to capture relationships between different types of variables. When they removed the "adaptive mixture" (the specialized chefs), the data became less accurate in its details. The study suggests that the magic of FUSE comes from having both: specialized processing for unique features and a shared space for them to communicate.

The authors also provided a theoretical safety net for their method. They showed mathematically that if the system makes a mistake in predicting the final data point, that mistake translates into a predictable amount of error in the final generated data. This gives them confidence that their method isn't just lucky; it's built on solid ground.

In the end, FUSE doesn't claim to have solved every problem in data generation, but it offers a highly effective, consistent way to handle the messy reality of mixed data types. By letting features be themselves while ensuring they stay connected, FUSE creates synthetic data that is not only statistically sound but also useful for training the next generation of AI tools. As the researchers demonstrated, when you give your data chefs the right tools and the right way to talk to each other, the meal turns out deliciously real.

Technical Summary: FUSE for Mixed-Type Tabular Flow Matching

Problem Statement

Generating synthetic mixed-type tabular data (containing both numerical and categorical variables) presents a dual challenge: accurately modeling diverse feature-specific distributions while capturing complex cross-column dependencies. While recent adaptations of Flow Matching, such as Exponential Family Variational Flow Matching (EF-VFM), successfully handle distinct data types by factorizing endpoint distributions, they often rely on a shared neural backbone. This architecture lacks an explicit mechanism for feature-dependent transformations, potentially failing to capture the unique statistical structures (e.g., skewness, tail behavior, cardinality) of individual columns while simultaneously preserving inter-feature dependencies.

Methodology: FUSE Architecture

The authors propose FUSE (Feature-Wise Unified Specialization with Cross-Column Exchange), an architecture designed to explicitly separate feature-specific processing from cross-column information exchange within a variational flow matching framework.

1. Adaptive Mixture Processing

To address feature-specific statistical variations, FUSE employs separate adaptive mixture modules for numerical and categorical features.

Mechanism: For each feature type, the module utilizes $M_r$ parallel subnetworks to process $S_r$ latent components.
Feature-Dependent Recombination: Using a differentiable aggregate-transform-recombine operator, the model computes alignment scores to generate two sets of weights:
- Aggregation weights: Determine how feature representations are aggregated into latent components.
- Recombination weights: Determine how processed components are combined to update each feature representation.
Benefit: This allows each feature to form a distinct combination of shared specialized subnetworks. Consequently, the model achieves type-specific specialization without requiring a separate subnetwork for every feature, balancing parameter efficiency with representational capacity.

2. Joint Attention

To preserve dependencies across different feature types (e.g., between numerical and categorical columns), FUSE integrates a joint self-attention mechanism.

Mechanism: After adaptive mixture processing, the type-specific token representations are concatenated and passed through a multi-head self-attention layer.
Benefit: This enables every feature token to incorporate information from all other columns, regardless of type. This ensures that the conditioning context for each endpoint predictor is unrestricted, allowing for the capture of multivariate dependencies that separate processing paths might miss.

3. Endpoint Prediction

The final layer maps the processed tokens to the parameters of the factorized endpoint distributions defined in EF-VFM:

Numerical endpoints are mapped to Gaussian means.
Categorical endpoints are mapped to categorical probability vectors.
These parameters define the velocity field used to transport the source distribution to the data distribution.

Theoretical Contributions

The paper provides a theoretical characterization of the risks associated with architectural restrictions in flow matching:

Conditioning Penalty: The authors derive the excess population risk induced by restricting the conditioning context. They show that if a restricted context omits features containing residual endpoint information, a positive penalty arises in the form of increased KL divergence (for categorical) or mean squared error (for numerical).
Architectural Bottlenecks: The analysis distinguishes between two types of bottlenecks:
- Conditioning Bottleneck: Arises when cross-type information is withheld (addressed by Joint Attention).
- Representation Bottleneck: Arises when features are forced to share a single representation without the ability to form distinct combinations (addressed by Adaptive Mixture Processing).
Error Bounds: The paper establishes a bound on the continuous-space Wasserstein generation error in terms of the endpoint-prediction risk, linking the quality of the generative process to the accuracy of the conditional endpoint inference.

Experimental Results

FUSE was evaluated on eight tabular datasets (including Adult, Default, and Diabetes) against seven baselines: CTGAN, TVAE, CoDi, TabDDPM, TabSyn, TabDiff, and TabbyFlow.

Distributional Fidelity (Shape & Trend): FUSE achieved the best average rank (1.25) for marginal distribution similarity (Shape) and demonstrated high consistency in preserving pairwise feature dependencies (Trend), outperforming the runner-up TabbyFlow on most datasets.
Discriminative Power (C2ST): FUSE achieved the best average rank (1.62) in the Classifier Two-Sample Test, indicating that discriminators struggle to distinguish FUSE-generated data from real data, suggesting strong multivariate fidelity.
Downstream Utility (MLE): In terms of Machine Learning Efficiency, FUSE achieved the best average rank (2.62) for downstream tasks (classification AUC and regression RMSE), demonstrating that the synthetic data is highly useful for training predictive models.
Ablation Studies: Component analysis confirmed that both Joint Attention and Adaptive Mixture Processing are complementary. Removing Joint Attention significantly degraded dependency preservation and downstream utility, while removing Adaptive Mixture Processing reduced marginal fidelity and coverage.

Significance and Claims

The paper claims that FUSE resolves the architectural trade-off between parameter sharing and feature-specific computation in mixed-type tabular generation. By explicitly separating feature-dependent transformations (via adaptive mixtures) from cross-column information exchange (via joint attention), FUSE avoids the implicit limitations of shared backbones.

The authors assert that their work:

Introduces a novel architecture that combines type-specific adaptive processing with joint attention for flow matching.
Quantifies the theoretical cost of restricted conditioning and shared representations, providing a theoretical justification for their architectural choices.
Demonstrates through comprehensive experiments that this architecture achieves strong and consistent performance across distributional fidelity, dependency preservation, and downstream utility metrics, outperforming existing state-of-the-art methods on mixed-type tabular data.