Hierarchical Synthetic Tabular Data Generation: A Hybrid Top-Down and Bottom-Up Framework

Technical Summary: Hierarchical Synthetic Tabular Data Generation

Problem Statement
Current approaches to synthetic tabular data generation face significant limitations when applied to complex, enterprise-scale scenarios. Existing methods generally fall into two categories: purely generative models (e.g., CTGAN, TVAE, TabDDPM) and Large Language Model (LLM)-based approaches (e.g., GReaT, REaLTabFormer).

• Generative Models: While effective at approximating distributions, they often require substantial training data, struggle with heterogeneous feature types, and suffer from mode collapse. Consequently, they frequently fail to capture rare events and tail distributions, which are critical in domains like fraud detection. In low-data regimes, these models tend to overfit and produce low-diversity samples.
• LLM-Based Methods: These leverage in-context learning and semantic priors but often fail to enforce structural consistency, logical constraints, or functional dependencies among attributes. Autoregressive token generation can lead to invalid or incoherent samples, and sequential row tokenization weakens the modeling of heterogeneous numerical-categorical interactions.
• The Core Challenge: A deeper, often overlooked issue is that synthetic data generation is frequently treated as a purely generative problem, ignoring the highly structured, heterogeneous, and multi-modal nature of real enterprise data. Relying solely on model-generated data introduces risks of distribution collapse, where recursive training on synthetic outputs progressively erodes the underlying data distribution, particularly in the tails.

Methodology: The H-TDBU Framework
The authors propose a Hierarchical Hybrid Top-Down and Bottom-Up (H-TDBU) framework. This approach decouples semantic structures from stochastic texture, integrating generative capabilities with hierarchical rule construction. The framework operates through three distinct phases:

1. Top-Down Path (Structure):

   • Goal: Define the logical skeleton and semantic structure of the data.
   • Mechanism: Structure rules are defined via human instructions or LLM prompts. This creates a structural template $S$ that acts as a backbone, breaking down abstract factors into concrete, sampleable attributes.
   • Role of LLMs: LLMs are used only in this stage to generate the scheme representing structure rules (e.g., mapping target variables to sentiment text). They do not generate the actual tabular rows.

2. Bottom-Up Path (Texture):

   • Goal: Learn local statistical patterns and realistic "texture" from real data.
   • Mechanism: Lightweight, low-compute tabular generators (e.g., ensemble methods like Random Forests or XGBoost, or generative models like CTGAN) are trained on real data ($D_{real}$).
   • Output: These models produce a latent space representation ($Z$) that captures detailed, complex patterns resembling real data but may not inherently follow the logical rules defined in the top-down phase.

3. Synthesis and Reconciliation:

   • Goal: Unify the logical skeleton ($S$) with the latent texture ($Z$).
   • Mechanism: A unified synthesis engine, denoted as $X_{Syn} := G(z \in Z | S)$, generates data from latent noise consistent with the logical constraints of $S$. This conditional generator reconciles the two paths.
   • Feedback Loop: The system employs an iterative feedback loop. If the synthetic data ($D_{syn}$) fails cross-modal fidelity metrics ($X_{Modal}$), it triggers a "Retrain Model" signal. If it fails constraint validation, it triggers an "Adjust Constraints" signal to the Top-Down provider. This ensures logical integrity and prevents model drift.

Key Contributions

• Decoupling of Structure and Texture: The paper introduces a framework that separates the definition of semantic/logical rules from the generation of statistical texture, addressing the limitations of monolithic generative models.
• Hybrid Architecture: It organically integrates the generative power of lightweight agents (Random Forest, XGBoost) with the structural reasoning of LLMs, using LLMs strictly for rule definition rather than direct data generation.
• Iterative Feedback Mechanism: The inclusion of a reconciliation loop that validates cross-modal metrics and logical constraints allows for dynamic refinement of both the top-down rules and bottom-up models.
• Weak Multimodal Benchmarking: The authors evaluate the framework on weak multimodal financial benchmarks combining tabular data with sentiment-text data, a scenario where existing methods struggle with alignment and consistency.

Experimental Results
The framework was evaluated on four benchmarks: two weak multimodal benchmarks (Manual and Gemini-generated) and two tabular-only benchmarks (Adult Income and German Credit).

• Downstream Utility (TSTR): In the manually specified weak multimodal benchmark, tree-based conditional methods (RandomForest and XGBoost) outperformed neural baselines (CTGAN, TVAE). For instance, RandomForest achieved an AUROC of 0.9188 and F1 of 0.6122, compared to TVAE's 0.8827 AUROC and 0.5581 F1.
• Strict Alignment (Gemini Benchmark): When using an LLM-generated rule provider that enforced stricter alignment (Target 1 aligned only with positive text), conditional generators showed significant improvement. RandomForest achieved near-perfect metrics (0.9998 AUROC, 0.9878 F1), outperforming neural baselines which also improved but remained slightly lower.
• Cross-Modal Fidelity: A trade-off was observed between downstream utility and cross-modal fidelity ($X_{Modal}$). While RandomForest excelled in utility, Gaussian Copula and TVAE sometimes achieved lower $X_{Modal}$ values (better distributional alignment of text and tabular targets).
• Ablation Studies: Results indicated that performance is driven by the interaction between the number of conditioning columns and the strictness of the top-down rules. The Gemini-generated rules (stricter) benefited from fewer conditioning columns, while the manual rules (more permissive) benefited from richer conditioning contexts.

Significance and Claims
The paper claims that hierarchical rule-guided synthesis provides an effective mechanism for balancing controllability, semantic coherence, and statistical fidelity in synthetic data generation.

The authors emphasize that their approach does not rely on expensive, recursive LLM generation for every data point. Instead, by using LLMs only for the top-down construction of the "coverage space" and employing cheaper agent models for the bottom-up generation, the framework achieves high realism and logical consistency. The results suggest that this hybrid approach is particularly effective in low-data regimes and complex multi-modal scenarios where purely generative models fail to capture rare events or logical dependencies, and where pure LLM generation lacks structural rigor. The framework is presented not as a universal dominance of one generator type, but as a flexible benchmark framework where the workflow adapts to the specific structure of the dataset.