🤖 AI

PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models

The paper introduces PLuRel, a lightweight framework for synthesizing diverse multi-table relational databases that demonstrates, for the first time, that scaling synthetic data leads to predictable performance improvements and strong generalization for Relational Foundation Models.

Original authors: Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik, Vijay Prakash Dwivedi, Johannes Hoffart, Carlos Guestrin, Jure Leskovec

Published 2026-07-15

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik, Vijay Prakash Dwivedi, Johannes Hoffart, Carlos Guestrin, Jure Leskovec

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to understand the messy, interconnected world of business data. In the real world, companies store their secrets in massive "relational databases"—think of these as giant, multi-room libraries where every book (a table) is connected to others by specific threads (like a customer's ID linking their name to their order history). To make a robot truly smart at reading these libraries, we need to show it millions of different examples. But here's the catch: real company data is private. It's locked away behind walls of privacy laws and trade secrets, so we can't just hand it to our robot.

This is where the idea of "synthetic data" comes in. It's like building a perfect, fake replica of a library from scratch, using math to guess how the books and threads should look without ever seeing a real one. Scientists have been doing this for single tables (like a simple list of names), but making a fake multi-room library where the connections between rooms make sense has been incredibly hard. If the robot learns from a fake library where the connections are broken or weird, it won't be able to solve real problems later. This paper tackles that exact puzzle: how do we build a factory that can churn out endless, diverse, and perfectly connected fake databases so our AI can learn the rules of the game before it ever sees a real one?

The PLUREL Factory: Building Fake Worlds for AI

Meet PLUREL, a new framework that acts like a master architect and a creative writer rolled into one. Its job is to build synthetic relational databases from the ground up, creating "fake worlds" for AI models to practice on. The researchers behind PLUREL wanted to see if they could unlock a secret power for these AI models: the ability to get smarter simply by seeing more and more diverse examples, a concept known as "scaling laws."

Think of training an AI like teaching a teenager to drive. If you only let them practice in one empty parking lot (a single, small database), they might get good at that specific lot but will crash the moment they hit a busy city street (a real-world database). PLUREL solves this by generating thousands of different "driving courses"—some with narrow streets, some with roundabouts, some with heavy traffic, and some with foggy weather. Each course is a unique database with its own set of tables (like "Users," "Items," and "Transactions") and the complex web of connections between them.

How PLUREL Builds These Fake Worlds

PLUREL doesn't just copy-paste existing data; it builds everything from scratch in three creative steps:

The Blueprint (Schema): First, it draws a map of the library. It decides how many rooms (tables) there are and how they connect. It uses a "directed graph" (a fancy term for a map with arrows) to decide which room leads to which. For example, it might decide that the "Users" room connects to the "Orders" room, but not directly to the "Ships" room.
The Threads (Connectivity): Next, it weaves the threads that tie the rooms together. In a real database, a specific order belongs to a specific user. PLUREL uses a clever "bipartite graph" (a way of matching two groups) to decide which user gets which order. It doesn't just pick randomly; it uses a "hierarchical" pattern, like a family tree, to ensure that connections feel natural. Maybe a "VIP" user gets more orders, or orders from a specific region cluster together.
The Content (Features): Finally, it fills the rooms with data. It uses "Structural Causal Models" (think of these as logic engines) to decide what goes in the cells. If a user buys a winter coat, the system knows that the "date" should be in winter and the "price" might be higher. It even adds "temporal patterns," meaning the data changes over time just like real life (e.g., sales spike during Black Friday).

The Big Discovery: More Variety = Smarter AI

The most exciting part of the paper is what happened when they started training a model called the Relational Transformer (RT) on these PLUREL-generated databases.

The researchers tested two things:

Size: How much data did the model see? (Total number of tokens, or "words" of data).
Diversity: How many different fake databases did the model see? (The number of unique "worlds").

They found a beautiful, predictable pattern: The more diverse the training data, the better the model got.

Imagine you are learning a language. If you read 10,000 pages of the same book, you'll memorize that book perfectly but won't know how to speak to a stranger. But if you read 10,000 pages spread across 1,000 different books (different genres, different authors, different styles), you'll learn the rules of the language. PLUREL showed that when the AI saw more unique databases (increasing diversity), its ability to predict things on real data improved in a smooth, predictable curve. This is a "power law," meaning the improvement follows a steady, mathematical rule rather than being random.

Does It Work on Real Life?

The ultimate test was: "Can this fake training help with real problems?"

The researchers took their AI, which had been trained on billions of tokens of PLUREL's fake data, and gave it a quick "finishing school" on a small amount of real-world data (from a benchmark called RelBench). The results were impressive:

The Hybrid Approach Won: An AI that learned on PLUREL's fake data and then fine-tuned on real data performed significantly better than an AI that only learned on real data.
The Gains: On average, this "Synthetic + Real" approach improved the model's accuracy by 1.2% for classification tasks (like guessing if a user will leave) and 3.0% for regression tasks (like predicting a sales number).
The Highs: On some specific tasks, the improvement was huge—up to 7.4% better at predicting user engagement and 5.2% better at predicting customer lifetime value.

However, the paper is careful to note that synthetic data alone isn't a magic bullet. If they tried to use only the fake data without ever seeing real data, the model struggled. The fake data is great for learning the general rules, but the real data is needed to align with the specific quirks of the real world.

What This Means

PLUREL suggests that we don't need to wait for companies to share their secret private data to build better AI. Instead, we can build our own "training gyms" using synthetic data. By generating thousands of diverse, mathematically sound fake databases, we can teach AI models the fundamental rules of how data connects. This allows the models to generalize—meaning they can take what they learned in the fake world and apply it successfully to the messy, real world.

The paper concludes that this is a promising new path. It's not just about making more data; it's about making better, more varied data. By unlocking the ability to scale up the diversity of training data, PLUREL helps us build Relational Foundation Models that are ready to tackle the complex, interconnected data challenges of the future, all while keeping real-world privacy safe.

Technical Summary: PLUREL – Synthetic Data Unlocks Scaling Laws for Relational Foundation Models

1. Problem Statement

Relational Foundation Models (RFMs) aim to facilitate data-driven decision-making by learning from complex, multi-table databases. While Foundation Models (FMs) for text, image, and video have achieved remarkable success through large-scale pretraining, RFMs face a critical bottleneck: the scarcity of diverse, public, multi-table relational databases. Real-world enterprise data is typically inaccessible due to privacy constraints and business confidentiality.

Existing synthetic data generation methods for tabular data primarily focus on single-table generation using Structural Causal Models (SCMs). However, these approaches fail to capture the essential complexities of relational databases, specifically the primary–foreign key (P→F) connectivity between rows across different tables. This connectivity determines information locality and shapes the joint data distributions that RFMs must learn. Furthermore, existing relational data generation techniques (e.g., GANs, diffusion models) often rely on existing real-world examples to reproduce statistical properties, rather than generating novel schemas and structures from scratch. Consequently, current RFMs are trained on limited, non-diverse datasets (e.g., only a few databases from RelBench), hindering the realization of scaling laws and robust generalization.

2. Methodology: The PLUREL Framework

The authors introduce PLUREL, a lightweight framework designed to synthesize multi-tabular relational databases from scratch. PLUREL operates through three distinct stages of abstraction to model the structural and statistical properties of relational data:

Stage 1: Schema Generation via Directed Graphs

The database schema is modeled as a directed acyclic graph (DAG), $G$ , where nodes represent tables and directed edges represent inter-table connectivity.

Graph Sampling: $G$ is sampled from a family of random DAGs, including Barabasi-Albert (modeling hub tables), Reverse Random-Tree (modeling strict hierarchies), and Watts-Strogatz (modeling local clustering).
Table Classification: Tables are partitioned into entity tables (nodes with out-degree $\ge 1$ ) and activity tables (remaining nodes).
Metadata: The number of rows and feature columns for each table is sampled independently from specific distributions. A topological ordering of $G$ dictates the generation sequence, ensuring parent tables are synthesized before their children.

Stage 2: Foreign Key Generation via Bipartite Graphs

This stage populates the foreign key columns by modeling row-level connectivity between parent and child tables.

Hierarchical Clustering: Rows in both the parent table ( $\tilde{T}$ ) and child table ( $T$ ) are clustered into a hierarchy of blocks using a Hierarchical Stochastic Block Model (HSBM).
Connectivity Probabilities: A level-wise probability matrix $P^{[l]}$ determines the likelihood of a row in $\tilde{T}$ linking to a row in $T$ . This allows for flexible connectivity patterns, ranging from uniform distribution to highly localized clusters, mimicking real-world hierarchical data relationships.

Stage 3: Feature Generation via Structural Causal Models (SCMs)

Once the schema and connectivity are established, cell values are generated using SCMs.

Causal Graphs: Each table is associated with an SCM defined by a causal graph $C_T$ . Nodes represent variables, and edges encode cause-and-effect relationships.
Conditional Generation: Feature values are generated by propagating values through the causal graph. For tables with foreign keys, the SCM mechanism incorporates feature nodes from parent tables, allowing child rows to depend on specific parent rows.
Temporal Patterns: To capture temporal correlations, exogenous inputs for source nodes are modeled as a combination of trend (power-law), cycle (sinusoidal), and fluctuation (Gaussian noise) components. This avoids the unrealistic assumption that rows sharing foreign keys are independent and identically distributed (i.i.d.).
Data Types: The framework supports numeric and categorical features, with mechanisms to project and reconstruct values into shared latent spaces using Multi-Layer Perceptrons (MLPs).

3. Key Contributions

A Novel Synthetic Data Framework: PLUREL is the first framework to synthesize multi-tabular relational databases from scratch, explicitly modeling schema structure, primary–foreign key connectivity, and conditional feature distributions.
Discovery of Scaling Laws for RFMs: By removing data bottlenecks, the authors conduct the first scaling analysis for RFMs across two axes: diversity (number of synthetic databases, $N$ ) and size (total pretraining tokens, $S$ ).
Validation of Power-Law Scaling: The study demonstrates that pretraining loss exhibits a power-law dependency on both $N$ and $S$ , provided neither axis acts as a bottleneck.
Enhanced Generalization: The paper shows that synthetic pretraining improves zero-shot generalization to real-world databases and serves as a robust foundation for continued pretraining on real data.

4. Experimental Results

The authors pretrain a Relational Transformer (RT) on billions of tokens generated by PLUREL and evaluate performance on the RelBench benchmark (6 real-world datasets, 18 tasks).

Scaling Laws:
- Loss Scaling: Validation loss follows a power law with respect to the number of synthetic databases ( $N$ ) and dataset size ( $S$ ).
- Interplay of Axes: Scaling $N$ with a fixed $S$ leads to underfitting, while scaling $S$ with a fixed $N$ leads to overfitting. Optimal performance requires scaling both axes in tandem.
Zero-Shot Generalization:
- Models pretrained on larger and more diverse synthetic datasets show improved zero-shot performance on unseen real-world tasks compared to models trained on smaller synthetic sets or real data alone.
Continued Pretraining:
- Synthetic + Real: Pretraining on PLUREL data followed by continued pretraining on real data yields the best results.
- Performance Gains: This approach achieves an average absolute gain of +1.2% in AUROC (classification) and +3.0% in R² (regression) over models trained on real data alone.
- Task-Specific Improvements: Significant gains are observed in regression tasks (e.g., +5.2% on item-LTV) and behavior-driven classification tasks (e.g., +7.4% on user-engage).
- Limitations: "Synthetic only" pretraining generally underperforms compared to "Real only" or "Synthetic + Real," indicating that while synthetic data provides a strong base, continued pretraining on real data is critical for distribution alignment, particularly for tasks requiring specific column semantics.

5. Significance and Claims

The paper positions PLUREL as a paradigm shift for the development of Relational Foundation Models. Its primary significance lies in unlocking data scaling for RFMs, a field previously constrained by the lack of public, diverse relational data.

Privacy-Preserving Development: PLUREL enables the creation of unlimited, diverse relational databases without relying on sensitive real-world data, addressing privacy and confidentiality barriers.
Foundation for Scaling Laws: The work establishes that RFMs, like language and vision models, follow scaling laws where performance improves predictably with increased data diversity and size.
Synergistic Pretraining: The results suggest a synergistic relationship where synthetic pretraining acts as a powerful initializer, enhancing the model's ability to learn from subsequent real-world data, thereby accelerating the development of next-generation RFMs for enterprise applications.

The authors conclude that while synthetic data alone is insufficient for robust zero-shot transfer due to the lack of specific column semantics, it is a necessary and powerful component for building scalable, generalizable relational foundation models.