Technical Summary: Incorporating Deep Learning Design in Database Queries

Problem Statement

The integration of deep learning (DL) and relational databases has traditionally been hindered by a "round-trip" engineering overhead. Conventional approaches require translating relational data into graph representations (where records become nodes and foreign keys become edges) and then exporting this data to external machine learning frameworks (e.g., PyTorch Geometric, DGL) to train Graph Neural Networks (GNNs). This process forces developers to work across disparate paradigms: using declarative query languages for data management and imperative code for model design. Furthermore, existing in-database ML frameworks often bridge these domains by integrating tensor types and linear algebra operators directly into the query engine, requiring users to explicitly maintain the connection between logical data and vector representations. The authors argue that this separation is artificial and that specifying a neural network over relational data should be as natural as querying the data itself.

Methodology: Neuro-Relational Algebra (NRA) and RelaNN

The paper proposes RelaNN, a proof-of-concept system that integrates deep learning directly into database queries by extending the relational model. The core methodology rests on Neuro-Relational Algebra (NRA), a formal framework that associates every database tuple with a learnable vector embedding.

1. Neuro-Relational Data Model (NRM)

The data model extends the classic relational model by defining an embedded relation $(r, \eta)$, where $r$ is the standard content relation and $\eta$ is an annotation mapping each tuple to an embedding in $\mathbb{R}^d$. This allows the system to operate simultaneously on the logical domain (content) and the numerical domain (embeddings).

2. Neuro-Relational Operators

NRA extends standard relational algebra with operators that manipulate both content and embeddings:

• Join ($\bowtie$): Performs a standard natural join on content. For embeddings, it concatenates the embeddings of the contributing tuples from the input relations.
• Projected Union ($\cup_{\alpha, A}$): A unified operator combining projection and union. It projects tuples onto a common attribute sequence and aggregates the embeddings of tuples with identical content using a multiset aggregator $\alpha$ (e.g., sum, mean, max). This handles the aggregation of embeddings required when multiple input tuples map to the same output tuple.
• Transformation ($T_\tau$): Applies a differentiable transformation $\tau$ (e.g., a feed-forward network, ReLU, or linear layer) to the embeddings of a relation. This is the primary mechanism for introducing learnable parameters.
• Selection, Difference, Renaming: These operators pass through embeddings unchanged, similar to classical algebra.

3. The RelaNN Language

RelaNN is a declarative language built on top of NRA. Programs consist of rule statements that define how to derive new embedded relations:

• Join Rules: Combine relations via joins, apply a transformation to the concatenated embeddings, and aggregate via a projected union.
• Union Rules: Unify relations with identical schemas, filter, and transform embeddings.
• Fit Statements: A language construct (?fit) that minimizes a loss function defined over an output relation, updating the global parameter assignment (including both tuple embeddings and transformation weights).
• Predict Statements: Evaluate the program with current parameters to materialize results.
• Extensions: The language supports aliases for parameter sharing, function definitions for code reuse, encoding/decoding to move data between content columns and embeddings, and templates to generate repetitive rules (e.g., for multi-head attention).

4. Implementation Pipeline

RelaNN compiles a program into a Term Graph (a DAG of NRA operators). When executed, this graph is translated into a Physical Plan that runs across SQL, cuDF, and PyTorch:

• Data Loading: SQL queries fetch content from the database.
• Join/Selection: Implemented via cuDF DataFrame operations (merge, index_select) that emit row-index tensors.
• Embedding Manipulation: PyTorch operations (e.g., torch.index_select, scatter_add) fetch and aggregate embeddings based on the content indices, maintaining full differentiability for end-to-end training.
• Transformation: Custom nn.Module layers are applied directly via PyTorch.

Key Contributions

1. Formal Framework (NRA): The introduction of Neuro-Relational Algebra, which unifies relational query processing and neural model design by extending the relational model with learnable tuple embeddings.
2. Declarative System (RelaNN): An open-source system that allows users to implement state-of-the-art graph-learning architectures using familiar SQL/Datalog-like constructs.
3. Expressiveness: Demonstrated ability to express complex architectures including Graph Convolutional Networks (GCN), Relational GCN (R-GCN), Heterogeneous Graph Transformers (HGT), Hypergraph Neural Networks (HyGNN), and Deep Homomorphism Networks (DHN) with significantly fewer lines of code than imperative baselines.
4. Performance: A proof-of-concept implementation that achieves competitive predictive accuracy and runtime performance compared to specialized libraries like PyTorch Geometric.

Experimental Results

The authors evaluated RelaNN against reference implementations (PyTorch, PyG) on five architectures across multiple datasets (Cora, DBLP, AIFB, CSL, EXP, DrugBank, etc.).

• Code Conciseness: RelaNN programs were approximately 4x shorter than baseline implementations. For example, a Deep Homomorphism Network (DHN) implementation required 76 lines in RelaNN versus 259 lines in the baseline.
• Accuracy: RelaNN reproduced the reference accuracy for GCN and HGT. For other models (R-GCN, HyGNN, DHN), results were within one standard deviation of baselines or matched them exactly (e.g., 100% accuracy on the CSL dataset for DHN).
• Runtime:
  • For GCN and HGT, RelaNN runtime was within a factor of 2 of the baseline.
  • For DHN, RelaNN was 2x to 14x faster than the baseline, attributed to the efficiency of the declarative compilation and avoidance of complex custom tensor code.
  • For R-GCN on the AIFB dataset, RelaNN was roughly 30x slower than the highly optimized torch-rgcn baseline. The authors attribute this to the current implementation being an unoptimized proof-of-concept and the specialized nature of the baseline.

Significance and Claims

The paper claims that RelaNN provides a declarative foundation for relational deep learning, demonstrating that it is possible to design and train state-of-the-art neural networks directly over relational databases without intermediate graph representations.

• Simplicity: The system makes implementing complex architectures as simple as writing a query, removing the engineering overhead of data translation and graph construction.
• Integration: By treating embeddings as first-class citizens within the relational model, the approach naturally merges logical reasoning with numerical reasoning.
• Future Potential: While the current implementation is a prototype, the authors argue it represents a "concrete path" toward making deep learning over databases as accessible as standard querying. They identify future work in scaling to datasets larger than GPU memory (via sampling and streaming) and optimizing query execution (pushing down transformations, fusing kernels) to close the performance gap with specialized libraries.

The authors explicitly state that their work does not claim to replace enterprise systems but rather to demonstrate the feasibility and simplicity of a unified approach, paving the way for future optimizations and the potential implementation of relational foundation models.