Samyama: A Unified Graph-Vector Database with In-Database Optimization, Agentic Enrichment, and Hardware Acceleration

This paper introduces Samyama, a high-performance, unified graph-vector database written in Rust that integrates persistent storage, vector indexing, native optimization solvers, and agentic LLM enrichment into a single engine, achieving competitive throughput and latency on commodity hardware while offering GPU-accelerated enterprise features.

The Gist

Imagine you are running a massive, bustling city. In this city, you have three distinct departments that usually don't talk to each other:

1. The Map Makers (Graphs): They track how everything connects (roads, friendships, supply chains).
2. The Librarians (Vectors): They understand the meaning of things, finding books or ideas that are similar even if they use different words.
3. The Planners (Optimization): They solve complex puzzles, like how to deliver 1,000 packages in the shortest time or how to schedule a factory to save money.

The Problem:
In most modern cities (data systems), these three departments live in separate buildings. If the Map Makers need to talk to the Planners, they have to print out their maps, drive them to the Planning Department, wait for the planners to do their math, and then drive the results back. This is slow, expensive, and prone to errors. This is what data engineers call "ETL" (Extract, Transform, Load), and it's a huge bottleneck.

The Solution: Samyama
The paper introduces Samyama (a Sanskrit word meaning "Integration"). Think of Samyama not as three separate buildings, but as a super-efficient, all-in-one skyscraper where the Map Makers, Librarians, and Planners all work in the same room.

Here is how it works, broken down into simple concepts:

1. The "Lazy" Delivery Service (Late Materialization)

Imagine you order a pizza. Usually, the kitchen makes the whole pizza, boxes it, and then you eat it.
Samyama uses a technique called Late Materialization. Instead of making the whole pizza immediately, the kitchen just sends you a ticket (a reference) saying, "We have a pizza ready." You only ask for the specific toppings you want right before you take a bite.

• Why it matters: In data terms, this means the system doesn't waste time copying huge amounts of information it doesn't need yet. The paper shows this makes the system 4 to 5 times faster when navigating complex connections (like finding a friend of a friend of a friend).

2. The "Self-Driving" Librarian (Agentic Enrichment)

Usually, a database is like a passive library: you have to go there and ask for a book. If the book isn't there, it stays empty.
Samyama introduces Agentic Enrichment. Imagine a librarian who, when you ask a question and the answer isn't in the library, automatically goes out, finds the information on the internet, reads it, and adds it to the library shelves for you.

• The Catch: It uses AI (Large Language Models) to do this. It's like having a research assistant that never sleeps, constantly updating the city's knowledge base with new facts it discovers on its own.

3. The "In-House" Math Wizards (In-Database Optimization)

Normally, if you want to solve a complex logistics problem, you have to take your data out of the database, send it to a super-computer math solver, and wait for the answer.
Samyama puts the math wizards inside the database. You can ask a question in the database language (Cypher), and it will instantly solve the optimization problem right there.

• The Benefit: No more moving data back and forth. It's like asking your bank teller to not only check your balance but also instantly calculate the best investment strategy for you, all in one conversation.

4. The "Super-Speed" Engine (Hardware Acceleration)

Samyama is built with Rust, a programming language known for being incredibly fast and safe (it prevents the computer from crashing due to memory errors).
It also has a special mode for GPUs (the powerful chips in graphics cards). Imagine the CPU is a single chef cooking a meal, while the GPU is a team of 1,000 chefs chopping vegetables simultaneously. For massive calculations (like analyzing millions of connections), Samyama can switch to "Team Chef Mode," making it 8 times faster for certain tasks.

5. The "Universal Translator" (RDF & SPARQL)

Samyama speaks many languages. It understands standard graph queries, but it also speaks RDF (a standard for the "Semantic Web"). It's like a diplomat who can talk to any other database system without needing a translator, ensuring your data can be shared easily with the rest of the world.

The Results: How Fast is it?

The authors tested this on a standard, off-the-shelf computer (a Mac Mini).

• Ingestion: It can add 255,000 new connections per second. That's like adding a new road to a map every 4 milliseconds.
• Speed: It handles over 100,000 questions per second.
• Accuracy: It passed 100% of the industry-standard tests (LDBC Graphalytics), proving it doesn't just look fast; it gets the math right.

The Trade-off

The paper is honest about one weakness: The "brain" that plans how to answer a question (the query planner) is still learning. For the very first time you ask a complex question, it takes a little longer to figure out the best route compared to older, more established systems. However, once it learns the route (caching), it becomes incredibly fast.

Summary

Samyama is a new kind of database that stops forcing you to move data between different tools. It combines maps, meaning, and math into one powerful, safe, and fast engine. It's like upgrading from a city with separate, disconnected departments to a smart, self-updating, all-in-one command center that can solve problems as fast as you can ask them.

Technical Summary

Here is a detailed technical summary of the paper "Samyama: A Unified Graph-Vector Database with In-Database Optimization, Agentic Enrichment, and Hardware Acceleration."

1. Problem Statement

Modern data architectures are increasingly fragmented, forcing developers to stitch together disparate systems to handle complex workloads:

• Graph Databases (e.g., Neo4j) for relational structure.
• Vector Stores (e.g., Pinecone) for semantic similarity and LLM RAG.
• Analytics Engines (e.g., Spark) for large-scale processing.
• Optimization Solvers (e.g., Gurobi/Python) for resource allocation and scheduling.

This fragmentation results in:

• Complex ETL Pipelines: Data must be extracted, transformed, and loaded between systems, incurring significant I/O overhead ($O(N+M)$).
• Synchronization Overhead: Maintaining consistency across multiple databases is operationally difficult.
• Latency: Round-trips between systems degrade performance for real-time applications.

2. Methodology & System Architecture

Samyama is a high-performance, memory-safe database engine written in Rust that unifies property graphs, vector search, analytics, and metaheuristic optimization into a single binary.

Core Architecture Components

• Storage Engine: Built on RocksDB (LSM-tree) with LZ4/Zstd compression. It uses a versioned-arena MVCC model where NodeId and EdgeId are direct u64 indices into contiguous memory, eliminating hash lookups and ensuring $O(1)$ access.
• Query Engine:
  • Supports ~90% of OpenCypher.
  • Uses a PEG parser (pest crate) and a hybrid Volcano-Vectorized execution model (batch size 1,024).
  • Implements 35 physical operators including a cost-based planner with plan enumeration, predicate pushdown, and join reordering.
  • Late Materialization: Adapts columnar database strategies to graphs. Instead of cloning full nodes, the engine produces lightweight NodeRef pointers during traversal, resolving properties only at the final projection step.
• Vector Engine: Implements HNSW indexing for approximate nearest neighbor search (Cosine, L2, Dot Product) with native integration into graph traversal for Graph RAG.
• Optimization Module: Integrates 22 metaheuristic solvers (e.g., NSGA-II, Jaya, PSO, GA) directly into the query language via Cypher procedures, allowing optimization problems to be solved without leaving the database.
• Agentic Enrichment (GAK): An autonomous loop where LLM agents use tool-calling to discover missing data, parse unstructured sources, and generate Cypher CREATE commands to expand the knowledge graph dynamically.
• Hardware Acceleration: Uses wgpu (WGSL) for cross-platform GPU acceleration (Metal, Vulkan, DX12) of graph algorithms (PageRank, PCA) and sorting.

3. Key Contributions

The authors categorize contributions into research innovations and engineering implementations:

Primary Research Contributions

1. Late Materialization for Graphs: Adapts columnar strategies to graph traversal. By deferring property resolution, it reduces memory bandwidth usage by 4–5×.
2. In-Database Metaheuristic Optimization: Eliminates ETL overhead by exposing 22 solvers directly via Cypher, enabling "graph-native" optimization.
3. Agentic Enrichment (GAK): Transforms the database from a passive store into a self-evolving knowledge graph using LLMs for autonomous data acquisition and schema expansion.

Engineering Contributions

4. Unified Engine: A single binary handling OLTP, OLAP, vector search, and optimization.
5. Cost-Based Query Planner: Features plan caching, predicate pushdown, and early LIMIT propagation.
6. Cross-Platform GPU Acceleration: Offloads heavy graph algorithms and PCA to GPUs via compute shaders.
7. RDF Interoperability: Native support for RDF/SPARQL with SPO/POS/OSP indices.
8. Industry Validation: Achieved 100% pass rate (28/28) on the LDBC Graphalytics benchmark.

4. Experimental Results

Evaluated on commodity hardware (Mac Mini M4, 16 GB RAM):

• Ingestion Throughput:
  • 255K nodes/sec (CPU) and 412K nodes/sec (GPU-accelerated).
  • Edge ingestion reaches ~5.2M ops/sec with GPU.
• Query Performance (OLTP):
  • 115K Cypher queries/sec at 1M nodes.
  • Late Materialization Impact: Reduced latency by 4.0× for 1-hop and 4.7× for 2-hop traversals compared to full node cloning.
• GPU Acceleration:
  • 8.2× speedup for PageRank at 1M nodes (92.4ms CPU vs. 11.2ms GPU).
  • Crossover point for GPU benefit is ~100K nodes for general algorithms.
• Vector Search:
  • ~15K QPS for 128-dimensional vectors (k=10).
• Memory Efficiency:
  • 450 MB RSS for 1M nodes (vs. ~1,200 MB for Neo4j), attributed to arena allocation and lack of GC overhead.
• Benchmarks:
  • 100% LDBC Graphalytics pass rate.
  • LDBC SNB: All 21 interactive queries passed; 16/20 BI queries passed (one timed out due to combinatorial explosion).
  • LDBC FinBench: 100% pass rate on financial transaction patterns.

5. Significance and Limitations

Significance

• Architectural Unification: Demonstrates that a single engine can effectively handle graphs, vectors, and optimization, removing the need for complex multi-system ETL pipelines.
• Performance on Commodity Hardware: Achieves competitive performance on a single consumer-grade Mac Mini, challenging the notion that such workloads require massive clusters.
• AI-Native Design: The integration of Agentic Enrichment and Graph RAG positions the database as a proactive component of AI systems rather than a passive storage layer.
• Memory Safety: Leverages Rust to provide high performance without the garbage collection pauses common in JVM-based graph databases (e.g., Neo4j).

Limitations

• Query Latency: Single-query latency (41ms for 1-hop) is currently higher than specialized engines (e.g., Memgraph at ~1.1ms) because the parser and planner account for 98% of the latency. While plan caching helps repeated queries, first-execution overhead remains high.
• Single-Node Evaluation: Benchmarks were run on a single node; distributed performance with Raft replication has not been scaled to large clusters.
• SPARQL Execution: The parser is implemented, but full query execution over the RDF triple store is incomplete.
• Optimizer Maturity: The cost model uses multiplicative cardinality estimates and lacks histogram-based correlation analysis found in mature RDBMS optimizers.

Conclusion

Samyama represents a significant step toward unified data platforms, proving that a Rust-based engine can successfully integrate graph, vector, and optimization workloads. While the query optimizer requires further maturation to match the raw speed of specialized engines, the elimination of ETL overhead and the introduction of autonomous, agent-driven data enrichment offer a compelling new paradigm for AI-native data infrastructure.