🤖 AI

Breaking Database Lock-in: Agentic Regeneration of High Performance Storage Readers for Database Bypass

The paper introduces "Jailbreak," an LLM-assisted approach that bypasses traditional database drivers by directly reading and decoding storage files into in-memory columnar buffers, achieving up to 27x performance improvements in analytical workloads while breaking data lock-in.

Original authors: Victor Giannakouris, Immanuel Trummer

Published 2026-07-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Victor Giannakouris, Immanuel Trummer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, high-security library (a database) where all the information is stored in a very specific, complex filing system. If you want to read a book from this library, you usually have to follow a strict rule: you must ask a librarian (the database engine) to find the book, walk it over to the counter, wrap it in a protective plastic sleeve (serialization), hand it to you, and then you have to unwrap it (deserialization) before you can actually read it.

This process is slow and tedious, especially if you just want to scan thousands of books quickly to find a specific fact. The paper calls this the "wire protocol" bottleneck. Even if you have a super-fast computer ready to read the books, you are stuck waiting for the librarian to do their job.

The Problem: The "Librarian" Bottleneck
The authors, Victor Giannakouris and Immanuel Trummer, noticed that this "librarian" step is unnecessary for certain tasks. The books (data) are already sitting on the shelves in a known format. The problem is that writing a program to read those shelves directly is incredibly hard. It requires a human expert to memorize the exact layout of every shelf, how the books are stacked, and how to handle special cases (like books that were deleted but still sitting there). It's like trying to build a robot that can navigate a library without ever seeing the blueprints.

The Solution: "Jailbreak" and the AI Architect
The paper introduces a system called Jailbreak. Instead of asking the librarian for help, Jailbreak builds a robot that walks straight to the shelves, grabs the books, and reads them directly.

But how do you teach a robot to read a library it's never seen before? That's where Large Language Models (LLMs) come in.

Think of the LLM as a super-smart, tireless architect who has read every manual and blueprint for the library ever written.

The Blueprint Reader: The LLM looks at the database's source code and documentation (the blueprints).
The Builder: Instead of a human spending weeks writing code to build the robot, the LLM writes the code for the robot in minutes.
The Quality Control: The system has a "tester" agent. It runs the robot, checks if it read the books correctly, and if the robot makes a mistake, the LLM fixes the code and tries again. This happens automatically in a loop until the robot works perfectly.

The Result: Zero-Copy Speed
Once the robot is built, it doesn't just read the books; it hands them over to your computer in a format that requires zero effort to unwrap.

Old Way: Librarian wraps book -> You unwrap it -> You read. (Slow)
Jailbreak Way: Robot grabs book -> Hands it to you -> You read immediately. (Fast)

The paper tested this on two popular libraries: PostgreSQL and MySQL. They compared the new "direct shelf" method against the traditional "ask the librarian" method.

The Performance Boost
The results were dramatic:

For PostgreSQL, the new method was up to 5 times faster.
For MySQL, the new method was up to 27 times faster.

Why the huge difference for MySQL? The paper explains that MySQL's "librarian" (its standard connector) is particularly slow at wrapping and unwrapping the books. By cutting out the librarian entirely, Jailbreak removed the biggest bottleneck.

Why This Matters
The authors call this "breaking database lock-in." Usually, if you want to use a new, super-fast analysis tool (like a GPU-accelerated engine), you are stuck using the slow, old way of getting data because the database vendor controls the access. Jailbreak proves that if you have the blueprints (source code/docs), you can use AI to build your own "key" to the shelves, bypassing the vendor's slow gatekeeper entirely.

In short: Jailbreak uses AI to write custom software that reads database files directly, skipping the slow, middleman steps and making data analysis significantly faster.

Technical Summary: Breaking Database Lock-in via Agentic Regeneration of High-Performance Storage Readers

1. Problem Statement

Analytical workloads operating on data stored in external relational database systems (RDBMS) face a fundamental performance bottleneck: data access is strictly guarded by the database engine via wire protocols (e.g., JDBC, ODBC). This architecture forces all reads through query execution layers and serialization/deserialization processes not designed for bulk columnar analytics.

Existing literature, such as ConnectorX [14], indicates that over 85% of the wall-clock time in typical read_sql calls is consumed by client-side deserialization and format conversion rather than network transfer or query execution. While federated systems (e.g., XDB [6], Wayang [4]) optimize query placement, they still rely on the wire protocol for data transfer, leaving the database server as the sole gatekeeper of its storage. This creates a "data lock-in" where specialized analytical engines (e.g., DuckDB, cuDF, Spark) cannot efficiently access data without incurring significant latency and throughput penalties.

2. Methodology: The Jailbreak System

The authors propose Jailbreak, an agentic approach that bypasses the database engine entirely by reading storage files directly and materializing data as in-memory columnar buffers (Apache Arrow). The core insight is that database file formats, while complex, are fully specified by source code and documentation—artifacts that Large Language Models (LLMs) can ingest to regenerate operator-specific table reading components without human-engineered parsing logic.

2.1 The Agentic Pipeline

Jailbreak employs a four-agent LLM pipeline to automate the synthesis of high-performance C++ readers:

Dataset Generator: Creates a small synthetic test table (≈50 rows) with the target schema. Values are generated deterministically to establish a known ground truth for correctness verification.
Architect: Produces a structured JSON specification encoding page-level constants (page size, headers, magic numbers), type encodings, MVCC visibility rules, and known edge cases ("gotchas"). This agent is grounded in a knowledge base of database format documentation.
Coder: Generates a self-contained C++17 source file based on the specification. The code must compile with standard flags and export a C-linkage function (e.g., pg_to_arrow) that reads the storage file and returns an Apache Arrow RecordBatch. The agent is constrained to output only code, stripping natural language.
QA Tester: Compiles the generated code, runs it against the synthetic file, and compares the output against the ground truth. If failures occur, an LLM diagnoses the error (compiler output, runtime logs, ground truth) and produces a structured diagnosis to guide the next iteration.

This loop repeats with a retry budget (default $N_{max}=10$ ) until the reader passes all checks or the budget is exhausted. Checkpointing allows the process to resume with accumulated feedback.

2.2 Implementation Details

The system implements two specific readers:

PostgreSQL Heap Reader: Parses 8KB heap pages, handles HeapTupleHeader, performs lightweight MVCC visibility checks (skipping deleted/in-progress rows), decodes varlena and numeric types, and supports BRIN index pruning to skip irrelevant page ranges.
MySQL InnoDB Reader: Parses 16KB .ibd pages, traverses B-tree leaf pages, decodes the "Compact" row format (handling variable-length prefixes and null bitmaps), and correctly interprets big-endian numeric encodings with sign flips.

Both readers output Apache Arrow RecordBatches via the Arrow C Data Interface. This enables zero-copy data transfer to downstream engines (DuckDB, Spark, cuDF, etc.), where the consumer owns the memory buffers until release.

3. Key Contributions

The paper makes the following contributions:

Principled Alternative to Wire Protocols: Identifies direct storage-file access as a viable method to remove the database engine from the analytical data path, eliminating serialization overheads.
Agentic Synthesis Pipeline: Proposes and validates a four-agent LLM pipeline for the automated synthesis of correct, type-complete C++ readers for PostgreSQL and MySQL.
Zero-Copy Integration: Produces Apache Arrow buffers that integrate seamlessly with six downstream analytical engines, including GPU-accelerated frameworks, without format conversion.
Empirical Validation: Evaluates synthesized readers on the TPC-H benchmark, demonstrating full correctness across all 22 queries and significant performance gains.

4. Experimental Results

The authors evaluated Jailbreak on a g4dn.xlarge AWS instance (4 vCPUs, 16 GiB RAM, NVIDIA T4 GPU) using TPC-H SF=1 (6 million rows).

PostgreSQL Performance: Jailbreak consistently outperformed wire protocol baselines across all engines.
- DuckDB: 1.76× speedup (5.46s vs. 9.58s).
- PyArrow/DataFusion: Up to 3.5× speedup (reducing time from ~24s to <7s).
- cuDF: Best absolute time of 3.98s.
- Overall Speedup: Ranged from 1.3× to 5.1×.
MySQL Performance: Gains were more pronounced due to the inefficiency of MySQL's JDBC connector, which serializes rows individually.
- PyArrow: 26.9× speedup (8.5s via Jailbreak vs. 228.6s via JDBC).
- DuckDB: 6.5× speedup (6.8s vs. 44.2s).
- Overall Speedup: Ranged from 2.1× to 27×.
Correctness: The generated readers passed all 22 TPC-H queries against JDBC/ODBC baselines with full correctness validation.

5. Significance and Claims

The paper claims that Jailbreak is the first system to leverage LLM-assisted code synthesis for database bypass. Its significance lies in demonstrating that:

LLMs can synthesize complex systems code: The ability to generate correct, high-performance storage decoders from documentation and source code without human parsing logic is a viable methodology.
Data lock-in can be broken generally: Any database system whose file format is recoverable from public documentation or source code is a candidate for automated reader generation, removing the need for reverse engineering or vendor cooperation.
Performance is transformative: By eliminating the wire protocol, analytical throughput can be improved by an order of magnitude, particularly for row-oriented connectors like MySQL JDBC.

The authors position Jailbreak not as a replacement for query execution, but as a solution for the data ingestion layer, enabling specialized engines to consume data directly from storage with minimal overhead. Future work includes extending this to SQLite, MongoDB, and systematically evaluating the pipeline's convergence behavior across broader schemas.