💻 computer science

Prompt as a Data Type: In-Database LLM Prompt Management and Rewriting

This paper introduces PromptDB, a database system that treats prompts as first-class tuple-level data types to enable in-database management, optimization, and rewriting of Large Language Model interactions, thereby improving output validity and cost-quality trade-offs compared to static, external prompt storage.

Original authors: Denis Mayr Lima Martins, Gottfried Vossen

Published 2026-07-27

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Denis Mayr Lima Martins, Gottfried Vossen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Secret Language of Databases and Robots

Imagine you have a giant, super-organized library (a database) that holds millions of facts, and you have a brilliant, chatty robot (a Large Language Model, or LLM) that can read and understand almost anything. Usually, to get the robot to do a job using the library's facts, a human has to act as a middleman. They grab a piece of paper from the library, write a specific set of instructions (a "prompt") for the robot, send it off, and wait for the answer. The problem is that the library itself doesn't know what instructions the robot is following. It's like a chef who knows exactly what ingredients are in the fridge but can't see the recipe the cook is using; the chef can't help fix a bad recipe or suggest better ingredients because the recipe is hidden in a notebook on the counter.

This paper steps into the world of computer science where databases and artificial intelligence meet. It tackles a specific headache: right now, the instructions we give to AI robots are often stored outside the database, in messy code or notebooks. This makes it hard for the database to check if the instructions are good, fix them if they are confusing, or make them cheaper to run. The authors propose a radical idea: what if we treated these instructions just like any other piece of data, like a name or a number, stored right inside the database tables? By making the instructions "visible" to the database, the system could automatically rewrite them to be smarter, faster, and more accurate, using the database's own knowledge to help the robot do its job better.

The Paper's Big Idea: Prompts as Data

The authors, Denis Mayr Lima Martins and Gottfried Vossen, introduce a new system called PromptDB. Think of it as a database that doesn't just store facts, but also stores the instructions on how to talk to an AI. In traditional systems, if you wanted an AI to classify a customer complaint (like "My package never arrived!"), you would write a prompt in your application code, fetch the text from the database, and send it to the AI. The database just sees a string of text; it doesn't know that this text is a set of instructions meant to be executed.

In PromptDB, the prompt is a first-class citizen. It's a special data type called PROMPT. Instead of hiding the instructions in a notebook, the database stores them right inside the table rows, alongside the data they describe. A row might look like this:

Ticket ID: 1001
Message: "Package never arrived"
Priority: High
Instruction: PROMPT("Classify this ticket: {{body}}", output: [refund, delivery, technical, other])

Here, the instruction is a structured object. It knows the template ("Classify this ticket..."), it knows which part of the row to plug in (the body), and it knows the valid answers (the output list). Because the database can "see" the instruction, it can act like a smart editor.

The Magic Editor: PromptOpt

The real magic happens with a component called PromptOpt. Imagine you are a teacher grading a student's essay. If the student writes a long, rambling answer, you might tell them, "Be concise." If they forget to list the required steps, you might say, "Add the missing steps." PromptOpt does this automatically for AI prompts.

The system uses the database's own knowledge to rewrite the instructions before sending them to the AI. It has a few tricks up its sleeve:

Constraint Injection: If the database knows the only valid answers are "refund" or "delivery," PromptOpt rewrites the prompt to explicitly tell the AI, "You must choose one of these exact words." This stops the AI from making up weird answers like "The customer is sad."
Column Projection: If a row has 50 columns of data but the AI only needs to read the "message" column, PromptOpt cuts out the other 49. This saves money (fewer tokens to read) and reduces confusion.
Few-Shot Examples: The system can look at other rows in the database, find good examples of correct answers, and insert them into the prompt to show the AI exactly what to do.

The system treats these rewrites like a query optimizer. Just as a database optimizer decides the fastest way to search for data, PromptOpt decides the best way to write the prompt. It weighs the cost (how many words the AI has to read/write) against the quality (how likely the answer is to be correct). It might decide that for a simple task, a short prompt is fine, but for a tricky one, it's worth paying the extra cost to add examples.

What They Found

The authors tested PromptDB on three different datasets: fake support tickets, a car evaluation dataset, and a standard industry dataset called TPC-H. They compared three ways of doing things:

Static: The old way, where the prompt is written once and never changed.
All Rules: A version that blindly applies every possible rewrite rule.
PromptOpt: The smart version that picks the best rewrite for each specific job.

The results suggest that treating prompts as data works.

Better Quality: When the task required the AI to pick from a specific list of options (like classifying a ticket), the database-guided rewriting made the AI much more accurate. The "All Rules" approach often gave the best results, but it was expensive.
The Trade-off: The PromptOpt system managed to find a sweet spot. It didn't always pick the most expensive, complex prompt. Instead, it chose simpler prompts that were "good enough" for the job, saving on computing costs while still getting high-quality results. For example, in some tests, PromptOpt achieved competitive quality with significantly fewer input tokens than the static method.
Task Matters: The paper notes that one size does not fit all. A rewrite that helps with "semantic filtering" (checking if a row matches a rule) might not help with "value normalization" (cleaning up messy text). The system suggests that future versions need to be even smarter about knowing which rule to use for which task.

What This Means (and What It Doesn't)

The paper argues that we should stop treating AI instructions as hidden, unchangeable code. By making them visible data, we can use the database's superpowers—its knowledge of constraints, its ability to filter data, and its optimization skills—to make AI interactions more reliable and efficient.

However, the authors are careful not to claim this is a perfect, solved problem. They admit their current system uses simple "heuristic" rules (educated guesses) to estimate quality and cost, rather than a fully learned AI model that knows exactly how good a prompt will be. They suggest that while the prototype works well in simulations and on specific datasets, the "quality estimator" needs to get smarter to handle every possible task perfectly.

In short, PromptDB suggests a future where your database isn't just a warehouse for facts, but an active coach for your AI, rewriting its instructions in real-time to ensure it gets the job done right, fast, and without wasting resources. It's a step toward making AI feel less like a black box and more like a natural part of the data world.

Technical Summary: Prompt as a Data Type: In-Database LLM Prompt Management and Rewriting

Problem Statement
Large Language Models (LLMs) are increasingly integrated into database-backed applications for tasks such as tuple classification, semantic filtering, attribute extraction, and result enrichment. However, current integration patterns suffer from a fundamental architectural disconnect: the prompts driving these computations are typically stored outside the Database Management System (DBMS) in unstructured formats (e.g., application code, notebooks, or configuration files). Consequently, prompts are invisible to the DBMS, preventing the system from inspecting, rewriting, explaining, or optimizing them. While SQL-based User-Defined Functions (UDFs) allow LLM invocation, the prompt remains an opaque argument, and the database cannot leverage its own metadata (schema, constraints, statistics) to improve prompt validity or efficiency. This leads to issues such as LLMs generating outputs that violate database domains or constraints, and a lack of systematic optimization for the cost-quality trade-off of prompt execution.

Methodology and System Design
The paper introduces PromptDB, a database system that elevates prompts to first-class citizens by treating them as tuple-level database values. Drawing inspiration from Stonebraker's concept of "QUEL as a Data Type" and reflective programming, the system introduces a logical PROMPT data type.

Data Model: The PROMPT type is a self-describing structured object containing:
- Template ( $T$ ): A string with variables.
- Bindings ( $B$ ): A map linking template variables to tuple attributes.
- Model Metadata ( $M$ ): Specification of the target LLM.
- Task Metadata ( $D$ ): The output domain or specification (e.g., valid labels).
Execution Model: Prompts can reside in base tables or views. When a user queries a prompt-valued attribute, the system internally renders the prompt against the tuple, rewrites it using database context, and executes it via an internal EVAL operator.
Prompt Rewriting and Optimization: PromptDB treats prompt engineering as a database optimization problem. It utilizes a database-derived context ( $\Omega$ $Ω$ ) comprising schema, constraints, query predicates, and statistics to rewrite prompts.
- Rewrite Rules: The system employs a rule-based approach including:
  - Constraint Injection ( $R_{CI}$ ): Injecting valid output domains derived from database constraints.
  - Query-Aware Column Projection ( $R_{QACP}$ ): Pruning irrelevant tuple attributes to reduce token costs.
  - Output Format Minimization ( $R_{OFM}$ ): Enforcing concise, parseable outputs.
  - DB-Selected Few-Shot Examples ( $R_{FSE}$ ): Selecting relevant examples from the database to include in the prompt.
- PromptOpt: An optimizer that selects the best rewrite plan by estimating a cost-quality trade-off ( $\hat{Q} - \lambda\hat{C}$ ), balancing estimated output quality against token costs.

Implementation
The authors implemented a prototype of PromptDB on top of DuckDB.

The PROMPT type is defined as a DuckDB STRUCT.
To handle execution without invasive kernel modifications, the system generates evaluation views that expose prompt columns. Internally, these views expand into EVAL calls.
The EVAL operator is a scalar UDF that deserializes the prompt, applies the selected rewrite strategy (via PromptOpt), renders the final prompt, calls an LLM (via LM Studio), and returns the result as a SQL string.
Execution metadata (strategy, cost, latency, tokens) is logged for analysis.

Experimental Results
The system was evaluated on synthetic and real-world datasets (Support Tickets, Car Evaluation, TPC-H) across three tasks: Attribute Extraction, Semantic Filtering, and Semantic Value Normalization.

Task Quality: Database-guided rewriting significantly improved end-to-end task quality compared to static, manually written prompts, particularly for tasks requiring constrained or structured outputs (e.g., attribute extraction).
Rule Sensitivity: Ablation studies revealed that no single rewrite rule dominates all tasks; effectiveness is highly task-dependent. For instance, constraint injection is critical for classification, while projection aids in cost reduction.
Cost-Quality Trade-off: The PromptOpt strategy successfully selected lower-cost rewrite plans that maintained competitive quality for certain tasks, demonstrating the feasibility of cost-aware prompt planning. However, the heuristic quality estimator showed limitations in always selecting sufficiently rich plans for complex structured-output tasks.

Significance and Claims
The paper claims that treating prompts as database-visible, typed values creates a new optimization space analogous to traditional query optimization. By bringing query-optimizer thinking to prompts, PromptDB enables the DBMS to:

Improve Validity: Use schema constraints to prevent invalid LLM outputs.
Reduce Costs: Use projection and context pruning to minimize token usage.
Enhance Quality: Use database-selected examples and task-aware rewriting to improve performance, especially for smaller models.

The authors position PromptDB as a complementary approach to existing prompt engineering frameworks (like DSPy) and semantic operators (like LOTUS or EVAPORATE). While others treat prompts as ephemeral strings or operator parameters, PromptDB focuses specifically on how database metadata (schema, constraints, statistics) can systematically rewrite prompt templates. The paper concludes that this approach moves prompt engineering from ad-hoc application logic to a systematic, database-managed process, opening new avenues for AI-native database research.