A Query Engine for the Agents

Technical Summary: A Query Engine for the Agents

Problem Statement
The fastest-growing data in production today is unstructured text, including agent traces, chat logs, reasoning chains, and model outputs. Analyzing this data requires questions that SQL alone cannot answer, such as "show me where the agent got confused," necessitating the integration of Large Language Models (LLMs) directly into the query path. However, existing data infrastructure (Spark, Trino, managed warehouses) is designed for server-side execution and is ill-suited for the new class of AI applications that run client-side (e.g., Claude Code, Cursor, browser-based agents). These applications host both human users and LLM agents in the same process (JavaScript runtimes like Node, Deno, or Web Workers) and require immediate, low-latency access to data without the overhead of a backend query gateway. Furthermore, traditional engines struggle with the cost and latency of interleaving analytic operators with expensive, billable LLM inference calls.

Methodology and Architecture
The authors present Hyperparam, a browser-native lakehouse client designed to run entirely within the JavaScript runtime where the agent operates. The system is composed of three open-source JavaScript libraries totaling under 70 KB (gzipped):

1. Hyparquet (14 KB): A Parquet reader that performs range-efficient reads. It fetches only the necessary column chunks via parallel HTTP range requests, skipping irrelevant row groups using min/max statistics. It supports nested types (crucial for LLM trace schemas) and avoids full-file downloads.
2. Icebird (32 KB): A client-side Apache Iceberg client that resolves snapshots and applies position/equality deletes locally. It utilizes a custom-fetch hook to inject credentials (e.g., AWS SigV4, GCS HMAC) per request, keeping the engine credential-agnostic and allowing direct access to object storage without a backend proxy.
3. Squirreling (22 KB): A from-scratch, async-native SQL engine. Its core innovation is per-cell, lazy execution. Unlike vectorized engines that process rows in batches, Squirreling treats every cell as a deferred computation (a Promise).
   • Lazy Materialization: Columns are not fetched, decompressed, or evaluated until a downstream operator explicitly demands them.
   • Async-Native Operators: All operators are implemented as JavaScript AsyncGenerators. This allows the engine to pipeline operations, overlapping I/O and LLM inference calls.
   • LLM Integration: The engine supports llm() as a first-class User-Defined Function (UDF). Because of the lazy execution model, an llm() call is only triggered if the row passes all preceding filters and reaches the output limit.

Key Contributions

• JS-Native Distribution: The engine drops directly into the application's existing JavaScript runtime, eliminating the need for a second runtime, Foreign Function Interface (FFI), or server-side coordinator.
• Extreme Bundle Size: The entire stack is under 70 KB, enabling instant loading in cold browser tabs or per-turn agent sandboxes, a significant reduction compared to the ~8 MB (gzipped) core of DuckDB-WASM.
• Cost-Aware Execution: By deferring expensive operations (network fetches, LLM inferences) until absolutely necessary, the system aligns cost with demand. For example, a query with a LIMIT 5 will only trigger five LLM calls, regardless of how many rows match the filter, provided the query does not require global sorting or aggregation.
• Direct Storage Access: The system reads Parquet and Iceberg directly from object storage (S3, GCS, Azure) using client-side credentials, removing the latency and complexity of a query gateway.

Results
The paper provides empirical comparisons against DuckDB-WASM, the closest prior art:

• Latency: In a "cold-start" scenario (loading a 40 GB Iceberg table of agent traces), Hyperparam returns the first row in 0.6 seconds, compared to 19 seconds for DuckDB-WASM and minutes for server-side clusters (Trino/Spark).
• LLM Efficiency: In filter-bounded queries (e.g., WHERE llm(...) = 1 LIMIT N), Squirreling is 300× faster than DuckDB-WASM. This is because DuckDB-WASM evaluates rows in vectorized batches (morsels) and cannot push the LIMIT past a filter on a generated column, causing it to evaluate thousands of unnecessary LLM calls. Squirreling short-circuits at the limit.
• Agent Cost: In a benchmark where an Anthropic Haiku agent answered ten analyst-style questions on a 50,000-row dataset, both engines achieved 100% correctness. However, Squirreling achieved this at two-thirds lower cost ($0.067 vs $0.203 per pass). The cost reduction was driven by a 3.67× reduction in input tokens, as Squirreling's "did you mean" hints and error handling allowed the agent to resolve queries in fewer rounds, preventing the quadratic growth of input token costs associated with conversation history.
• Pipelining: Even when call counts were similar (e.g., in sort-bounded queries), Squirreling was 192× faster in wall-clock time due to its ability to pipeline up to 256 concurrent async invocations, whereas DuckDB-WASM's synchronous WASM boundary pinned concurrency to one.

Significance and Claims
The paper argues that data engineering as a discipline must evolve to support "AI-native" client applications. The conventional read path, which places heavy infrastructure between the consumer and data, breaks the "curate-review-act" loop required by agents. Hyperparam demonstrates that a lightweight, client-side engine can effectively handle unstructured text analysis by interleaving analytic operators with model-based interpretation. The authors position this not as a replacement for server-side clusters for massive joins or spill-to-disk workloads, but as the necessary solution for the specific shape of data and queries emerging from agentic workflows, where the engine must come to the consumer to enable tight cost bounds and rapid iteration.