Technical Summary: A Case for Agentic Tuning: From Documentation to Action in PostgreSQL

1. Problem Statement

Modern computer systems, particularly database management systems like PostgreSQL, expose hundreds of configurable parameters that govern runtime behavior. The optimal configuration for these parameters is not static; it shifts dynamically based on hardware generations, workload characteristics (e.g., OLTP vs. OLAP, read-heavy vs. write-heavy), and software versions.

Current tuning practices rely heavily on static documentation (official manuals, community guides, and heuristic tools). The authors identify three fundamental deficiencies in this documentation-driven approach:

1. Staleness: Documentation often encodes recommendations calibrated for obsolete hardware (e.g., rotational HDDs) that remain unchanged despite system evolution (e.g., to NVMe SSDs). For instance, PostgreSQL's random_page_cost has recommended a value of 4.0 since version 7.4, despite optimal values on SSDs being closer to 1.1.
2. Context Insensitivity: Documentation provides generic defaults (e.g., shared_buffers = 25% RAM) without conditioning on specific workload types or deployment contexts. A configuration optimal for a read-heavy OLTP workload may degrade performance by 18.6% on an OLAP workload.
3. Correlation Absence: Documentation treats parameters in isolation, ignoring inter-parameter dependencies. The authors' analysis reveals that 60% of parameter pairs exhibit interaction strengths >15%, with some pairs (e.g., shared_buffers × work_mem) explaining up to 39% of performance variance.

The core insight is that documentation records the results of expert tuning (what to set) rather than the process (how to determine what to set). This gap renders static guides ineffective when assumptions about hardware or workloads change.

2. Methodology: PERFEVOLVE

The authors propose PERFEVOLVE, a tool that shifts the paradigm from static documentation to dynamic, executable procedural knowledge. PERFEVOLVE operationalizes expert tuning methodologies into a set of "skills" that equip Large Language Model (LLM) agents to perform autonomous, version-consistent, and workload-specific tuning.

The system operates in two distinct phases:

A. Offline Profiling Phase (Run once per software version)

This phase synthesizes procedural knowledge by analyzing the system's configuration space:

1. Dimensionality Reduction (Sensitivity Analysis):
   • The system performs a single-parameter sweep across hundreds of parameters to identify a small subset ($k \ll n$) that materially affects performance.
   • It calculates the Coefficient of Variation (CV) for each parameter across representative workloads.
   • Output: A ranked list of top-$k$ sensitive parameters, their safe operating ranges (to prevent crashes), and their response-curve shapes (monotonic, non-monotonic, etc.).
2. Topology Discovery (Correlation Screening):
   • The system screens all pairs of sensitive parameters using a two-stage Factorial Analysis of Variance (ANOVA).
   • Stage A (Coarse): A $2 \times 2$ factorial experiment identifies pairs with high interaction potential.
   • Stage B (Fine): A $4 \times 4$ factorial experiment with replication measures the interaction strength ($\eta^2$).
   • Output: A correlation graph decomposed into connected components. Parameters within the same component must be jointly optimized, while isolated components can be tuned independently.

B. Online Deployment Phase (Run per deployment)

An LLM-based agent executes the generated procedural document against a live system:

• Executable Skills: The procedural knowledge is formalized as a Directed Acyclic Graph (DAG) of skills. Each skill contains preconditions, procedures (benchmarking, measuring), decision criteria, and postconditions.
• Agent Execution: The agent interprets the skills, performs targeted profiling (e.g., scanning only the top-$k$ parameters), detects correlations, and executes joint optimization for correlated components.
• Modes: PERFEVOLVE can operate as a full procedural executor or inject its structured knowledge (sensitivity rankings, safe ranges, correlation hints) into existing tuning frameworks like GPTuner or E2ETune.

3. Key Contributions

1. Systematic Identification of Documentation Gaps: The paper empirically demonstrates that existing tuning documentation suffers from staleness, context insensitivity, and a lack of correlation modeling, leading to significant performance degradation and instability.
2. Procedural Knowledge Encoding: The authors propose a methodology to transform static "what-to-set" recommendations into reproducible "how-to-determine" workflows. This shifts the knowledge representation from declarative values to executable processes.
3. Design and Implementation of PERFEVOLVE: A tool that automates the generation of process-oriented tuning documentation. It introduces dimensionality reduction and topology discovery to make profiling scalable and effective.
4. Empirical Validation: Extensive evaluation on PostgreSQL demonstrates that PERFEVOLVE significantly outperforms state-of-the-art baselines.

4. Experimental Results

The system was evaluated on PostgreSQL v16 using TPC-C (OLTP) and TPC-H (OLAP) benchmarks across two environments: a matched VM cluster and a high-end server (for cross-hardware transfer).

• Performance Improvement: PERFEVOLVE improved PostgreSQL performance by up to 35.2% compared to the default configuration and outperformed state-of-the-art documentation-driven baselines (GPTuner, E2ETune) by up to 35.2% in matched environments.
• Cross-Hardware Transfer: When transferring tuning knowledge from an 8GB VM to a 192-core server (a 96x core and 128x memory gap), PERFEVOLVE recovered performance for GPTuner from catastrophic failure (e.g., -54.2% throughput) to near-default or improved levels.
• Safety and Validity: PERFEVOLVE eliminated invalid or harmful configurations, raising the success rate of tuning experiments from 68% to 100%. It prevented the 32% crash rate observed in baseline systems by enforcing empirically derived safe ranges.
• Ablation Studies:
  • Knowledge Form: Injecting procedural structure alone outperformed injecting correct declarative values. Even "correct" static values degraded performance because LLMs anchored on them, reducing exploration.
  • Component Contribution: Dimensionality reduction (ranking) was most critical for read-heavy workloads, while correlation discovery (joint optimization) was critical for write-heavy workloads where parameter interactions are strong.
  • Joint vs. Independent: Tuning correlated parameters independently resulted in a 48.6% performance gap compared to joint optimization on specific workloads.

5. Significance and Claims

The paper argues that the future of automated system tuning lies not in better search algorithms alone, but in grounding tuning knowledge in reproducible processes.

• Paradigm Shift: The authors claim that documentation should encode the methodology of tuning (how to measure, how to reason about correlations) rather than just the results (specific parameter values). This makes tuning knowledge resilient to hardware and workload changes.
• Agentic Tuning: PERFEVOLVE serves as a bridge for LLM-based agents, providing them with structured, verifiable skills to navigate complex configuration spaces without relying on stale heuristics.
• Generalizability: While evaluated on PostgreSQL, the authors posit that this methodology is applicable to other systems with large configuration spaces (OS, runtimes, distributed systems), provided that offline profiling can be conducted to discover sensitivity and interaction structures.

The paper concludes that while PERFEVOLVE does not eliminate the cost of offline profiling, it amortizes this cost across many deployments and provides a robust framework for agentic tuning that adapts to the specific realities of a deployment environment.