Technical Summary: Query Cost Model Calibration in Confidential Virtual Machines

Problem Statement

The adoption of Confidential Computing, specifically Confidential Virtual Machines (CVMs) like AMD SEV-SNP, offers a practical path for running legacy Database Management Systems (DBMSs) without source code modifications while protecting data in use. However, analytical queries in CVMs suffer from substantial performance overheads compared to conventional unencrypted VMs (KVMs).

Prior research has largely focused on benchmarking these slowdowns rather than optimizing them. This paper identifies the root cause as a hardware–software mismatch: traditional query optimizers rely on cost models calibrated for KVMs, which no longer hold in CVM environments. Consequently, optimizers misrank execution plans, selecting strategies that are efficient in KVMs but suboptimal in CVMs.

The authors attribute this mismatch to two specific architectural properties of SEV-SNP:

1. Data Movement Overhead: In CVMs, data transfers between storage and memory involve encryption processing and "bounce-buffer" copying (staging data in shared memory). This overhead is non-uniform; different query plans (e.g., Hash Join vs. Nested Loop) induce different patterns of data movement, leading to variable penalties.
2. RMP-Related Translation Overhead: SEV-SNP utilizes a Reverse Map Table (RMP) to validate page ownership and permissions during address translation. This introduces latency in memory access paths, particularly for random-access operations like index scans and probe-heavy nested loops. Experimental evidence shows this manifests as a significant increase in Cycles Per Instruction (CPI) due to translation and protected-memory checks, rather than an increase in the total number of instructions executed.

Methodology

The authors propose a lightweight, CVM-aware cost calibration integrated directly into the query optimizer's cost model. Rather than redesigning the execution engine, the approach adjusts the estimated costs of operators to reflect the true relative prices of execution paths in SEV-SNP.

Core Mechanism

The calibration augments standard cost formulas (which typically decompose cost into CPU, Memory, and I/O) with two additional overhead terms derived from physical proxies already available to the optimizer:

1. Data Movement Cost (datamovecost): This term penalizes operators based on their working set size ($ws$) and an overflow factor ($x$). The overflow factor is calculated when the working set exceeds the effective cache budget. The cost is modeled as $ws \cdot \eta_{dm} \cdot \sigma(x)$, where $\eta_{dm}$ is a calibrated penalty and $\sigma(x)$ activates the term when the active footprint spills out of the cache, triggering encrypted memory traffic and bounce-buffer interactions.
2. RMP Overhead Cost (rmptcost): This term penalizes operators based on page-access intensity ($a$) and locality. It is modeled as $a \cdot \eta_{rmp} \cdot (1 + \alpha_{rmp} \cdot \sigma(x))$. This captures the increased latency of address translation and RMP validation, which disproportionately affects random-access operators (e.g., Index Scans, Nested Loop Joins with index probes).

Operator-Specific Calibration

The authors apply these terms to specific operators in PostgreSQL:

• Scans (Sequential, Bitmap, Index): Sequential and bitmap scans are charged primarily for data movement when the working set overflows. Index scans are charged for both data movement and RMP overhead, as index traversal involves pointer chasing and non-sequential page accesses that trigger translation checks.
• Joins:
  • Nested Loop: Charged for data movement and conditional RMP overhead if the inner side is index-driven.
  • Hash Join: The cost model explicitly accounts for the hash table size relative to the Last-Level Cache (LLC). If the hash table exceeds the LLC, the operator incurs significant data movement and translation costs.
  • Merge Join: Primarily charged for data movement, with conditional RMP costs if inputs lack locality.
• Materialize/Memoize: Charged based on the size of the buffered intermediate state.

The parameters for these cost terms are tuned using a representative subset of queries from each workload, aiming for a robust calibration across different scenarios without exhaustive optimization.

Key Contributions

1. Identification of Performance Distortions: The paper demonstrates that RMP checks and data movement overheads are the primary drivers of optimizer-facing performance distortions in CVMs. It provides empirical evidence (via CPI inflation and bounce-buffer activity analysis) that these overheads are operator-dependent and plan-sensitive.
2. CVM-Calibrated Cost Model: The authors propose a practical extension to the cost model that injects targeted penalties for RMP-sensitive random access and data movement. This preserves the optimizer's architecture while making plan decisions more faithful to CVM realities.
3. Implementation and Evaluation: The calibration is implemented within PostgreSQL 16.9. The evaluation covers standard OLAP benchmarks (JOB, CEB, Stack, and TPC-DS) on AMD SEV-SNP hardware.

Results

The evaluation shows that the calibration significantly narrows the performance gap between KVM and SEV-SNP:

• Time Savings: The calibration reduces total execution time across workloads by 3.54% to 48.35%.
  • JOB: 6.90% improvement.
  • CEB: 17.14% improvement.
  • Stack: 48.35% improvement.
  • TPC-DS: 3.54% improvement.
• Cross-Environment Performance: On the CEB and Stack workloads, the calibrated SEV-SNP execution outperforms the uncalibrated KVM baseline, indicating that the cost model correction can recover enough efficiency to surpass the non-encrypted baseline in specific scenarios.
• Query Coverage: A substantial fraction of queries benefited from the calibration (ranging from 38.8% to 62.5% across workloads).
• Latency Distribution: The calibration shifts the query latency distribution downward, reducing median latency and compressing the spread (standard deviation). For example, in the Stack workload, the maximum latency dropped from 50.80s to 13.84s, and the standard deviation decreased from 7.00s to 2.24s.

Significance and Claims

The paper claims that the primary barrier to high-performance database workloads in CVMs is not just the raw hardware overhead, but the misalignment of the software cost model. By recalibrating the optimizer to understand the specific costs of data movement and RMP checks, the system can automatically select execution plans that minimize these penalties.

The authors emphasize that their approach is lightweight: it introduces negligible overhead because it only modifies the optimizer's cost estimation logic without adding new execution-time mechanisms. They position this work as a necessary step beyond benchmarking, moving toward optimization by addressing the "hardware–software mismatch" that causes legacy DBMSs to underperform in confidential environments. The paper concludes that while workload characteristics influence the degree of improvement, a reasonable cost-model calibration is sufficient to significantly mitigate the performance gap between KVMs and CVMs.