HPC-vQPU: A Service-Export Architecture for Virtual QPUs on Batch-Scheduled HPC Systems

This paper presents HPC-vQPU, a service-export architecture that enables secure, device-faithful virtual QPU simulation on batch-scheduled HPC systems by decoupling a cloud-facing control plane from an HPC-resident execution plane using outbound coordination and immutable, calibration-aware device snapshots.

The Gist

The Big Problem: Two Different Worlds

Imagine you have a very powerful, high-tech kitchen (a Supercomputer) that can cook massive meals for thousands of people at once. However, this kitchen has strict rules:

1. No Walk-ins: You can't just walk in and order a meal. You have to fill out a form, wait in line, and the kitchen staff will only start cooking when they have a free stove.
2. No Direct Contact: Once your food is being cooked in the kitchen, the chefs are isolated. They can't call you on the phone to ask questions or tell you how it's going. They can only send a note out when they are done.

Now, imagine you are a Quantum Scientist. You want to run a very specific, delicate experiment (a "Quantum Circuit") that requires a specific set of ingredients and a specific cooking style (calibration and topology). You expect a service where you can say, "Make this dish on the 'IBM Fez' stove," and get the result back instantly.

The Conflict: The scientist wants an interactive, "order-and-wait" experience. The Supercomputer only offers a "fill-out-a-form-and-wait-in-line" experience. If you try to force the kitchen to talk directly to you, you break the kitchen's security rules.

The Solution: HPC-VQPU

The authors built a system called HPC-VQPU to bridge this gap. Think of it as a Smart Waiter who knows exactly how to talk to both the scientist and the kitchen without breaking any rules.

Here is how it works, step-by-step:

1. The Two-Part Team

The system is split into two distinct roles, just like a restaurant has a Front of House and a Back of House.

• The Control Plane (Front of House): This is the part the scientist talks to. It looks like a normal, friendly app. It takes your order, checks if your ingredients are valid, and gives you a receipt. It lives outside the secure kitchen.
• The Execution Plane (Back of House): This is the part inside the secure kitchen. It's a humble "Runner" (an agent) that lives on the kitchen's entry desk. It cannot call the Front of House; it can only ask the Front of House for work.

2. The "Outbound Only" Rule (The One-Way Door)

The kitchen has a strict security policy: No one inside can call out to the outside world to start a conversation. The outside world can't call in either.

• How HPC-VQPU solves this: The Runner inside the kitchen keeps knocking on the door (polling) and asking, "Do you have any orders for me?"
• The Front of House never calls the Runner. It just waits for the Runner to ask. This keeps the kitchen secure because no "backdoors" are opened.

3. The "Snapshot" Contract (The Frozen Recipe)

This is the most important part of the paper.

• The Problem: Quantum computers are like living things; their "flavor" (calibration) changes every day. If you order a dish today, but the kitchen doesn't start cooking it until tomorrow, the ingredients might have changed, and the dish won't taste right.
• The Old Way: If you just sent a request, the kitchen might look up the recipe when it starts cooking. But by then, the recipe might have changed, or the kitchen might be too busy to look it up.
• The HPC-VQPU Way: When the Runner asks for an order, the Front of House doesn't just say "Go cook this." It hands over a Frozen Recipe Card (a "Snapshot").
  • This card contains the exact state of the ingredients, the stove settings, and the cooking instructions at the exact moment the Runner took the order.
  • The Runner takes this card, goes into the isolated kitchen, and cooks only using that card. It doesn't need to look up the recipe again.
  • Why this matters: Even if the kitchen's main recipe book changes an hour later, your dish is cooked using the "Frozen Recipe" you were given. The result is guaranteed to be consistent with the specific "Virtual Quantum Computer" you asked for.

4. The "Claim" (Taking Ownership)

Imagine a busy kitchen with two runners.

• The Risk: If both runners ask for the same order at the same time, they might both try to cook it, wasting ingredients and getting two different results.
• The Fix: The Front of House has a special lock. When a Runner asks for work, the Front of House says, "Okay, YOU have this order now." It instantly marks the order as "Taken" and gives the Frozen Recipe Card to that specific Runner.
• If another Runner asks for the same order a second later, the Front of House says, "Sorry, that's already taken." This ensures the job is done exactly once.

5. The "Heartbeat" (Checking In)

Since the kitchen runners can't call out, how does the Front of House know they are still alive?

• The Runner sends a little "I'm still here" signal (a heartbeat) every few seconds.
• If the Runner crashes or disappears, the Front of House notices the heartbeats stop. It doesn't panic; it just waits for a human manager to say, "Okay, let's give that order to a different Runner." This prevents the system from getting stuck or losing data.

What Did They Prove?

The authors tested this system on a real supercomputer (Setonix) and proved:

1. It's Fast Enough: The "waiter" doesn't slow down the cooking. The extra time added is tiny and doesn't get worse as the cooking gets harder.
2. It's Accurate: The "Frozen Recipe" works. When they used real-world data that changes, the system cooked the dish using the current recipe at the moment of ordering, not an old one.
3. It's Safe: Even if the Runner crashes, the system knows exactly what happened and can recover without losing the order or cooking it twice.
4. It's Secure: It works perfectly without ever breaking the supercomputer's security rules (no one inside calls out to start a conversation).

Summary

HPC-VQPU is a clever way to let scientists use a super-powerful, secure supercomputer as if it were a friendly, interactive quantum computer. It does this by using a "Runner" that asks for work, handing out "Frozen Recipe Cards" so the cooking is consistent, and making sure no one breaks the kitchen's strict security rules. It turns a rigid, bureaucratic system into a smooth, reliable service.

Technical Summary

Technical Summary: HPC-VQPU

Problem Statement

The paper addresses the fundamental mismatch between the requirements of device-aware quantum simulation and the operational constraints of secure, batch-scheduled High-Performance Computing (HPC) systems. While modern quantum research increasingly relies on simulators that model specific device topologies, native gate sets, and calibration-derived noise parameters, HPC infrastructure is designed for scheduler-mediated execution rather than interactive, device-oriented services.

Three primary challenges prevent the direct export of quantum simulation as a service from HPC environments:

1. Connectivity Inversion: Secure HPC clusters typically prohibit inbound connections to compute nodes. Quantum developers expect to submit circuits to a named backend and retrieve results interactively, but HPC security models only allow outbound communication from login nodes. A standard "push" model where a service dispatches jobs to workers is structurally unavailable.
2. Semantic Preservation: A "virtual QPU" is not merely a circuit executor; it is a software-defined endpoint exposing topology, calibration, and noise constraints. In a batch system, a task may sit in a queue for hours while the device's calibration state changes. If the execution semantics are not bound to a specific point in time, the result may no longer correspond to the requested virtual device, rendering the simulation scientifically invalid for compiler development or routing studies.
3. Operational Complexity: Batch-mediated execution introduces a failure surface beyond simple application errors, including scheduler delays, node failures, agent disappearance, and malformed artifacts. A viable service must reconcile interactive lifecycle semantics (submit, monitor, retrieve) with the partial observability and asynchronous nature of HPC job scheduling.

Methodology and Architecture

The authors propose HPC-VQPU, a service-export architecture that separates service authority from scheduler-governed execution. The system is organized into two distinct planes that communicate exclusively through outbound, agent-initiated messages, preserving the HPC security boundary.

1. Two-Plane Decomposition

• Control Plane (Cloud-Facing): Hosted externally to the HPC cluster, this plane manages the API, task lifecycle, device identity, and event projection. It is authoritative for task state but contains no scheduler-facing functionality or inbound connectivity to the cluster.
• Execution Plane (HPC-Resident): Implemented by an unprivileged vqpu_agent running on an HPC login node. This agent initiates all coordination. It polls the control plane for work, claims tasks, submits jobs to the scheduler (e.g., Slurm), and reports results. It owns no durable service truth and is reconstructible upon restart.

2. The Snapshot Contract

The core abstraction is the device snapshot ($\Delta$), a graph-structured representation of a virtual device containing topology, native gates, and calibration parameters (coherence times, error rates).

• Claim-Time Binding: To resolve the tension between queue delay and calibration freshness, the snapshot is bound atomically at claim time, not at submission. When an agent claims a task, the control plane transfers ownership and provides the immutable snapshot ($\delta_i$) as part of the execution contract.
• Hermetic Execution: The compute-node runner receives a local payload containing the circuit and the bound snapshot. It executes the simulation (e.g., using Qiskit-Aer/cuQuantum) entirely offline, deriving noise models from the snapshot without contacting the control plane. This ensures that execution semantics are invariant to device state changes occurring after the claim.

3. Task Lifecycle and Protocol

The system employs a five-state task automaton ($\{QUEUED, RUNNING, COMPLETED, FAILED, CANCELLED\}$) managed by the control plane.

• Asymmetric Protocol: The agent initiates CLAIM, HEARTBEAT, and REPORT operations. The control plane never pushes state inward.
• Atomic Ownership: The CLAIM operation is the linearization point for ownership. It atomically transitions a task from QUEUED to RUNNING and assigns a unique owner, preventing duplicate execution by concurrent agents.
• Recovery: If an agent fails, the task remains in RUNNING with a stale owner. Recovery requires an explicit administrative action (e.g., REQUEUE), ensuring that ambiguity is not silently resolved and execution lineage is preserved.

Key Contributions

The paper presents four main contributions:

1. Service-Export Architecture: A control/execution-plane decomposition that resolves connectivity inversion via outbound-only coordination, allowing secure HPC systems to export device-aware simulation without weakening security postures.
2. Device Snapshot Abstraction: A portable, graph-structured execution contract that binds topology and calibration semantics atomically at claim time, ensuring that scheduled jobs remain faithful to the requested virtual device state.
3. Scheduler-Aware Lifecycle: A five-state task automaton with an asymmetric claim/report/heartbeat protocol that reconciles interactive service semantics with batch-mediated execution and partial observability.
4. Reference Implementation and Validation: A production deployment at the Pawsey Supercomputing Research Centre using Setonix GPUs, Qiskit-Aer, and IBM Fez calibration data, demonstrating the feasibility of the architecture.

Experimental Results

The authors validate the architecture through regression tests and end-to-end production experiments, yielding the following results:

• Bounded Overhead: Service export overhead is additive and bounded. Interface-visible latency (admission and polling) remains constant (~0.4s admission, ~22-27s queue-to-claim) regardless of circuit size (28–32 qubits). The exponential scaling of simulation remains confined to the compute substrate.
• Calibration Fidelity: Experiments using real IBM Fez calibration data versus ideal (null) snapshots demonstrate that the snapshot content directly alters output distributions. The system correctly propagates calibration-derived noise parameters to the simulator.
• Claim-Time Binding: When a device's calibration is mutated administratively after submission but before claim, the system correctly executes the task using the post-mutation (ideal) state. This confirms that the snapshot is bound at claim time, preventing stale execution.
• Concurrency and Integrity: In a multi-agent test with 50 tasks and two concurrent agents, all tasks completed exactly once with a near-even split (24:26). No duplicate claims or ambiguous ownership occurred.
• Recovery: After an agent crash, three RUNNING tasks were successfully recovered via explicit requeueing, completing without duplicate execution or silent state reinterpretation.

Significance and Claims

The paper claims that HPC-VQPU demonstrates that secure, batch-scheduled supercomputers can export semantically rich, interactive services without ceasing to operate like HPC systems. The significance lies in the ability to preserve device fidelity (topology, native gates, calibration) across the scheduler boundary.

The authors position this work not as a performance optimization for simulation kernels, but as a systems solution for the service boundary. By treating the device snapshot as a first-class, time-indexed contract, the architecture enables quantum software workflows (compiler evaluation, routing, noise studies) to interact with HPC resources as if they were interactive virtual QPUs. The design invariants (outbound-only connectivity, atomic claim, hermetic execution) are presented as necessary conditions to make results scientifically interpretable as execution on a specific virtual device, even in the face of queue delays and infrastructure failures.

The paper concludes modestly, acknowledging that the evaluation is centered on a single production site and simulation path, and that the architecture provides authoritative lifecycle semantics rather than a proof of "exactly-once" execution in the full distributed-systems sense. However, it asserts that the proposed separation of authority and realization offers a viable path for integrating HPC-scale simulation into the interactive quantum software ecosystem.