⚛️ quantum physics

RubriQ: Rubric-Guided Group Relative Policy Optimization for Constraint-Aware Quantum Circuit Synthesis

RubriQ is a scalable, HPC-driven framework that leverages rubric-guided Group Relative Policy Optimization (GRPO) with GPU-accelerated simulation to automatically synthesize fault-tolerant quantum circuits that achieve significant T-gate compression while strictly adhering to hardware constraints and maintaining high fidelity.

Original authors: Ziqing Guo, Ziwen Pan

Published 2026-07-09

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Ziqing Guo, Ziwen Pan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a complex machine using a very specific, expensive, and fragile set of Lego bricks. You have a master blueprint (the algorithm you want to run), but the instructions you have are written in a vague, high-level language like "build a flying car."

The problem is that the factory floor (the quantum computer) has strict rules:

The Bricks are Rare: One specific type of brick (the "T-gate") is incredibly expensive to make. You want to use as few of them as possible.
The Layout is Weird: Some machines (like IBM's) only let you connect bricks that are right next to each other. Others (like IonQ's) let you connect any brick to any other brick.
The Goal: You need to turn that vague "flying car" idea into a precise, step-by-step instruction manual that uses the fewest expensive bricks possible while obeying the factory's layout rules.

RubriQ is a new system designed to solve this puzzle automatically. Here is how it works, broken down into simple concepts:

1. The "Smart Writer" (The LLM)

Instead of using a rigid computer program that follows a fixed list of rules, RubriQ uses a Large Language Model (LLM). Think of this as a very smart, creative writer who has read millions of instruction manuals.

You give the writer a prompt: "Write a quantum circuit for a 4-qubit Fourier Transform."
The writer tries to generate the code. Sometimes it writes perfect code; sometimes it writes nonsense or leaves out steps.

2. The "Strict Teacher" (The Rubric)

In the past, when training AI to do this, the computer would only say "Good job" or "Bad job" at the very end. This is like a teacher waiting until the final exam to tell a student they failed, without explaining why. This makes learning slow and frustrating.

RubriQ introduces a Rubric, which is like a detailed grading rubric a teacher uses. Instead of a simple "Pass/Fail," the system checks the generated code on five specific dimensions:

Clifford Score: Did you use the cheap, common bricks?
T-Count Score: Did you minimize the expensive, rare bricks?
Hardware Score: Does this code actually fit on the specific machine (IBM or IonQ) you are targeting?
Fidelity Score: Does the machine actually do what you asked it to do?
Efficiency Score: Is the instruction manual short and tidy?

The system gives the AI a score for each of these areas. This provides a "dense" signal—lots of feedback—so the AI knows exactly which parts of its code to fix, rather than guessing.

3. The "Group Contest" (GRPO)

To teach the AI, RubriQ uses a method called Group Relative Policy Optimization (GRPO).

Imagine you ask the AI to solve the problem 8 times in a row.
You don't need a separate "judge" AI to tell you which one is best. Instead, you just compare the 8 answers against each other.
The one with the highest rubric score gets a "thumbs up," and the ones with lower scores get a "thumbs down."
The AI learns by looking at the winners and losers within its own group and adjusting its writing style to produce more "winners" next time.

4. The "Super-Factory" (HPC)

Training this AI is incredibly heavy lifting. Simulating quantum circuits is like trying to calculate the weather for every possible future at the same time; it requires massive computing power.

The researchers ran this on NERSC Perlmutter, a supercomputer with hundreds of powerful graphics cards (GPUs).
They built a pipeline where the AI generates code, a parser checks if it's readable, and a simulator runs it on the "virtual factory floor" to check the scores.
Because they used the "Rubric" method, the AI learned 2 to 3 times faster than previous methods that relied on vague "Pass/Fail" signals.

The Results

When they tested RubriQ:

It saved resources: It managed to reduce the number of expensive "T-gates" by an average of 3.3 times compared to standard tools.
It was accurate: The circuits it wrote actually worked when tested on real quantum computers from IBM and IonQ.
It was efficient: It reached its goals in fewer training steps, saving massive amounts of electricity and computer time.

In short: RubriQ is a system that teaches a creative AI writer how to build quantum circuits by giving it a detailed checklist (the rubric) and letting it learn by comparing its own attempts against each other, all running on a massive supercomputer to ensure the results are ready for real-world quantum machines.

Technical Summary: RubriQ – Rubric-Guided Group Relative Policy Optimization for Constraint-Aware Quantum Circuit Synthesis

1. Problem Statement

The synthesis of fault-tolerant quantum circuits (FTQC) that are both algorithmically correct and hardware-compatible represents a critical bottleneck in scalable quantum computing. The primary challenges include:

Resource Overhead: In FTQC, non-Clifford operations (specifically T-gates) require magic-state distillation, consuming orders of magnitude more physical resources than Clifford gates. Minimizing T-count is essential for reducing distillation overhead and scheduling complexity.
Hardware Constraints: Near-term devices impose strict topological and native gate constraints (e.g., IBM's heavy-hex connectivity vs. IonQ's all-to-all connectivity), requiring circuits to be transpiled efficiently to avoid excessive SWAP overhead.
Limitations of Existing Methods:
- Heuristic Compilers (e.g., Qiskit, TKET) rely on static rewrite rules and lack the generative capacity to discover novel algebraic simplifications.
- Standard Reinforcement Learning (RL) approaches often suffer from sparse terminal rewards, leading to high-variance gradients and poor exploration. They typically operate on narrow action spaces (gate insertion/deletion) rather than generating circuits from scratch.
- Neural Critics: Training multi-billion-parameter critic networks for RL introduces severe memory and communication bottlenecks in high-performance computing (HPC) environments.

2. Methodology: The RubriQ Framework

RubriQ reframes quantum circuit synthesis as a structured code-generation task for Large Language Models (LLMs), optimized via Group Relative Policy Optimization (GRPO) without a learned value-function critic.

A. LLM-as-Synthesizer

The system treats the generation of OpenQASM or Qiskit Python code as a conditional code-generation task.

Input: Natural language prompts specifying target algorithms (e.g., QFT, Hamiltonian simulation) and constraints.
Model: A 7B-parameter instruction-tuned code LLM (Qwen2.5-Coder-7B) fine-tuned with Low-Rank Adaptation (LoRA).
Generation: The policy samples $N=8$ candidate circuits per prompt using autoregressive sampling.
Parsing: A multi-strategy parser extracts executable circuit objects from the LLM's raw text output. Unparseable outputs are immediately assigned a reward of zero.

B. Rubric-Guided Reward Engine

Instead of a neural critic, RubriQ employs a deterministic, programmatic scoring engine that evaluates circuits across five orthogonal dimensions. The total reward $R(y)$ is a weighted sum:
$R(y) = w_C S_C + w_T S_T + w_H S_H + w_F S_F + w_E S_E$

Clifford Fraction ( $S_C$ ): The proportion of gates that are Clifford operations (efficient to simulate).
T-Count Reduction ( $S_T$ ): An exponential decay function penalizing high T-gate counts, prioritizing the elimination of the first few T-gates due to the batched nature of distillation.
Hardware Compatibility ( $S_H$ ): Evaluates compliance with specific backend topologies (e.g., IBM Heavy-Hex, IonQ All-to-All), penalizing SWAP routing overhead and non-native gates.
Functional Fidelity ( $S_F$ ): Computes process fidelity via the Hilbert-Schmidt inner product between the generated unitary and the target algorithm, ensuring algorithmic correctness.
Circuit Efficiency ( $S_E$ ): Rewards compactness and lower circuit depth.

C. GRPO Training Loop

RubriQ utilizes Group Relative Policy Optimization (GRPO) to update the policy:

Group Normalization: For a given prompt, the rewards of the $N$ sampled completions are normalized by subtracting the group mean and dividing by the group standard deviation.
Critic-Free: This approach eliminates the need for a learned value function, relying instead on the dense, deterministic signals provided by the rubric.
Advantage Estimation: The normalized advantage $\hat{A}_i$ drives the policy gradient updates, allowing the model to learn from graded feedback rather than sparse binary success/failure signals.

D. HPC Systems Architecture

To handle the computational cost of fidelity simulation (scaling as $O(4^{n_q})$ ), RubriQ is deployed on the NERSC Perlmutter supercomputer:

Hardware: NVIDIA A100 (80 GB) clusters connected via Slingshot-11.
Optimization: Uses DeepSpeed ZeRO-2 for distributed gradient and optimizer state partitioning.
Simulation: Integrates GPU-accelerated CUDA-Q simulation directly into the RL loop for high-throughput fidelity evaluation.
Efficiency: Achieves >85% strong-scaling efficiency across 8 GPUs.

3. Key Contributions

LLM-as-Synthesizer Formulation: The first application of billion-scale, instruction-tuned LLMs to gate-level quantum synthesis with RL-based post-training, moving beyond small transformer models or heuristic rules.
Multi-Objective Rubric Reward Engine: A novel replacement for neural critics with a domain-grounded, interpretable scoring system that provides dense, bounded rewards across Clifford, T-count, hardware, fidelity, and efficiency dimensions.
Scalable HPC Architecture: A full training pipeline implementation on NERSC Perlmutter that demonstrates strong scaling and reduces total simulation calls by 2–3× compared to sparse-reward baselines due to faster convergence.
Hardware-Validated Evaluation: Successful deployment of synthesized circuits on physical IBM Heron-3 and IonQ Forte processors via an automated transpilation pipeline.
Experimental Performance: Demonstrated superior performance across 1,500 benchmark circuits, achieving significant T-gate compression and faster convergence than existing baselines.

4. Results

T-Gate Compression: RubriQ achieves a mean T-gate compression of 3.31× across benchmark datasets, significantly outperforming sparse-reward RL baselines (2.05×) and rule-based compilers (Qiskit O3: ~1.4×, TKET O2: ~1.7×).
Convergence Speed: The framework converges 2–3× faster than sparse-reward RL methods, requiring fewer training epochs to reach target compression rates.
Resource Efficiency: Due to faster convergence, RubriQ reduces total GPU-hours by approximately 2.6× compared to Sparse-PPO and Sparse-GRPO baselines.
Constraint Adherence: The system maintains less than 1% hardware-constraint violations and achieves high functional fidelity (>99% in many cases).
Hardware Validation: Top-ranked circuits were successfully executed on IBM and IonQ hardware, confirming practical applicability beyond simulation.

5. Significance and Claims

The paper claims that RubriQ establishes a new paradigm for automated scientific discovery in quantum computing by unifying LLM-driven code generation with verifiable, rubric-based reinforcement learning. Its significance lies in:

Scalability: Proving that LLM-based synthesis can be scaled to HPC environments to solve the exponential complexity of quantum circuit optimization.
Constraint Awareness: Successfully addressing the dual requirements of FTQC readiness (T-count minimization) and near-term hardware compatibility (topology constraints) within a single optimization framework.
Efficiency: Demonstrating that dense, programmatic reward shaping is computationally superior to sparse rewards and neural critics for this domain, enabling the training of large models on realistic quantum workloads.
Practicality: Validating that AI-generated circuits are not only theoretically optimal but also executable on current generation quantum hardware.

The authors position RubriQ as a critical step toward automated, high-performance computing-driven pipelines for generating fault-tolerant, hardware-ready quantum circuits at scale.