💻 computer science

RubriQ: Rubric-Guided Group Relative Policy Optimization for Constraint-Aware Quantum Circuit Synthesis

RubriQ is a scalable, HPC-driven framework that leverages rubric-guided Group Relative Policy Optimization (GRPO) to automate the synthesis of quantum circuits that simultaneously satisfy algorithmic correctness, surface-code cost minimization, and near-term hardware constraints.

Original authors: Ziqing Guo, Ziwen Pan

Published 2026-08-03

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ziqing Guo, Ziwen Pan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a machine that can solve problems impossible for any computer we have today. This machine is a quantum computer, a device that uses the weird rules of tiny particles to do math at lightning speed. But here's the catch: these machines are incredibly fragile. To make them work, we need to translate our big, complex ideas into a very specific, low-level language of "gates" (like switches) that the machine understands.

The problem is that this translation is a nightmare. If you just ask a standard computer to do it, the math gets so huge it crashes. If you try to build the machine with today's technology, it's too noisy and error-prone. So, scientists are stuck in a middle ground: they need to design circuits that are perfect enough for the future super-machines, but simple enough to actually run on today's shaky prototypes. It's like trying to write a recipe for a five-star banquet that a toddler can actually cook without burning the kitchen down.

Enter RubriQ, a new tool that acts like a super-smart, tireless editor for these quantum recipes. Instead of just guessing and checking, RubriQ uses a giant AI that learns by trying thousands of variations at once, guided by a strict "rubric" (a grading sheet) that tells it exactly how to improve. It doesn't just look for "good enough"; it hunts for the perfect balance between being mathematically flawless and being practical enough to run on real hardware right now.

The RubriQ Story: Teaching an AI to Be a Quantum Architect

Think of designing a quantum circuit like asking a student to write a story. In the past, if you asked a computer to write a quantum program, it was like giving a student a vague prompt like "Write something cool" and hoping they didn't make up nonsense. If they got it wrong, you'd just say, "Nope, try again," with no idea why it was wrong. This is what older methods did: they gave a "sparse reward," meaning the AI only got a point if the answer was 100% perfect, and zero points for everything else. This made learning incredibly slow and frustrating, like trying to learn to ride a bike in the dark.

RubriQ changes the game by acting like a strict but helpful teacher with a detailed grading rubric. Instead of just saying "Pass" or "Fail," it breaks the grade down into five specific categories:

Did it work? (The story makes sense).
Is it efficient? (Did it use too many words? In quantum terms, this is about minimizing "T-gates," which are the expensive, resource-hungry parts of the code).
Is it Clifford-heavy? (Are there too many fancy, hard-to-make moves, or mostly simple ones?).
Will it run on today's machines? (Does it fit the specific shape of the hardware, like fitting a square peg in a round hole?).
Is it fast? (How many steps does it take?).

The paper introduces a method called Group Relative Policy Optimization (GRPO). Imagine the AI is a chef trying to invent a new dish. Instead of cooking one meal and waiting for a review, RubriQ asks the chef to cook eight different versions of the dish at the same time. Then, it compares them against each other. If one version is slightly better than the others, the AI learns to make more of that kind. If one is a disaster, it learns to avoid that path. This "group" comparison is much faster and more stable than waiting for a single perfect answer.

How They Built the Machine

To make this work, the researchers had to build a massive engine. They couldn't just run this on a laptop; it needed the power of NERSC Perlmutter, a supercomputer cluster at Lawrence Berkeley National Laboratory. They used 8 NVIDIA A100 GPUs (the heavy-duty graphics cards used for AI) to run the simulation.

Here's the clever part: The AI doesn't just guess. It uses a programmatic rubric. This means the "teacher" isn't a black-box neural network that might be confused; it's a set of hard-coded rules that check the math, the cost, and the hardware limits instantly. If the AI generates a circuit that looks cool but fails the math, the rubric gives it a zero immediately. If it generates a circuit that works but uses too many expensive "T-gates," the rubric gives it a lower score, nudging it to be more efficient.

They also solved a major bottleneck: simulation speed. Checking if a quantum circuit works usually requires simulating the whole thing, which gets exponentially harder as you add more qubits (quantum bits). RubriQ integrates CUDA-Q, a tool that lets them run these simulations directly on the GPUs, making the process thousands of times faster than running it on a standard CPU.

What They Found

The results are promising, though the authors are careful to frame them as a significant step forward rather than a magic bullet.

It works better than the old ways: When tested on 1,500 different quantum tasks, RubriQ achieved a 96% pass rate for correctness. That means almost every time it tried, it produced a circuit that actually did the math right.
It's much more efficient: The most important metric for quantum computers is the "T-count" (the number of expensive gates). RubriQ managed to compress the number of these gates by an average of 3.31 times compared to circuits that were just "correct" but not optimized. This is a huge deal because fewer T-gates mean the computer needs fewer resources and less time to run.
It learns faster: Because it uses these detailed rubrics instead of waiting for a perfect score, RubriQ converged (learned the task) 2 to 3 times faster than other reinforcement learning methods that rely on sparse rewards.
It's ready for real hardware: The team didn't just stop at simulations. They took the best circuits generated by RubriQ and ran them on real quantum computers from IBM and IonQ. They found that the circuits were compatible with these machines, with less than 1% violation of hardware constraints.

What This Means

The paper suggests that by treating quantum circuit synthesis as a code-generation task guided by a strict, multi-dimensional rubric, we can automate the creation of high-quality quantum programs. It argues against the idea that we need complex, learned "critic" networks to judge the AI's work; instead, a clear, rule-based scoring system works better and is cheaper to run.

While the paper doesn't claim to have solved all of quantum computing's problems, it demonstrates a viable path forward. By combining the creative power of Large Language Models (LLMs) with the rigorous discipline of a programmatic rubric, RubriQ offers a way to design circuits that are not only mathematically sound but also practical enough to run on the noisy, imperfect machines we have today, while preparing for the fault-tolerant giants of the future. It's a bridge between the messy reality of current hardware and the pristine world of future quantum computing.

Technical Summary: RubriQ – Rubric-Guided Group Relative Policy Optimization for Constraint-Aware Quantum Circuit Synthesis

1. Problem Statement

The transition to scalable quantum computing is bottlenecked by the classical compilation problem: translating high-level algorithms into low-level gate sequences that satisfy both fault-tolerant quantum computing (FTQC) resource constraints and near-term hardware executability.

FTQC Constraints: In surface-code architectures, non-Clifford operations (specifically $T$ gates) require expensive magic-state distillation, consuming orders of magnitude more physical resources than Clifford gates. Minimizing the $T$ -count and $T$ -depth is critical for reducing the classical computational budget and scheduling complexity.
Near-Term Constraints: Current Noisy Intermediate-Scale Quantum (NISQ) devices (e.g., IBM's heavy-hex topology, IonQ's all-to-all connectivity) impose strict limits on native gate sets, qubit connectivity, and circuit depth.
Limitations of Existing Methods: Mainstream compilers (Qiskit, TKET) rely on static heuristic rules and lack generative capabilities to discover novel algebraic simplifications. Previous Reinforcement Learning (RL) approaches often suffer from sparse terminal rewards, high-variance gradients, and narrow action spaces (gate insertion/deletion) that prevent the discovery of structurally novel circuits. Furthermore, training RL agents for quantum synthesis at scale is computationally prohibitive due to the $O(4^{n_q})$ complexity of fidelity simulation.

2. Methodology: The RubriQ Framework

RubriQ reframes quantum circuit synthesis as a large language model (LLM) code-generation task, optimized via Group Relative Policy Optimization (GRPO) without a learned value-function critic.

2.1 LLM-as-Synthesizer

The system treats the synthesis of OpenQASM or Qiskit Python code as a conditional generation task.

Base Model: A 7-billion-parameter instruction-tuned code-generation LLM (Qwen2.5-Coder-7B) serves as the policy $\pi_\theta$ .
Parameter Efficiency: The model is fine-tuned using Low-Rank Adaptation (LoRA) with rank 64, keeping trainable parameters at ~0.4% of the total, enabling deployment on 4–8 NVIDIA A100 GPUs.
Generation: For each prompt (natural language specification of an algorithm), the model samples $N=8$ candidate circuits autoregressively.

2.2 Rubric-Guided Reward Engine

Instead of a sparse binary reward or a neural critic, RubriQ employs a programmatic, multidimensional rubric to generate dense, deterministic rewards. The reward function $R(y)$ is a weighted sum of five bounded component scores ( $S_d \in [0, 1]$ ), gated by a hard correctness condition:
$R(y) = g(c) \cdot [w_F S_F + w_C S_C + w_T S_T + w_H S_H + w_E S_E]$
Where $g(c)$ is a hard gate that sets the reward to zero if the average gate fidelity $F_{avg} < 0.99$ .

The five dimensions are:

Functional Fidelity ( $S_F$ ): Measured via Hilbert-Schmidt inner product against the target unitary.
Clifford Fraction ( $S_C$ ): The ratio of Clifford gates to total operational gates, encouraging Clifford-heavy circuits.
$T$ -Count Reduction ( $S_T$ ): An exponential decay score based on the total equivalent $T$ -gate count (accounting for decompositions of Toffoli and arbitrary rotations).
Near-Term Hardware Validation ( $S_H$ ): A score evaluating compatibility with specific backends (IBM Eagle/Heron, IonQ Forte/Aria), penalizing non-native gates and SWAP routing overhead.
Circuit Efficiency ( $S_E$ ): A depth-normalized compactness metric.

2.3 GRPO Training Loop

The framework utilizes Group Relative Policy Optimization (GRPO), which eliminates the need for a separate critic network.

Advantage Estimation: For a group of $N$ completions sampled from the same prompt, advantages $\hat{A}_i$ are computed by normalizing rewards within the group: $\hat{A}_i = (R_i - \mu_R) / (\sigma_R + \epsilon)$ .
Scalability: This approach reduces the variance of gradient estimates compared to sparse rewards and avoids the memory/communication bottlenecks of maintaining a multi-billion-parameter critic across distributed GPUs.
HPC Integration: The training loop integrates GPU-accelerated CUDA-Q simulation directly into the RL loop for high-throughput fidelity evaluation. The system is deployed on NERSC Perlmutter using DeepSpeed ZeRO-2 for distributed gradient and optimizer-state partitioning.

3. Key Contributions

LLM-as-Synthesizer Formulation: The first application of billion-scale instruction-tuned LLMs to gate-level quantum synthesis using RL-based post-training, moving beyond small transformer models or static heuristics.
Multi-Objective Rubric Reward Engine: A domain-grounded scoring system that decouples FTQC cost metrics (Clifford+T) from near-term hardware validation, replacing neural critics with interpretable, deterministic rubrics.
Scalable HPC Systems Architecture: A full training pipeline implemented on NERSC Perlmutter with >85% strong-scaling efficiency across 8 A100 GPUs, utilizing DeepSpeed ZeRO-2 and CUDA-Q to manage the computational cost of $O(4^{n_q})$ simulations.
Near-Term Hardware Validation: An automated pipeline that validates synthesized circuits on physical IBM Heron-3 and IonQ Forte processors, demonstrating executability beyond simulation.
Empirical Performance: Demonstrated significant improvements in synthesis quality and convergence speed compared to sparse-reward baselines and traditional compilers.

4. Experimental Results

Evaluated on 1,500 benchmark circuits (QFT, QPE, Hamiltonian simulation, data encoding) across 2–8 qubits:

Correctness: Achieved a per-dataset correctness pass rate of at least 96%.
$T$ -Gate Compression: Attained a mean $T$ -gate compression of 3.31 $\times$ over correct-only circuits, significantly outperforming sparse-reward RL baselines (2.05 $\times$ ) and traditional tools (TKET, Qiskit).
Convergence: The rubric-guided approach converged 2–3 $\times$ faster than sparse-reward baselines, reducing the total number of training epochs required.
Constraint Violations: Maintained less than 1% near-term hardware constraint violations.
Hardware Validation: Successfully deployed top-k circuits on IBM Heron-3 and IonQ Forte, confirming the pipeline's ability to generate hardware-aware code.
Computational Efficiency: The dense rubric shaping reduced total simulation calls by 2–3 $\times$ compared to sparse-reward methods, translating to significant GPU-hour savings (e.g., 9.2 vs. 24.2 GPU-hours for 8-qubit tasks).

5. Significance and Claims

The paper posits that RubriQ establishes a new paradigm for automated quantum circuit synthesis by unifying LLM generative capabilities with rigorous, domain-specific constraints.

Separation of Concerns: By explicitly separating FTQC cost metrics from near-term hardware validation, RubriQ addresses the conflicting requirements of future fault-tolerant systems and current NISQ devices.
Elimination of the Critic: The work demonstrates that for deterministic, programmatically computable tasks like quantum synthesis, a learned value-function critic is unnecessary; dense, rubric-based rewards combined with GRPO provide sufficient signal for effective policy optimization.
HPC-Driven Pipeline: The research highlights the necessity of High-Performance Computing (HPC) resources for training AI-driven quantum tools, providing a blueprint for scaling RL-based synthesis to larger qubit counts where classical simulation becomes intractable.
Robustness: The framework is shown to be robust across different algorithmic families and qubit counts, avoiding overfitting to specific circuit templates.

The authors conclude that RubriQ provides an automated, high-performance pipeline for generating surface-code-cost-aware circuits, bridging the gap between theoretical algorithm design and practical hardware execution.