⚛️ quantum physics

GroverFigureOfMerit: An Agnostic Figure of Merit for Quantum Backend Characterization in the NISQ Era

This paper proposes and validates "GroverFigureOfMerit," a holistic, architecture-agnostic metric based on Grover's algorithm that overcomes the limitations of static hardware parameters by evaluating quantum backend performance through dynamic success probabilities, uniformity, and leakage penalties across diverse NISQ-era providers.

Original authors: Tiago Restucha, Marcos Guillermo Lammers, Alejandro Fernández

Published 2026-07-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Tiago Restucha, Marcos Guillermo Lammers, Alejandro Fernández

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to pick the best car for a road trip, but every manufacturer gives you a different, confusing list of specs. One says, "My engine has 99% efficiency!" Another shouts, "My tires last 50,000 miles!" But they don't tell you how the car actually handles a bumpy road, a steep hill, or a sudden rainstorm. You're left guessing which one will actually get you to your destination without breaking down.

This is exactly the problem quantum computer developers face today. We are in the "NISQ era" (Noisy Intermediate-Scale Quantum), a time when quantum computers exist but are still a bit glitchy and messy. Different companies like IBM, IonQ, and Rigetti build their machines in totally different ways. They all speak different languages and show off different "static" stats, like how long a single qubit (the quantum bit) can hold its thought before forgetting it.

The authors of this paper argue that looking at these static stats is like judging a marathon runner only by how fast they can tie their shoes. It misses the point! A runner might have great shoes (low error rates) but trip over their own feet when the race gets complicated. The paper suggests that to really know which quantum computer is the best, you need to run a real race.

The "Stress Test" Race: Grover's Algorithm

To solve this, the team invented a new way to measure performance called the GroverFigureOfMerit. Think of it as a standardized "stress test" race. Instead of looking at parts, they run a specific, tricky puzzle called Grover's algorithm on the computers.

Grover's algorithm is like a game of "Find the Needle in a Haystack." In a perfect world, a quantum computer can find the needle incredibly fast. But in the real, noisy world, the hay might be shaking, the needle might be slippery, and the computer might get confused.

The authors designed this test to see how well a computer handles the whole process, including:

The Noise: The static and glitches that happen during the race.
The Translation: The work the computer has to do to translate the "haystack" puzzle into its own specific language (this is called transpilation).
The Map: The physical layout of the computer's wires (topology). Some computers have qubits that can't talk to each other directly, so they have to pass messages through neighbors, which slows things down.

How the Score Works

After running the race 2,000 times (called "shots"), the system gives the computer a score. It's not just about finding the needle; it's about how it found it.

Did it find the right answer? (Good!)
Did it find the wrong answers too? (Bad! That's "leakage.")
Did it find the right answer every single time, or was it lucky sometimes and unlucky others? (Bad! That's "non-uniformity.")

The final score is a single number that combines all these factors. If the computer is noisy or gets stuck translating the puzzle, the score drops. If it's clean and efficient, the score stays high.

The Big Experiment: Simulating the Real Thing

The team didn't just guess; they ran this test on a framework called Qonscious. This is like a universal remote control that lets you run the same race on any brand of quantum computer without changing the code.

They tested this on nine different providers (though they focused their detailed results on simulators based on real hardware from IBM and IonQ). They ran the test on two sizes of "haystacks":

A small one with 8 items (using 3 qubits).
A bigger one with 32 items (using 5 qubits).

What they found:

The Ideal Simulators: When they ran the test on a perfect, noise-free computer simulation, the score was nearly perfect (around 0.960 for the small haystack and 0.999 for the bigger one). This shows the test works as expected when there are no glitches.
The IBM Models: When they ran the test on noise models derived from real IBM processors, the scores dropped significantly. For the 32-item haystack, the IBM models collapsed to a score of about 0.040. This is basically the same as guessing randomly! The paper explains this is because the IBM computers had to work extra hard to connect distant qubits, creating a "traffic jam" of errors.
The IonQ Model: The IonQ Aria 1 model did much better. It kept its score high and didn't collapse like the IBM models. This suggests that, for this specific type of puzzle, IonQ's architecture is more resilient to noise.

What This Means (and What It Doesn't)

The paper makes it clear that this isn't a magic bullet that solves all quantum problems. The authors explicitly state that they are not trying to prove that Grover's algorithm is the best way to solve real-world search problems right now. In fact, they admit that on current noisy hardware, Grover's algorithm might not be useful for actual tasks yet.

Instead, they are using Grover's algorithm purely as a tool to measure the hardware. It's like using a crash test dummy not to see if the car is safe for passengers, but to see how the car's frame holds up during a crash.

The results are based on simulations using noise models derived from real hardware, not on running the test on live, physical quantum computers in the cloud (though the authors say testing on real machines is the "natural next step").

The Takeaway

The main finding is that this new "GroverFigureOfMerit" score can successfully tell the difference between quantum computers, even when they are built in totally different ways. It captures the messy reality of noise, translation errors, and physical layout in a single, easy-to-compare number.

The paper suggests that this approach helps developers stop guessing based on confusing spec sheets and start making informed choices about which quantum computer to use for their specific needs. However, the authors warn that the "translation" step (transpilation) is currently a huge bottleneck, adding so much extra work that it ruins the performance of some machines. They hope that in the future, smarter software can fix this so that the "race" is fair for everyone.

In short: The paper proposes a new, fair way to grade quantum computers by making them run a specific, tricky puzzle, and early simulations show that this test can spot which machines are actually ready for the big leagues and which ones are still tripping over their shoelaces.

Technical Summary: GroverFigureOfMerit for Quantum Backend Characterization

Problem Statement

The Noisy Intermediate-Scale Quantum (NISQ) era presents a significant challenge for developers attempting to select appropriate quantum hardware. Current quantum hardware providers expose capabilities through heterogeneous interfaces and proprietary metrics (e.g., coherence times $T_1/T_2$ , gate error rates) that vary widely across architectures (superconducting, trapped-ion, photonic, etc.). These static metrics suffer from three primary limitations:

Limited Predictive Power: They fail to account for the impact of the transpilation process, where high-level circuits are decomposed into native gates, often resulting in physical depths substantially greater than the original logical depth.
Lack of Contextual Relevance: Static metrics do not encapsulate the specific behavior of algorithms, ignoring how gate interactions, idle times, and topology affect final results.
Temporal Variability: Hardware properties fluctuate due to environmental factors, causing static snapshots to lose validity quickly.

Consequently, there is a need for a backend-agnostic Figure of Merit (FoM) that evaluates the holistic performance of quantum backends by capturing the combined effects of noise, transpilation, and topological constraints.

Methodology

The authors propose the GroverFigureOfMerit, an algorithmic stress test based on Grover's search algorithm. The methodology is implemented using the Qonscious framework, an open-source platform designed for resource-aware conditional execution.

1. Implementation Platform (Qonscious):
The framework utilizes Polymorphic Adapters (BackendAdapters) to abstract differences in SDKs and APIs across providers (e.g., IBM, IonQ). This allows the same logical circuit to be transpiled and executed on any supported backend without modifying the FoM code. The execution flow is conditional, enabling the FoM result to feed a decision function for runtime optimization.

2. Metric Design:
The GroverFigureOfMerit adapts the scoring function from the GRADE approach (Manor et al., 2025) to the Qonscious architecture. The metric evaluates the performance of Grover's algorithm, which quadratically amplifies the probability of target states. The score is calculated after $S=2000$ shots and incorporates:

Cumulative Relative Frequency on Targets ( $P_T$ ): The sum of probabilities for marked states.
Probability on Non-Targets ( $P_N$ ): The probability mass leaking to unmarked states.
Standard Deviation among Targets ( $\sigma_T$ ): A measure of uniformity to penalize coherent errors where one target is amplified significantly more than others.

The final score is defined as:
$\text{Score} = \max(0, P_T - \lambda \cdot \sigma_T - \mu \cdot P_N)$
Where $\lambda$ and $\mu$ are balancing factors (typically set to 1) that penalize non-uniformity and probability leakage, respectively. If the leakage penalty exceeds the target probability, the score is zero.

3. Experimental Setup:
To validate the metric without relying on real-time cloud QPU availability, the authors utilized noise models derived from real processors:

IBM Fake Providers: Models preserving the topology and error profiles of specific IBM processors.
IonQ Aria 1 Model: A noise model for the IonQ Aria 1 architecture.
Ideal Simulators: AerSimulator and IonQ Simulator for baseline comparison.

Experiments varied the search space size ( $N$ ) and the number of target states ( $M$ ) to test sensitivity to circuit depth and noise accumulation.

Key Results

The experiments demonstrated that the GroverFigureOfMerit is sensitive to noise, topology, and transpilation overhead:

Small Scale ( $N=8, n=3$ ): Ideal simulators achieved a perfect score of 1.0 at the optimal iteration count ( $M=2$ ). Noisy models (IBM Fake Providers) showed degraded scores (ranging from 0.495 to 0.792). At the algorithmic limit ( $M=4$ ), all noisy hardware scores collapsed toward 0.5, reflecting classical random guessing.
Medium Scale ( $N=32, n=5$ ): A more severe degradation was observed. While ideal simulators maintained high fidelity, IBM models collapsed completely due to decoherence, reducing scores to classical random guessing probabilities ( $\approx 0.25$ ). In contrast, the IonQ Aria 1 model demonstrated superior resilience, maintaining scores significantly above the random limit.
Scalability ( $N=8$ to $N=128$ ): As the search space scaled, IBM processors exhibited a sharp drop in scores (e.g., collapsing to $\approx 0.040$ at $N=32$ ). This degradation was attributed not only to static gate errors but specifically to the topological cost of decomposing multi-controlled gates (mcx) to connect distant qubits, which drastically increased circuit depth.

Significance and Claims

The paper claims that the GroverFigureOfMerit provides a unified and comparable score across physically distinct hardware architectures. Its primary significance lies in:

Agnostic Characterization: It enables the direct comparison of heterogeneous backends by abstracting proprietary API calls and native gate sets.
Holistic Evaluation: Unlike static metrics, it captures the end-to-end performance, including the cumulative effects of transpilation, noise, and topological constraints.
Resource Optimization: By integrating with Qonscious, the metric can conditionally trigger or cancel execution, potentially reducing costs in cloud quantum computing environments.

The authors explicitly state that the choice of Grover's algorithm is strictly for its utility as a "stress test" to evaluate routing capabilities and coherence, rather than to demonstrate algorithmic advantage on current noisy hardware. The paper identifies validation on real physical processors as the necessary next step to capture dynamic phenomena such as crosstalk and thermal fluctuations that noise models may not fully represent. Furthermore, the authors highlight the current ecosystem bottleneck: the high cost of transpilation when delegating complex gate decomposition to provider-specific compilers, suggesting a need for smarter abstractions or intermediate representations to mitigate topological overhead.