⚛️ quantum physics

Quantum Error Management in Practice: A Cross-Stack Benchmark

This paper benchmarks commercial quantum error management solutions (IBM Qiskit Runtime, Q-CTRL, and Qedma QESEM) on a 156-qubit IBM Heron processor, demonstrating that while all managed approaches significantly reduce error compared to raw execution, they present distinct trade-offs between accuracy improvements and execution time costs.

Original authors: Daniel Sierra-Sosa, Begonya Garcia-Zapirain, Cristian Marquez, Kelly Garces

Published 2026-08-07

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniel Sierra-Sosa, Begonya Garcia-Zapirain, Cristian Marquez, Kelly Garces

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a secret message across a stormy ocean. You have a fleet of tiny, incredibly fragile boats (quantum computers) that can carry information faster and in more complex ways than any ship ever built. But here's the catch: the ocean is rough. Every time a wave hits a boat, it might tip over, or a sailor might drop a crate, or the compass might spin wildly. In the world of quantum computing, these "waves" are called noise. They are tiny errors that happen when the computer tries to do math. Because these boats are so sensitive, even a little bit of noise can turn a brilliant answer into total gibberish.

Scientists have been trying to fix this for years. The "perfect" solution is like building a giant, armored fleet where every boat is protected by a shield of other boats, so if one tips, the others keep it upright. This is called Quantum Error Correction, but right now, it's too heavy and expensive to build for the boats we have today. So, instead of waiting for the perfect fleet, engineers are trying two other tricks. The first is Error Suppression: trying to make the boats sturdier and the sailors more careful before the storm hits, so fewer mistakes happen in the first place. The second is Error Mitigation: admitting that mistakes will happen, but then using a smart computer program after the trip to figure out what the original message probably was, effectively "cleaning up" the noise in the results. The big question for anyone using these quantum computers today is: Which trick actually works better, and how much does it cost?

This paper is like a massive, fair race to find the answer. The authors took a brand-new, powerful quantum computer with 156 "boats" (qubits) and ran the exact same set of difficult puzzles through three different teams of software helpers. One team was the computer's own built-in tools (IBM), one was a high-tech "performance manager" called Q-CTRL, and the third was a specialized "error cleaner" called QESEM. They wanted to see which team could get the most correct answers and how much "fuel" (computer time) each one burned to do it.

The results were a tale of two different strategies. When the goal was to get a single, perfect answer (like guessing a hidden code or preparing a specific state of matter), the Q-CTRL team was the clear winner. They managed to keep the boats on course even when the storms got really bad. For example, on a tricky puzzle called "Quantum Phase Estimation" with 30 boats, the computer's own tools got the right answer zero times out of 32,768 tries. But Q-CTRL got it right 12.7% of the time. That's the difference between a failed mission and a working one. Surprisingly, the computer's own built-in "noise symmetrizer" (a tool called measurement twirling) didn't help much at all; in fact, it sometimes made things slightly worse.

When the goal shifted to measuring average values (like calculating the average magnetism of a chain of atoms), the race got more interesting. Here, both Q-CTRL and QESEM beat the computer's raw, unassisted performance by a huge margin. Q-CTRL reduced the total error by about 3 times, while QESEM reduced it by nearly 5 times. However, there was a catch: QESEM was a "heavy lifter." To get those super-accurate results, it used about 7.5 to 11 times more computer time than Q-CTRL. Q-CTRL, on the other hand, gave great results using almost the same amount of time as the basic settings.

The paper also found that the "best" tool depends entirely on what you are measuring. The computer's built-in tools helped improve the measurement of magnetism but actually made the measurement of correlations (how atoms talk to each other) worse. This shows that there is no single "magic button" that fixes everything; the right tool depends on the specific job you are doing. Ultimately, the study shows that while we wait for the perfect, fault-tolerant quantum computers of the future, these modern software tools can already rescue us from the noise, provided you pick the right one for your budget and your goal.

Technical Summary: Quantum Error Management in Practice: A Cross-Stack Benchmark

Problem Statement
Quantum processors have surpassed the 100-qubit threshold, yet they operate within the Noisy Intermediate-Scale Quantum (NISQ) regime where noise degrades every circuit execution. While full quantum error correction remains too resource-intensive for routine use, a commercial ecosystem has emerged offering error suppression (preventing errors during execution) and error mitigation (removing biases in post-processing). However, independent, like-for-like comparisons of these offerings on identical hardware and workloads are scarce. Practitioners lack empirical data to determine the value of each layer in the software stack, often relying on vendor-specific benchmarks that vary in device, workload, and timing.

Methodology
The authors conducted a cross-stack benchmark on the ibm_pittsburgh processor, a 156-qubit IBM Quantum Heron r3 device. The study utilized a seeded, job-level "submit-and-collect" workflow to ensure reproducibility, recording logical circuits, provider configurations, and job identifiers in a JSON manifest. The benchmark evaluated three primary providers:

IBM Quantum: Using native Qiskit Runtime primitives (Sampler and Estimator) with varying resilience levels (raw, measurement twirling, TREX-plus-twirling).
Q-CTRL: Using the "Performance Management" Qiskit Function, an autonomous pipeline for error-aware layout, compilation, and suppression.
Qedma: Using the "QESEM" Qiskit Function, which combines noise characterization, execution suppression, and unbiased post-processing mitigation.

The evaluation was split into two tracks:

Sampler Track: Focused on preserving raw bitstrings. Workloads included Bernstein–Vazirani (25–75 qubits), Quantum Phase Estimation (10–30 counting qubits), GHZ-state preparation (25–50 qubits), and randomized mirror circuits (25–100 qubits). Success was measured by the probability of obtaining the exact ideal bitstring or valid set. All providers used a matched budget of $2^{15}$ (32,768) shots, except QESEM which was not evaluated here as it lacks a Sampler interface.
Estimator Track: Focused on expectation values of observables. Workloads involved chain-averaged magnetization ( $m_X$ ) and nearest-neighbor correlators ( $c_{ZZ}$ ) of an eight-layer transverse-field Ising model at 25, 50, and 75 qubits. Results were compared against a numerically exact matrix-product-state (MPS) reference. IBM and Q-CTRL used matched shot budgets, while QESEM operated under a precision target (0.1) and a QPU-time cap (600s).

Key Contributions

Standardized Benchmarking: The paper provides the first independent comparison of IBM's native resilience features, Q-CTRL's suppression pipeline, and Qedma's suppression-plus-mitigation stack on the same hardware and logical circuits.
Sampler Results: Demonstrates that managed suppression (Q-CTRL) significantly outperforms raw execution and measurement twirling alone in preserving exact bitstrings, particularly for deep circuits like Quantum Phase Estimation and wide randomized mirror circuits.
Estimator Results: Quantifies the trade-off between accuracy and QPU time. It shows that while IBM's TREX-plus-twirling improves magnetization, it can degrade correlator accuracy. In contrast, managed stacks (Q-CTRL and QESEM) provide consistent improvements across observables, with QESEM achieving the highest accuracy at a higher time cost.
Reproducibility Framework: Introduces a protocol for seeding and manifesting jobs to isolate software stack performance from hardware calibration drift.

Results

Sampler Performance:
- Raw vs. Managed: Unmitigated success probabilities collapsed rapidly with circuit width. For Quantum Phase Estimation at 30 counting qubits, raw execution yielded 0% success, whereas Q-CTRL achieved 12.69%.
- Measurement Twirling: Enabling measurement twirling on IBM primitives provided no consistent improvement over raw execution for exact-bitstring metrics; in some cases, it slightly reduced success probability.
- Q-CTRL: Consistently delivered the highest success probabilities across all Sampler workloads. For randomized mirror circuits at 100 qubits, Q-CTRL achieved 76.5% success compared to 9.1% for raw execution.
- Cost: Q-CTRL's reported QPU times were within the same order of magnitude as IBM's configurations for structured workloads.
Estimator Performance:
- Accuracy: The aggregate Mean Absolute Error (MAE) across all observable-size cases was 0.0883 for IBM raw, 0.0807 for IBM TREX-plus-twirling, 0.0285 for Q-CTRL, and 0.0188 for QESEM.
- Error Reduction: Relative to raw execution, Q-CTRL reduced aggregate error by a factor of 3.10, and QESEM by 4.70.
- Observable Dependence: IBM's TREX-plus-twirling improved magnetization accuracy but systematically underestimated the correlator, resulting in a modest net aggregate improvement.
- Time vs. Accuracy: Q-CTRL operated at ~28 seconds per job (comparable to IBM's 17.8–37.8s). QESEM required 211–311 seconds per job (7.5–11.1× the time of Q-CTRL) to achieve the lowest error.

Significance and Claims
The paper claims to offer practical guidance for practitioners navigating the current commercial quantum software landscape. It establishes that:

Managed suppression is effective: For sampling tasks where bitstrings must be preserved, autonomous suppression pipelines (like Q-CTRL) can rescue algorithms that fail completely under raw execution, often at no additional QPU time cost.
Mitigation is observable-dependent: Built-in mitigation strategies (like IBM's TREX) may improve specific observables while degrading others, suggesting that "one-size-fits-all" resilience settings may not be optimal.
Distinct operating points exist: For estimation tasks, providers offer different trade-offs. Q-CTRL represents a lower-time, high-accuracy option, while QESEM represents a maximum-accuracy option at a significantly higher time cost.
Limitations: The authors note that the study is limited to single execution jobs per configuration and does not include IBM's Zero-Noise Extrapolation (ZNE) or Probabilistic Error Cancellation (PEC) due to compute budget constraints. Furthermore, QESEM was not evaluated on sampling tasks due to interface limitations.

The paper concludes that as the field moves toward fault tolerance, independent, like-for-like measurements of software layers will remain essential for guiding practitioners in a rapidly evolving commercial ecosystem.

More like this