⚛️ quantum physics

Evaluating Calibration-Based Digital Twins for IBM Quantum Hardware Simulation

This paper evaluates calibration-based digital twins for IBM Quantum hardware, demonstrating that models constructed from downloadable calibration CSV files often achieve the closest agreement with real device outcomes, while highlighting that twin accuracy depends on specific devices and transpilation settings rather than being universally transferable.

Original authors: Edgars Bautra, Maksims Dimitrijevs, Abuzer Yakaryilmaz

Published 2026-03-17

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Edgars Bautra, Maksims Dimitrijevs, Abuzer Yakaryilmaz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to perfect a new recipe for a soufflé. You have a very expensive, temperamental oven (the Quantum Computer) that is in high demand. Everyone wants to use it, so you have to wait in a long line, and you only get a few minutes to bake before someone else takes over.

If you keep burning your soufflé in the real oven while waiting for your turn, you waste precious time and resources. What you really need is a perfect replica of that specific oven sitting right in your kitchen. You can test your recipe there, see exactly how it behaves, and only take it to the real oven when you're sure it will work.

This paper is about building that "perfect replica oven" for IBM's quantum computers. The authors call this a Digital Twin.

Here is a breakdown of what they did, using simple analogies:

1. The Problem: The "Noisy" Real Oven

Real quantum computers are messy. They aren't perfect; they suffer from "noise" (like a drafty kitchen or a flickering flame). This noise causes errors. If you just simulate a perfect quantum computer on your laptop, it's like simulating a soufflé in a world where gravity doesn't exist—it looks great on paper, but it won't match reality.

To get useful results, your simulation needs to be noisy, just like the real machine.

2. The Solution: Building the Twin from "Instruction Manuals"

IBM provides a free "instruction manual" (a CSV file) for every quantum computer they own. This manual lists the quirks of every single part:

How long does a qubit (a quantum bit) stay alive before it gets tired? (Coherence time)
How often does a gate (a switch) fail? (Error rates)
How long does it take to read the result? (Readout time)

The authors built a workflow to take these instruction manuals and turn them into a digital twin.

The Old Way: You could ask IBM, "Give me a twin of your machine," and they would give you a pre-made one. But this only works if you have access to that specific machine right now.
The New Way (This Paper): The authors wrote a script that reads the raw instruction manual (the CSV file) and builds the twin from scratch. This means you can build a twin for any IBM machine, even ones you don't have direct access to yet, as long as you can download their manual.

3. The Experiment: The "Taste Test"

To see if their new method worked, they set up a massive taste test.

The Test Subjects: They used two real IBM ovens (called ibm_brisbane and ibm_sherbrooke).
The Recipes: They created random, complex circuits (like random soufflé recipes) of different difficulties.
The Competitors: They ran these recipes on the real ovens and compared the results against four different types of "digital twins":
1. The CSV Twin: Built manually from the downloaded instruction manual (The authors' method).
2. The Official Twin: Built automatically by IBM's software using the live machine.
3. The Noise Twin: Just the noise settings from the official machine.
4. The "Fake" Twin: A pre-packaged, slightly outdated version IBM provides for testing.

4. The Results: Who Made the Best Soufflé?

They used a math tool called "Weighted Jaccard Similarity" to measure how close the digital results were to the real results. Think of it as a "flavor match score."

The Winner: The CSV Twin (built from the downloaded manual) often matched the real oven the best! It was so accurate that it frequently scored a 95% or higher match.
The Runner-Up: The Official Twin (IBM's automatic method) was also very good and is a great "baseline" for most people.
The Loser: The Fake Twins (the pre-packaged ones) were okay, but they were a bit outdated and didn't match as well, especially when the recipes got complicated.

5. The Big Lesson: "One Size Does Not Fit All"

The most important discovery was that you cannot assume a twin works everywhere.

A twin that perfectly mimics the Brisbane oven might not perfectly mimic the Sherbrooke oven.
Changing how you prepare the recipe (called "transpilation" or "optimization") changes how the twin behaves.

The Takeaway: If you want to simulate a quantum computer accurately, you can't just grab a generic model. You need to build a specific twin for the specific machine you are targeting, using its latest instruction manual.

Summary in One Sentence

This paper proves that you can build a highly accurate "digital clone" of a quantum computer just by reading its public instruction manual, and that this custom-built clone is often more accurate than the pre-made versions provided by the company.

1. Problem Statement

Quantum computers (specifically IBM Quantum Processing Units or QPUs) are subject to long queue times and limited availability due to high demand. While classical simulators can help reduce the load on real hardware, ideal simulations fail to account for noise (decoherence, gate errors, readout errors), leading to results that diverge significantly from real-world hardware behavior.

There is a need for Calibration-Based Digital Twins: simulator configurations that use device-specific calibration data to approximate the noisy behavior of a specific real QPU. While Qiskit offers tools to generate noise models from active backends (from_backend()), there is no native, end-to-end workflow to construct these twins directly from downloadable calibration CSV files. This gap limits the ability to simulate devices that are currently unavailable to users or to create custom, persistent simulation environments.

2. Methodology

The authors developed a workflow to construct, execute, and validate digital twins using Qiskit and IBM's downloadable calibration data.

A. Digital Twin Construction

The study compares four distinct twin variants:

CSV-Built Twin: A custom noise model constructed manually from downloadable CSV calibration files.
- Process: Extracts coherence times ( $T_1$ , $T_2$ ), gate/readout error rates, and operation durations.
- Mapping: Maps these attributes to thermal_relaxation_error, depolarizing_error, and readout_error channels in Qiskit.
- Coupling Map: Reconstructs the directed coupling map (essential for transpilation) by inferring connectivity from the target qubit fields in the CSV data, as this is not explicitly listed in the standard calibration attributes.
Backend-Derived Simulator: An AerSimulator instance automatically configured via from_backend().
Backend-Derived Noise Model: A NoiseModel instance automatically generated via from_backend(), applied to a generic simulator.
Fake-Backend Snapshots: Pre-packaged "fake" backends (e.g., FakeBrisbane) provided by Qiskit, which use historical system snapshots rather than live data.

B. Experimental Setup

Target Devices: Two IBM QPUs: ibm_brisbane and ibm_sherbrooke.
Benchmarks: Three randomly generated 5-qubit circuits with depths of 10, 20, and 30.
Transpilation: Circuits were transpiled at four optimization levels (0 to 3).
Shot Budget: Determined via a preliminary study to be 100,000 shots (the platform maximum), ensuring statistical stability while remaining within execution time limits (~40 seconds per job).
Simulation Engine: The density_matrix simulator was selected as it offered the best balance of speed and accuracy compared to statevector and matrix product state methods.

C. Validation Protocol

The agreement between the real QPU and the digital twins was measured using Weighted Jaccard Similarity ( $J_w$ ) on the measured state-count distributions.

Metric: $J_w(x, y) = \frac{\sum \min(x_i, y_i)}{\sum \max(x_i, y_i)}$ .
Thresholds:
- $\ge 95\%$ : Near-identical.
- $\ge 90\%$ : Close match.
- $\ge 85\%$ : Usefully similar.

3. Key Contributions

Practical Workflow: Established a method to build executable digital twins directly from raw IBM calibration CSV files, including a workaround to reconstruct directional coupling maps required for accurate transpilation.
Comparative Analysis: Formally compared four twin variants (CSV, Backend-Simulator, Backend-NoiseModel, Fake-Backend) under a unified execution protocol.
Validation Framework: Introduced a standardized validation procedure using Weighted Jaccard Similarity to quantify the fidelity of simulation against hardware.
Empirical Study: Provided a comprehensive dataset involving 24 hardware jobs and 96 simulator runs across two different QPUs and multiple circuit depths.

4. Key Results

CSV Twins Perform Best: Twins constructed from downloadable CSV data frequently achieved the highest similarity to the real QPU.
- In 13 out of 24 tests, the CSV-based twin was the closest match to the hardware.
- For ibm_brisbane, CSV twins often exceeded the 95% similarity threshold.
Backend-Derived Twins are Strong Baselines: The automatically generated backend simulators and noise models provided competitive results (often $\approx 95\%$ similarity) and serve as practical, low-effort baselines.
Fake-Backends Lag Behind: The pre-packaged "fake" backends generally showed lower similarity, particularly when transpilation optimization levels were set above 0.
Device and Transpilation Dependence:
- Device Specificity: A twin that performs well on ibm_brisbane does not guarantee similar performance on ibm_sherbrooke.
- Optimization Sensitivity: Similarity scores fluctuated unpredictably based on transpilation optimization levels (0–3). For example, increasing optimization from 0 to 1 improved similarity on one device but degraded it on the other.
Shot Count Impact: Increasing shot counts from 1,000 to 100,000 significantly improved result stability, with the most significant gains occurring between 1,000 and 15,000 shots.

5. Significance and Implications

Accessibility: The CSV-based approach allows researchers to simulate any IBM QPU (even those currently unavailable or restricted) locally, provided the calibration data is accessible.
Validation Necessity: The study proves that digital twins cannot be assumed to be transferable. A twin validated on one device or with one transpilation setting may fail on another. Validation must be specific to the target execution setup.
Limitations of Inter-Simulator Similarity: The paper notes that different simulator variants often produce highly similar results to each other but remain distinct from the real hardware. Therefore, high agreement between simulators does not guarantee hardware fidelity.
Future Directions: The authors suggest that while calibration-based models are effective, they may need to be augmented with parameter fitting from actual circuit execution data to capture non-Markovian effects and improve distribution-level agreement.

Conclusion

The paper demonstrates that calibration-based digital twins are feasible and highly effective for reproducing IBM Quantum hardware behavior. While automated backend tools are convenient, constructing twins from raw CSV calibration data often yields superior accuracy, provided the coupling topology is correctly reconstructed. However, the results emphasize that simulation fidelity is highly context-dependent, requiring rigorous validation for specific devices and transpilation configurations.