Technical Summary: Scalable Testing of Quantum Error Correction

Problem Statement

Benchmarking quantum error correction (QEC) algorithms is a critical step toward fault-tolerant quantum computing. The standard approach involves estimating the logical error rate ($P_L$) by simulating quantum circuits with injected faults. However, as QEC codes scale to larger distances ($d$) and lower physical error rates ($p$), logical errors become increasingly rare events.

The state-of-the-art tool, Stim, utilizes randomized fault injection to benchmark QEC implementations. While efficient for small distances (up to $d \approx 10$), Stim scales poorly for larger distances because it relies on Monte Carlo sampling. For high-quality QEC algorithms, the probability of a logical error is so low that Stim must run a massive number of tests to achieve statistical confidence. In many cases, Stim fails to observe a single logical error within reasonable time budgets (e.g., two hours on a desktop) for distances $d \ge 13$, rendering it unable to provide an estimate. This creates a bottleneck for researchers aiming to characterize QEC performance for real-world applications.

Methodology: ScaLER

The authors propose ScaLER (Scalable Logical Error Rate Testing), a method that combines stratified fault injection with extrapolation to overcome the limitations of random sampling. The core insight is that the "fault space" can be partitioned by the weight of the injected errors (the number of faults), and that high-weight samples can be sampled efficiently to predict the behavior of low-weight samples.

1. Modeling QEC-Test Data as S-Curves

The authors observe that the logical error rate as a function of error weight ($P_L^w$) forms an S-curve:

• Fault-Tolerant Zone: For weights $w \le t$ (where $t = \lfloor (d-1)/2 \rfloor$), the logical error rate is exactly zero.
• Transition Zone: As weight increases beyond $t$, the error rate rises steeply.
• Saturation Zone: At very high weights, the error rate saturates at 0.5 (equivalent to a coin flip).

The paper models this behavior using a continuous function $f(w)$. While IBM's existing "Min-Fail Enclosure Model" fits the data, the authors argue it lacks precision near the fault-tolerant threshold. They introduce a new S-curve model that explicitly encodes the fault-tolerant threshold $t$ by defining the function as zero for $w \le t$ and using a specific ansatz for $w > t$:
$$f_t^{\text{ours}}[\mu, \alpha, \beta](w) = \begin{cases} 0 & w \le t \\ \frac{1}{2} \cdot \frac{1}{1 + e^{-\frac{w-\mu}{\alpha} + \frac{\beta}{\sqrt{w-t}}}} & w > t \end{cases}$$
This model is fitted to data points collected from specific weight subspaces.

2. Stratified Sampling and Adaptive Search

Instead of sampling uniformly across all weights, ScaLER focuses on high-weight samples where logical errors occur frequently, then extrapolates back to the critical low-weight region. The algorithm operates in three stages:

1. Binary Search Stage: The algorithm searches for two boundary weights:
   • $w_{err}$: The lowest weight where 1,000 samples yield a logical error (approx. $P_L^w \approx 0.001$).
   • $w_{sat}$: The highest weight where the logical error rate is $\le 0.25$.
2. Initial Sampling Stage: Samples are collected in 5 subspaces evenly distributed between $w_{err}$ and $w_{sat}$. A Y-curve (log-transformed S-curve) is fitted to this data to estimate a "sweet spot" ($w_{sweet}$).
3. Iterative Stage: The algorithm iteratively samples subspaces moving from $w_{err}$ down toward $w_{sweet}$. After each subspace, the Y-curve is refitted, and the estimate of $w_{sweet}$ is updated. The process stops when 5 subspaces are tested or the time budget is exhausted.

3. Logical Error Rate Estimation

Once the S-curve parameters ($\mu, \alpha, \beta$) are fitted, the total logical error rate is estimated by summing the product of the modeled error rate and the probability of that weight occurring in the circuit:
$$\hat{P}_L = \sum_{w=w_{min}}^{w_{max}} f_t^{\text{ours}}(w) \times \binom{n}{w} p^w (1-p)^{n-w}$$
where $n$ is the number of locations in the circuit, and the summation covers the "critical region" (typically $[np - 5\sigma, np + 5\sigma]$).

Key Contributions

1. ScaLER Tool: An open-source tool (C++/Python) that implements stratified fault injection and S-curve fitting.
2. Novel S-Curve Model: A mathematical model that explicitly incorporates the fault-tolerant threshold $t$, offering better extrapolation accuracy near the threshold compared to existing models.
3. Adaptive Sampling Algorithm: A method (Algorithm 1) that dynamically determines the optimal range of weights to sample, balancing testing cost and accuracy.
4. Scalability Demonstration: Empirical evidence showing that ScaLER can estimate logical error rates for distances up to $d=17$ within a two-hour time budget on a standard desktop, a regime where Stim fails completely.

Results

The authors evaluated ScaLER against Stim using Surface codes, Toric codes, and Bivariate Bicycle (BB) codes with distances up to 17 and physical error rates of $p=0.0005$ and $p=0.0001$.

• Scalability (RQ1): For a Surface code with $d=17$ and $p=0.0005$, Stim observed zero logical errors in two hours. ScaLER estimated a logical error rate of $1.51 \times 10^{-11}$ with high confidence in the same timeframe. ScaLER successfully provided estimates for all tested configurations where Stim failed.
• Agreement (RQ2): In cases where Stim could generate ground-truth estimates (typically $d \le 10$), ScaLER's estimates agreed within an order of magnitude. For $d=3$ and $d=9$, the relative error was below 10%. For intermediate distances ($d=5, 7$), the error was approximately 20–40%, which the authors attribute to the selection of the "sweet spot" hyperparameter ($\Gamma$). Increasing $\Gamma$ improves accuracy at the cost of higher sampling.
• Model Comparison (RQ3): The authors' S-curve model outperformed IBM's model in fitting the data and extrapolating to the critical region. Variants of their model with different exponents ($s$) were tested, with $s=1/2$ (the square root term in the denominator) yielding the highest accuracy.

Significance and Claims

The paper claims that ScaLER provides a scalable alternative to randomized fault injection for benchmarking high-quality QEC algorithms. By shifting the testing focus to high-weight error subspaces and leveraging the predictable S-curve behavior of logical error rates, the method drastically reduces the sampling effort required to estimate extremely low logical error rates.

The authors emphasize that their approach enables the characterization of QEC circuits at scales ($d=17$) and error rates relevant to future fault-tolerant applications (such as factoring large integers) that are currently inaccessible to standard Monte Carlo tools like Stim. They acknowledge that the method relies on the assumption that QEC test data follows an S-curve and that the ground truth for very large distances remains elusive, but they demonstrate that their estimates are robust and consistent with lower-distance benchmarks.