Machine Learning Decoding of Circuit-Level Noise for Bivariate Bicycle Codes

Technical Summary: Machine Learning Decoding of Circuit-Level Noise for Bivariate Bicycle Codes

Problem Statement
Fault-tolerant quantum computing relies on classical decoders to interpret syndrome measurements and infer errors. While Machine Learning (ML) decoders have shown promise for the surface code, their application to Quantum Low-Density Parity-Check (QLDPC) codes remains underexplored, particularly for circuit-level noise. Circuit-level noise is more complex than code-capacity noise because it includes faulty measurements and error propagation, causing the input space to grow exponentially with the number of syndrome rounds. Furthermore, QLDPC codes lack the local stabilizer structure of the surface code, making standard convolutional or local-decoder approaches inapplicable. Existing benchmarks like Belief Propagation with Ordered Statistics Decoding (BP-OSD) are effective but suffer from high variance in runtime and cubic worst-case complexity, posing a challenge for real-time decoding.

Methodology
The authors propose a recurrent, transformer-based neural network designed to decode circuit-level noise on Bivariate Bicycle (BB) codes. The model aims to approximate a maximum likelihood decoder by predicting the probability of logical measurement flips (errors anti-commuting with logical operators) rather than reconstructing the full circuit-level error.

Key architectural and training innovations include:

1. Code-Aware Self-Attention: To address the lack of local structure in QLDPC codes, the authors adapt code-aware self-attention (originally for classical codes). This mechanism restricts attention weights between syndrome bits based on the Tanner graph structure, specifically considering detectors within a distance of two. This serves as an inductive bias to accelerate training and improve robustness.
2. Recurrent Architecture: The model employs a recurrent structure to process syndrome measurement rounds sequentially. This reduces the input size per iteration, improving trainability compared to processing all rounds simultaneously.
3. Conditional Probability Output: Instead of outputting a fixed set of error classes (which would be exponential in the number of logical qubits), the model outputs a conditional probability distribution. It uses autoregressive prediction to determine if specific logical errors have occurred.
4. Multi-Stage Training with Latent Space Predictions: Direct training on circuit-level noise proved difficult. The authors utilize a multi-stage training protocol inspired by "chain of thought" prompting. Initially, the model is trained to predict intermediate logical measurement flips after every round. Gradually, the training shifts to latent space predictions for intermediate rounds, where the model outputs internal representations without committing to a final error correction until the final syndrome round. This prevents the model from making premature decisions with insufficient data.

Experimental Setup
The study evaluates the model on two BB code instances:

• [[72, 12, 6]] BB code: A smaller code with 12 logical qubits.
• [[144, 12, 12]] BB code: A larger code with 12 logical qubits.

Performance is measured against a standard BP-OSD implementation (using third-order OSD for error rates and zero-order for runtime benchmarks) and an Integer Programming (IP) decoder (a most-likely error decoder). The experiments simulate memory experiments with circuit-level depolarizing noise, varying the physical error rate ($p$) and the number of noisy syndrome measurement rounds ($NR$).

Key Results

• [[72, 12, 6]] Code:

  • Logical Error Rate: At a physical error rate of $p = 0.1\%$, the ML decoder achieves a logical error rate approximately 5 times lower than BP-OSD-3. However, it remains roughly 5 times higher than the optimal IP decoder.
  • Runtime: The ML decoder exhibits consistent runtime, whereas BP-OSD shows a wide distribution with significant outliers. The ML model is roughly an order of magnitude faster than the worst-case BP-OSD instances.
  • Generalization: The model, trained at $p=0.6\%$, generalizes well to lower error rates.
  • Scalability: The model's performance improves significantly as the number of syndrome rounds ($NR$) increases, showing better error suppression per round compared to BP-OSD.

• [[144, 12, 12]] Code:

  • Logical Error Rate: The ML decoder performs comparably to BP-OSD-3 for physical error rates between $0.3\%$ and $0.7\%$. However, for $p < 0.3\%$, the ML decoder's logical error rate degrades significantly (e.g., roughly 7 times worse than BP-OSD at $p=0.1\%$), suggesting the current architecture struggles to scale to larger code distances.
  • Runtime: The speed advantage over BP-OSD is maintained.

Significance and Claims
The paper presents initial evidence that machine learning decoders can outperform conventional decoders (specifically BP-OSD) on small QLDPC codes in terms of both logical error rate and worst-case runtime consistency. The authors claim that their work provides a baseline for future ML decoders on QLDPC codes.

However, the authors are modest regarding the scalability of their current approach. They explicitly state that the results suggest more complex architectures and/or training procedures are necessary to achieve desired logical error rates on larger code sizes. The paper does not claim that the current model solves the decoding problem for large-scale QLDPC codes but rather demonstrates the viability of the approach and identifies specific areas (such as architecture scaling and training data density at low error rates) where future work is required. The authors also note that while their decoder is sufficient for Pauli-Based Computation (PBC) via logical measurement flips, real-time decoding for large computations will require further developments, such as parallel window decoding strategies.