Benchmarking Quantum Kernel Support Vector Machines Against Classical Baselines on Tabular Data: A Rigorous Empirical Study with Hardware Validation

1. Problem Statement

Quantum Kernel Methods (QKMs) are theoretically promising for supervised learning, leveraging parameterized quantum circuits to map data into high-dimensional Hilbert spaces where inner products serve as kernel evaluations for Support Vector Machines (SVMs). However, rigorous empirical evidence demonstrating a practical "quantum advantage" on real-world tabular data is scarce.

Existing benchmarks suffer from methodological flaws, including:

• Comparison against weak classical baselines (e.g., only linear SVMs).
• Use of simple hold-out validation rather than strict nested cross-validation (risking optimistic bias).
• Lack of statistical significance testing.
• Reliance solely on ideal simulations without hardware validation.

This study aims to address these gaps by conducting a large-scale, rigorous benchmark of Quantum Kernel SVMs (QSVMs) against strong classical baselines across diverse datasets, incorporating hardware validation on real quantum processors.

2. Methodology

The study employs a comprehensive experimental framework involving 970 experiments across nine binary classification datasets (from UCI/OpenML) with varying sample sizes ($N \in [208, 4601]$) and dimensions ($d \in [3, 60]$).

Experimental Design

• Validation Protocol: Strict nested cross-validation (5 outer folds $\times$ 3 or 5 inner folds) to ensure unbiased performance estimation and hyperparameter tuning ($C$).
• Quantum Feature Maps: Four distinct families were evaluated:
  1. Rot2DoF: Product-state encoding (no entanglement).
  2. Belis: CNOT ladder entanglement.
  3. Sakhnenko10: Ring-CNOT topology.
  4. ZZFeatureMap: Pairwise ZZ-interaction encoding.
• Classical Baselines: Linear, Radial Basis Function (RBF), and Polynomial (degree 3) kernels.
• Backend Types:
  1. Ideal Simulation: Statevector simulation (noise-free).
  2. Noisy Simulation: Density matrix simulation with depolarizing noise.
  3. Hardware: Execution on IBM ibm_fez (Heron r2, 156 qubits) using the compute-uncompute circuit.
• Quantum Kernel Training (QKT): A subset of experiments utilized learnable parameters to optimize Kernel-Target Alignment (KTA) via L-BFGS-B.
• Preprocessing: Dimensionality reduction (PCA, NMF, or Tree-based selection) was applied per fold to map features to $k \in \{3, \dots, 14\}$ dimensions, matching qubit constraints.

Analysis Phases

The study is structured into four analytical phases:

1. Statistical Significance: Paired Wilcoxon signed-rank tests and Kruskal-Wallis factorial analysis.
2. Learning Curves: Evaluation of data efficiency across six training fractions (10%–100%).
3. Hardware Validation: Direct comparison of hardware kernel matrices against ideal simulations.
4. Seed Sensitivity: Reproducibility analysis across 16 random seeds (8,400 SVM fits).

3. Key Contributions

• Rigorous Benchmarking: The first study to simultaneously combine 9 datasets, strict nested CV, statistical significance testing, hardware validation, and spectral analysis.
• Hardware Validation: Demonstrated high fidelity ($r \geq 0.976$) between real quantum hardware (IBM Heron r2) and ideal simulations for kernel matrices at the tested circuit scales.
• Spectral Analysis ("Goldilocks Hypothesis"): Provided a mechanistic explanation for QSVM underperformance by analyzing the eigenspectra of kernel matrices.
• Comprehensive QKT Evaluation: Quantified the computational overhead and convergence stability of trainable quantum kernels.
• Open Science: The full benchmark suite (~18,700 lines of code, 683 cached kernel matrices) is publicly available.

4. Key Results

A. Performance Comparison

• No Quantum Advantage: In 29 pairwise comparisons (best quantum vs. best classical per dataset), none reached statistical significance at $\alpha = 0.05$.
• Dominance of Classical Kernels: Classical kernels (specifically RBF) outperformed quantum kernels on 8 out of 9 datasets. The best quantum kernel (Rot2DoF) achieved a mean Balanced Accuracy (BA) of 0.649, compared to 0.830 for the best classical kernel.
• The Exception (Haberman): The only dataset showing a robust quantum advantage was Haberman (survival analysis), where quantum kernels outperformed classical baselines by +3.2 pp (BA 0.581 vs. 0.549). This was confirmed significant in the seed sensitivity analysis ($p=0.004$).

B. Spectral Analysis (The "Goldilocks" Hypothesis)

Spectral decomposition of kernel matrices revealed why quantum kernels underperform:

• Belis (Entangled): Produced near-identity matrices (flat spectrum, effective rank ratio 0.40–0.74), lacking discriminative structure.
• Rot2DoF (Product State): Produced near-rank-1 matrices (highly concentrated, top-1 eigenvalue >50%), collapsing information.
• RBF (Classical): Occupied an intermediate "Goldilocks zone" (effective rank 0.06–0.07), providing sufficient structure for decision boundaries without collapsing information.
• Conclusion: Current fixed quantum feature maps fail to produce the intermediate spectral profile required for effective SVM classification.

C. Learning Curves & Data Efficiency

• Quantum kernels exhibited steeper learning slopes on 6/8 datasets, suggesting they learn faster initially.
• However, due to a significantly lower baseline, they failed to close the gap with classical methods even at 100% training data.

D. Hardware Validation

• Fidelity: Hardware kernel matrices correlated with ideal simulations with Pearson $r \geq 0.976$ (mean $r=0.990$).
• Noise as Regularizer: In 4/6 hardware experiments, noise slightly improved performance (mean +1.5 pp) compared to ideal simulation, likely acting as implicit regularization.
• Entanglement Impact: Product-state encodings (Rot2DoF) achieved higher hardware fidelity than entangled ones (Belis) due to shallower circuit depths (22 vs. 62+ gates).

E. Quantum Kernel Training (QKT)

• QKT achieved the single competitive result: BA = 0.968 on the Breast Cancer dataset (approaching the classical ceiling of 0.976).
• Cost: This came at a massive computational cost of ~2,000x overhead compared to classical baselines due to iterative kernel recomputation.
• Stability: QKT showed low convergence rates (13.6%) and high parameter instability across folds.

5. Significance and Implications

• Dataset Dominance: A Kruskal-Wallis analysis revealed that dataset choice accounts for 73% of performance variance, while kernel type accounts for only 9%. This implies that claims of quantum advantage based on a single dataset are fragile.
• Design Guidelines: The study suggests that future quantum feature maps must be designed to target intermediate spectral profiles (avoiding both flatness and extreme concentration) rather than relying on generic entanglement structures.
• Benchmarking Standards: The authors advocate for strict nested cross-validation, multiple diverse datasets, and statistical significance testing as the new standard for QML claims.
• Hardware Readiness: While current hardware faithfully reproduces simulation results, the lack of advantage on standard tabular data suggests that near-term quantum advantage in this domain is unlikely without problem-specific circuit designs or advanced error mitigation.

Conclusion: The paper concludes that for standard tabular data, current quantum kernel methods do not offer a practical advantage over classical SVMs. The observed underperformance is structurally rooted in the spectral properties of current feature maps, and the computational overhead of training these kernels is currently prohibitive.