Towards Fair Benchmarking of Quantum Transfer Learning for Visual Classification

This paper establishes a controlled benchmarking methodology to fairly evaluate diverse Quantum Transfer Learning methods under unified conditions, revealing that no single approach universally dominates and highlighting the critical need for resource-aware evaluation in near-term quantum visual classification.

The Gist

Imagine you are trying to teach a very small, very expensive robot to recognize pictures. This robot (the Quantum Computer) is powerful but has a major limitation: it only has a few "brain cells" (qubits) and gets tired (noisy) if you ask it to think too deeply (deep circuits).

The paper tackles a problem called Quantum Transfer Learning (QTL). Think of it like this: instead of teaching the tiny robot to see the whole picture from scratch (which is too hard for it), you hire a giant, experienced human artist (a Classical AI) to look at the picture first. The artist describes the key features to the robot in a simple language, and the robot just has to make the final decision based on that description.

The problem the authors found is that different research teams were comparing their robots using different rules. One team used a different artist, a different picture size, and a different way of talking to the robot. It was like comparing a race car to a bicycle just because they both moved forward; you couldn't tell which was actually better.

What This Paper Did: The "Fair Play" Test

The authors created a strict, fair rulebook to test five different ways of teaching these tiny robots. They made sure every robot:

1. Listened to the same human artist (a pre-trained ResNet18 model).
2. Looked at the same pictures (Fashion-MNIST, Ants vs. Bees, and a bit of CIFAR-10).
3. Had the same amount of time and resources to train.

They tested five different "teaching styles" (Quantum Transfer Learning methods):

• DQN-QTL: The robot gets a simple, direct description and makes a quick guess.
• QPIE-QTL: The robot gets a more detailed, multi-angle description.
• AE-CQTL: The robot tries to memorize the whole description as a single, complex quantum state (like trying to swallow a whole book at once).
• PVCQTL: The robot uses a special, structured way of listening to the description to catch hidden patterns.
• ED-QTL: The robot is taught by a "teacher" robot that has already learned from the human artist, rather than learning from the raw pictures directly.

The Surprising Results

The biggest takeaway is that there is no single "best" robot. The winner depends entirely on the job you give it:

• For structured, black-and-white style pictures (Fashion-MNIST): The "Multi-Angle" (QPIE) and "Structured Listening" (PVCQTL) methods were the winners. They were accurate, but they took a long time to train (like a student who studies very hard but slowly).
• For natural, colorful pictures with few examples (Ants vs. Bees): The "Whole Book" method (AE-CQTL) won. It was surprisingly good at recognizing the difference between ants and bees, and it was actually quite fast to train.
• For the "Teacher" method (ED-QTL): It didn't do as well as expected. Just having a teacher didn't automatically make the student robot smarter; it needed more tuning.

The "Cost" of Being Smart

The paper emphasizes that accuracy isn't everything. You have to look at the "price tag."

• Some methods got 90% accuracy but took hours to train.
• Others got 89% accuracy but took minutes.
• Some methods needed more "brain cells" (qubits) to get better, but on some datasets, adding more brain cells actually made them worse or didn't help at all.

The Bottom Line

If you are building a quantum system for the near future (where resources are tight), you can't just pick the method with the highest score on a leaderboard. You have to ask:

1. What kind of pictures are you classifying? (Grayscale patterns vs. natural photos).
2. How much time do you have? (Do you need a fast result or the absolute best result?).
3. How many "brain cells" do you have? (Some methods need more qubits to work well, others don't).

The authors conclude that to move forward, scientists need to stop just shouting "Look how accurate I am!" and start saying, "Here is my accuracy, here is my cost, and here is exactly what kind of problem I am good at solving." This paper provides the ruler to measure all of that fairly.

Technical Summary

Technical Summary: Towards Fair Benchmarking of Quantum Transfer Learning for Visual Classification

Problem Statement
Quantum Transfer Learning (QTL) has emerged as a promising strategy for visual classification under near-term quantum constraints, where limited qubit counts, shallow circuit depths, and noisy operations prevent end-to-end quantum training. In this paradigm, pretrained classical backbones extract high-level features, while compact quantum circuits serve as trainable classification heads. However, the field lacks a unified basis for comparison. Existing QTL studies are difficult to benchmark against one another because they differ significantly in datasets, preprocessing pipelines, backbone architectures, qubit budgets, circuit designs, optimization hyperparameters, and reporting protocols. Consequently, it is unclear whether performance improvements stem from the transfer strategy itself or from confounding factors such as stronger classical backbones, larger qubit counts, or deeper circuits. Furthermore, accuracy alone fails to capture the practical utility of a method, as models with similar predictive performance may vary drastically in parameter counts, circuit size, and training time.

Methodology
To address these inconsistencies, the authors introduce a controlled benchmarking methodology that evaluates five representative QTL families under a unified transfer-learning pipeline. The benchmark standardizes the following elements:

• Datasets: Two primary datasets are used: Fashion-MNIST (structured grayscale multi-class) and Hymenoptera Ants vs. Bees (low-data natural-image binary classification). CIFAR-10 is included as an additional, harder natural-image setting for selected configurations to provide configuration-level evidence.
• Pipeline: All methods utilize a frozen, pretrained ResNet18 backbone for feature extraction. The extracted 512-dimensional features are projected or reduced to a quantum-compatible vector, encoded into a Parameterized Quantum Circuit (PQC), and processed before a classical readout layer generates the final prediction.
• Controlled Variables: The benchmark fixes the backbone, input preprocessing (resizing to 224x224, ImageNet normalization), and training conditions (Adam optimizer, specific learning rates, batch sizes, and epoch limits).
• Evaluated Methods: The study compares:
  1. DQN-QTL: A baseline using a "dressed" quantum head with RY angle encoding.
  2. QPIE-QTL: A method utilizing multi-axis encoding (RX, RY, RZ) per qubit.
  3. AE-CQTL: An amplitude-encoding approach that maps the 512-dimensional feature vector directly into the amplitudes of a 9-qubit state.
  4. PVCQTL: A post-variational approach using structured observable measurements (Pauli observables) rather than standard single-qubit readouts.
  5. ED-QTL: An ensemble-distillation method where a quantum student is guided by soft targets from an ensemble of classical teacher models.
• Metrics: Evaluation goes beyond accuracy to include precision, recall, F1-score, and ROC-AUC. Crucially, the benchmark reports efficiency metrics: total trainable parameters, quantum parameters, circuit width (qubit count), circuit depth, and total training time.

Key Contributions

1. Unified Benchmark Protocol: The paper establishes a controlled framework that isolates the contribution of the QTL head by freezing the classical backbone and standardizing preprocessing and training conditions across diverse methods.
2. Comprehensive Comparison: It provides the first side-by-side evaluation of five distinct QTL families (Dressed, Multi-axis, Amplitude-encoded, Post-variational, and Distillation-based) under identical resource constraints.
3. Multi-Dimensional Analysis: The study moves beyond accuracy to analyze the trade-offs between predictive performance, computational cost (training time), and architectural complexity (qubit count and circuit depth).
4. Architectural Sensitivity: The authors systematically examine how scaling qubit counts (4 vs. 6) and circuit depth affects performance, revealing that larger or deeper circuits do not universally improve results.

Results
The benchmark reveals that no single QTL family dominates across all settings; performance is highly dependent on the dataset, encoding strategy, and computational cost.

• Dataset Dependence: On Fashion-MNIST, QPIE-QTL and PVCQTL variants achieved the highest accuracy (approx. 88.28% and 88.24%, respectively). In contrast, on the Hymenoptera Ants vs. Bees dataset, AE-CQTL outperformed all other methods, reaching 94.99% accuracy. ED-QTL showed dataset-dependent competitiveness, performing better on Fashion-MNIST than on Hymenoptera.
• Scaling Effects: Increasing qubit counts from 4 to 6 improved performance for DQN-QTL and QPIE-QTL on Fashion-MNIST but yielded no gains or slight decreases on Hymenoptera. Similarly, increasing circuit depth significantly improved AE-CQTL on Fashion-MNIST (from 65.30% to 74.98%) but offered diminishing returns on Hymenoptera, where the pretrained features already provided strong class separation.
• Performance-Cost Trade-offs: The study highlights that high accuracy does not equate to efficiency. For instance, PVCQTL variants achieved competitive accuracy on Fashion-MNIST but incurred the highest training times. Conversely, ED-QTL offered moderate accuracy with significantly lower training costs. AE-CQTL demonstrated a favorable trade-off on Hymenoptera, achieving top accuracy with relatively efficient training times compared to other high-performing methods.

Significance and Claims
The paper claims that its primary significance lies in shifting the evaluation of QTL from isolated accuracy reports to a resource-aware, multi-dimensional analysis. The authors argue that for near-term quantum devices, the utility of a QTL method is defined not just by its predictive power but by its efficiency and scalability under strict resource constraints.

The findings suggest that selecting a QTL configuration requires matching the method to the specific target constraints and dataset characteristics. For example, amplitude-encoded methods (AE-CQTL) may be preferable when depth scaling is feasible and the dataset benefits from high-dimensional encoding, while angle-encoded baselines (DQN-QTL) remain robust compact options for limited qubit budgets. The authors conclude that future QTL research must prioritize controlled comparisons that jointly report performance, circuit size, parameter counts, and training time to provide actionable guidance for hybrid quantum-classical model selection.