Image classification via a quantum-inspired strategy involving a mixture of experts

Technical Summary: Image Classification via a Quantum-Inspired Mixture of Experts

Problem Statement
Pattern recognition in physical image processing often requires extracting relevant features from massive datasets where storage and processing time are constrained. Standard Convolutional Neural Networks (CNNs) address this through diffusion-based smearing and block-wise pooling to downsample data. However, these classical mechanisms are inherently lossy; they degrade fine-grained structural details to reduce computational complexity. While quantum algorithms offer potential efficiency gains, significant advantages typically require genuine quantum data inputs. This work addresses the challenge of improving feature extraction and classification efficiency for standard image datasets by proposing a hybrid classical-quantum framework that minimizes information loss during the downsampling process.

Methodology
The authors propose a Quantum Mixture of Experts (QMoE) framework, implemented as a hybrid system where quantum-inspired subroutines replace specific components of a standard CNN, followed by a classical fully connected neural network (FCL) for final classification. The methodology consists of five key stages:

1. State Encoding:

   • Input images (MNIST and Fashion-MNIST, originally 28×28) are padded to 32×32 and thresholded to binary values ($c_{x,y} = \pm 1$).
   • Images are embedded into 10-qubit quantum states using amplitude encoding: $|\psi\rangle = \frac{1}{32} \sum c_{x,y} |x, y\rangle$. This reduces the spatial resource requirement exponentially, allowing subsequent operations to act on the encoded state rather than raw pixels.

2. Convolutional Smearing (Unitary vs. Diffusion):

   • Classical Baseline: Uses a discrete Laplacian operator for iterative diffusion: $X^{t+1}_{i,j} = (1-\alpha)X^t_{i,j} + \frac{\alpha}{4}\sum X^t_{neighbors}$. This is a lossy process.
   • Quantum Strategy: Replaces diffusion with non-lossy unitary smearing. The image grid is split into even and odd subsets (checkerboard pattern). Local unitary operators ($U_o, U_e$) mix amplitudes at the corners of elementary squares using Hamiltonian blocks derived from Pauli matrices. The mixing is controlled by a parameter $s = \sin \theta$, which is optimized during training.

3. Mixture of Experts (MoE):

   • Multiple "experts" are defined by distinct parameter values ($\theta_i$) used in the unitary smearing.
   • Independent Strategy: Each expert processes the image, and predictions are combined via majority vote.
   • Joint Strategy: Features extracted by all experts are concatenated and processed together by the classifier. The paper emphasizes this joint approach.

4. Block-wise Pooling (Stabilizer Codes):

   • Classical Baseline: Reduces image resolution (32×32 $\to$ 16×16 $\to$ 8×8 $\to$ 4×4) by weighted averaging of 3×3 windows.
   • Quantum Strategy: Utilizes the [[5,1,3]] perfect nondegenerate stabilizer code. Since pixel coordinates are encoded in 5 qubits per axis, the code extracts syndromes (expectation values of commuting generators) to produce 16 distinct features per axis. This replaces irreversible downsampling with a deterministic syndrome extraction that preserves specific structural information.

5. Classification:

   • The extracted features (16 features per expert per layer) are fed into a standard fully connected neural network (FCL) with a funnel architecture (512 $\to$ 256 $\to$ 128 $\to$ 10).
   • The system is trained using the Adam optimizer with CrossEntropyLoss (including label smoothing) on GPU workstations using PyTorch.

Key Contributions

• Hybrid Framework: The paper introduces a specific architecture where classical diffusion is replaced by unitary convolution and classical pooling is replaced by quantum stabilizer code syndrome extraction.
• Mixture of Experts: It demonstrates that a joint decision-making process among multiple experts (each using different smearing parameters) significantly outperforms both independent expert voting and single-expert classical baselines.
• Efficient Feature Extraction: By leveraging amplitude encoding and stabilizer codes, the method reduces the input dimensionality from 784 pixels to 16 features (or $16 \times N_{exp} \times N_{lyr}$ in the MoE case) while retaining more relevant information than classical lossy downsampling.
• Practical Implementation: The algorithm is implemented entirely on classical GPU hardware using a "quantum-inspired" approach (tensor operations simulating quantum states), making it immediately deployable without requiring actual quantum hardware.

Results
The framework was benchmarked against classical diffusion baselines on MNIST and Fashion-MNIST datasets:

• MNIST Dataset:
  • The classical baseline (optimal $\alpha=1.5$) achieved 95.02% test accuracy.
  • The Quantum Joint Expert strategy (16 experts, 16 layers) achieved 97.63% test accuracy.
  • This represents a reduction in the failure rate by approximately a factor of two compared to the classical method.
• Fashion-MNIST Dataset:
  • Due to higher shape ambiguity, the classical baseline peaked at 76.80%.
  • The Quantum Joint Expert strategy (18 experts, 16 layers) achieved 86.20%.
  • Again, the failure rate was reduced by roughly a factor of two.
• Computational Overhead: The time per epoch for the joint expert quantum-inspired model showed only a moderate increase compared to the classical baseline, making the approach computationally practical on current GPU workstations.

Significance and Claims
The paper claims that the proposed quantum-inspired strategy offers a clear advantage in image classification tasks, particularly for datasets with complex structural features like Fashion-MNIST. The primary significance lies in the demonstration that:

1. Unitary convolution retains more image information than lossy classical diffusion.
2. Stabilizer code-based syndrome extraction provides a robust method for feature reduction that outperforms standard pooling.
3. Joint expert analysis extracts more relevant features than independent analysis, validating the "Mixture of Experts" approach within this quantum-inspired context.

The authors conclude that while the current implementation is classical (simulated), the framework is designed to be executable on a quantum processor (as detailed in the Appendix), suggesting a viable path toward hybrid classical-quantum processors for future machine learning tasks. They modestly note that the strategy is a practical alternative to existing classical schemes with moderate overhead, rather than a replacement for all classical methods.