Technical Summary: Model Stealing Through the Lens of Model Multiplicity

Problem Statement

Model stealing attacks, where adversaries query a target model to train a high-fidelity surrogate, are traditionally evaluated based on fidelity—the degree of label agreement between the surrogate and the target on a reference dataset. High fidelity is often interpreted as evidence of successful intellectual property theft, implying the surrogate is a functional clone of the target.

This paper challenges that assumption by introducing the concept of model multiplicity. It posits that query-based extraction provides only partial supervision, leaving the surrogate model underdetermined. Consequently, the extraction process does not yield a unique model but rather an extraction-induced Rashomon Set: a family of near-optimal surrogate models that achieve comparable fidelity to the target but may diverge significantly in individual predictions, fairness, and robustness. The central question is whether high-fidelity extraction truly guarantees behavioral equivalence in deployment scenarios.

Methodology

The authors propose a framework to evaluate model stealing not as a single-model fidelity problem, but as a multiplicity problem. The methodology involves:

1. Extraction Induction: Two black-box extraction attacks are employed across three domains (tabular, medical imaging, and NLP):
   • Knockoff Nets: Steals functionality via black-box interactions using adaptive query sampling.
   • Prediction-API Stealing: Trains a surrogate on query-response pairs (posterior outputs) from the target.
2. Rashomon Set Approximation: Instead of enumerating the full set of near-equivalent models (computationally infeasible for complex classes), the authors use Dropout at inference time on the extracted surrogate. By varying dropout rates (0.01 to 0.10) and sampling thousands of stochastic models, they approximate the local Rashomon Set surrounding the stolen model.
3. Evaluation Metrics: The diversity of the induced Rashomon Set is quantified using:
   • Multiplicity Metrics:
     • Ambiguity: The average pointwise disagreement between the reference model and the set.
     • Discrepancy: The worst-case disagreement rate within the set.
     • Rashomon Capacity: The dispersion of predictive probabilities (scores) across the set.
   • Fairness Metrics: Statistical parity, predictive equality, equal opportunity, and equalized odds, evaluated across demographic groups defined by sensitive attributes.

Key Contributions

• Methodological Shift: Introduces a method for generating dropout-based Rashomon Sets around extracted surrogates, reframing model stealing as a multiplicity problem rather than a single-model fidelity problem.
• Cross-Domain Evaluation: Systematically evaluates extraction-induced multiplicity across tabular data, medical imaging (ResNet-50), and NLP (BERT), demonstrating the phenomenon's generality.
• Diversity Characterization: Characterizes the diversity of extracted sets using ambiguity, discrepancy, Rashomon Capacity, and group fairness metrics.
• Empirical Evidence: Demonstrates that surrogates can exhibit high fidelity while displaying significant variances in sample-level predictions, attack-dependent multiplicity, and fairness values.

Experimental Results

The experiments were conducted on real-world datasets including Folktables (tabular), MedMNIST (medical imaging), and FST (NLP sentiment).

• High Fidelity vs. High Multiplicity: In all domains, extracted surrogates maintained high fidelity (often >0.90) across various dropout rates. However, the surrounding Rashomon Sets exhibited substantial multiplicity.
  • Tabular Data: Ambiguity ranged from 20% to 100% across datasets. For instance, in the low-dropout regime (high fidelity), ambiguity for ACSEmploy ranged from 0.28 to 0.65. Discrepancy values indicated that even among high-fidelity surrogates, competing models could assign conflicting predictions to up to 10% of individuals.
  • Medical Imaging: In the PneumoniaMNIST task, fidelity remained near 0.96, yet ambiguity rose to over 0.72. In OCTMNIST, high fidelity coexisted with high ambiguity (0.72–0.78) and substantial Rashomon Capacity, indicating significant variation in predictive scores.
  • NLP: The extracted BERT models showed consistent high fidelity (0.93–0.94). However, ambiguity grew rapidly even with mild fidelity changes, reaching up to 0.86 in higher dropout regimes.
• Fairness Implications: Predictive multiplicity translated into fairness variability. Surrogates with similar aggregate fidelity induced different group-level outcomes (e.g., shifts in equal opportunity and statistical parity), particularly in tabular tasks like ACSIncome.
• Attack Dependence: Different extraction pipelines (Knockoff Nets vs. Prediction-API) induced different local neighborhoods around the surrogate, leading to distinct multiplicity profiles even for the same target.

Significance and Claims

The paper argues that fidelity is an incomplete proxy for functional equivalence. The primary significance of this work lies in demonstrating that "successful" model theft, judged solely by fidelity, may fail to replicate downstream behavior, redistribute errors across populations, or alter individual recourse.

• Re-evaluating IP Loss: Claims of intellectual property loss based solely on fidelity may be incomplete. A stolen model may match the target's aggregate performance while disagreeing on specific instances or demographic groups.
• Evaluation Standards: The authors assert that evaluating model stealing attacks requires metrics beyond accuracy and fidelity, specifically including ambiguity, discrepancy, Rashomon Capacity, and fairness metrics.
• Deployment Risks: In high-stakes domains, the existence of a sizable Rashomon Set implies that an attacker may recover a model that matches the target's fidelity but produces substantially different outcomes for individuals or subpopulations.
• Conservative Estimate: The authors note that their findings likely represent a conservative estimate of extraction-induced multiplicity, as their dropout-based approximation captures a local neighborhood and assumes the surrogate architecture aligns with the victim. Relaxing these assumptions would likely increase the observed multiplicity.

In conclusion, the paper posits that model stealing should be viewed as an identification problem under multiplicity. The practical consequences of such attacks depend not just on average agreement, but on the structure of the extraction-induced Rashomon Set, necessitating more comprehensive evaluation frameworks for MLaaS security.