Technical Summary: SurrogateSHAP

Problem Statement

As Text-to-Image (T2I) diffusion models become integral to creative workflows, there is an urgent need for a principled framework to value data contributors for fair compensation and sustainable data marketplaces. While the Shapley value offers a theoretically sound method for attribution by quantifying marginal contributions across all possible subsets of contributors, its application to modern T2I models faces a dual computational bottleneck:

1. Training Cost: Exact Shapley computation requires retraining the model for every subset of contributors, which is prohibitively expensive for large-scale diffusion models.
2. Sampling Complexity: The utility functions in dataset valuation are often highly non-linear and involve complex interactions, requiring a combinatorial number of subset queries to obtain stable, low-variance estimates.

Existing alternatives, such as aggregating individual data attributions (e.g., D-TRAK, DAS), often fail to capture group-level non-linearities and downstream properties like aesthetics. Other Shapley-based approaches rely on sparsified fine-tuning or exact retraining, which remain computationally intensive.

Methodology: SurrogateSHAP

The authors propose SurrogateSHAP, a training-free framework that approximates the expensive "training game" through inference from a pretrained model and employs a gradient-boosted tree (GBT) surrogate to improve estimation efficiency. The method consists of two core components:

1. Training-Free Proxy Game

Instead of retraining a model $\theta^*_S$ for a specific subset of contributors $S$, SurrogateSHAP reinterprets the target model $\theta$ (trained on the full dataset) as a mixture of conditional generators.

• Mechanism: The output distribution for a coalition $S$ is approximated as a mixture of the full model's conditional distributions $p_\theta(x|\psi(y))$, weighted by the empirical frequency of prompts in the coalition.
• Theoretical Justification: The authors provide a theoretical bound (Proposition 4.2) showing that if the class-conditional distributions of the coalition-trained model and the full model are stable in a utility-aligned feature space (Assumption 4.1), the proxy utility $\hat{v}_\theta(S)$ closely tracks the true training utility $v(S)$. This eliminates the need for subset-specific training, reducing subset evaluation to a mixture sampling problem.

2. Tree-Based Surrogate for Shapley Estimation

To address the high variance and sampling inefficiency of estimating Shapley values for complex, non-linear utility functions, the framework learns a surrogate model.

• Procedure: The method samples $M$ coalitions, evaluates the proxy utility for each, and trains a Gradient-Boosted Tree (GBT) surrogate $\hat{f}_\omega$ to approximate the utility function over the Boolean hypercube of contributors.
• Analytical Computation: Shapley values are derived analytically from the trained tree ensemble using TreeSHAP. This approach captures hierarchical interaction priors without imposing explicit sparsity constraints, offering higher sample efficiency than Monte Carlo-based estimators like KernelSHAP.

Key Contributions

1. Training-Free Proxy: A novel formulation that bypasses retraining by leveraging the conditional generation capabilities of a single pretrained model, validated by theoretical error bounds and empirical fidelity.
2. Efficient Estimation: The integration of GBT surrogates and TreeSHAP to analytically compute Shapley values, significantly reducing the number of queries required for stable estimation compared to existing Shapley estimators.
3. Comprehensive Evaluation: The framework is evaluated across three diverse attribution tasks:
   • Image Quality: DDPM-CFG on CIFAR-20 (metrics: FID, Inception Score).
   • Aesthetics: Stable Diffusion on Post-Impressionist artworks (metric: Aesthetic Score).
   • Product Diversity: FLUX.1 on Fashion-Product data (metrics: LPIPS, Diversity).
4. Clinical Auditing Case Study: Application of the framework to audit a dermatology model trained on multi-site clinical data (ISIC dataset), successfully localizing spurious correlations between patient sex and melanoma diagnosis to specific hospital data sources.

Experimental Results

• Proxy Fidelity: SurrogateSHAP's proxy utility demonstrates high alignment with the true training utility, achieving lower representation drift (RMS LPIPS) and tighter utility error (NRMSE) compared to sparsified fine-tuning (sFT) baselines.
• Computational Efficiency: The method is significantly faster than retraining or fine-tuning baselines (e.g., 23.4$\times$ faster on CIFAR-20, 12.8$\times$ on ArtBench).
• Attribution Accuracy:
  • Linear Datamodel Score (LDS): SurrogateSHAP consistently outperforms prior baselines (including TRAK, Influence Functions, and sFT-based Shapley) in predicting held-out model behaviors across all datasets and metrics. For instance, on CIFAR-20 (FID), it achieved an LDS of 51.21% compared to sFT's negative performance.
  • Counterfactual Evaluation: Removing low-ranked contributors identified by SurrogateSHAP led to performance gains or minimal degradation, while removing top-ranked contributors caused the most significant quality drops, confirming the method's ability to identify influential data.
• Sample Efficiency: On synthetic games with non-linear interactions, SurrogateSHAP achieved significantly lower $\ell_2$ error relative to the oracle Shapley value with the same query budget compared to KernelSHAP and ProxySPEX.

Significance and Claims

The paper claims that SurrogateSHAP provides a scalable and principled path toward contributor valuation for large-scale generative models. By decoupling attribution from the computational cost of retraining, it enables:

• Fair Compensation: Accurate quantification of group-level contributions for data marketplaces.
• Model Auditing: The ability to identify and localize spurious correlations and biased data sources (as demonstrated in the clinical case study) without requiring access to retraining pipelines.
• Democratization of Auditing: The framework allows researchers with limited compute resources to perform transparency audits on large-scale models, facilitating fair credit assignment and principled dataset curation.

The authors emphasize that while the method enhances transparency, it should be used as a diagnostic tool within a broader governance pipeline, noting that metric-specific optimization could inadvertently suppress minority data if not carefully managed.