📊 statistics

Membership Inference Attacks for Unseen Classes

This paper introduces the "unseen class" setting for membership inference attacks, demonstrating that existing state-of-the-art methods fail when auditors lack access to representative harmful content, while showing that quantile regression attacks significantly outperform shadow model-based approaches in this constrained scenario.

Original authors: Pratiksha Thaker, Neil Kale, Zhiwei Steven Wu, Virginia Smith

Published 2026-07-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pratiksha Thaker, Neil Kale, Zhiwei Steven Wu, Virginia Smith

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery about a secret recipe. In the world of artificial intelligence, this "recipe" is the data used to train a computer model. Sometimes, we need to know if a specific, dangerous ingredient—like a harmful image or a private medical record—was secretly mixed into that recipe. This is called a "Membership Inference Attack." Think of it like a food critic trying to guess if a specific, banned spice was in a soup they are tasting, just by looking at the soup's flavor.

Usually, to solve this mystery, the detective needs to taste a bunch of other soups that don't have the banned spice, so they can learn what the "normal" flavor looks like. This helps them spot the weird, extra flavor of the banned spice. But what if the banned spice is so illegal or dangerous that the detective is strictly forbidden from ever touching it, even for practice? They can't buy a sample to study. They have to guess if it's in the target soup without ever having seen the spice before. This is the tricky, real-world problem this paper tackles: how do you catch a "secret ingredient" when you aren't allowed to hold a sample of it?

The researchers in this paper decided to investigate exactly this impossible-sounding scenario, which they call the "unseen class" setting. In the world of AI safety, this is like trying to audit a model to see if it was trained on Child Sexual Abuse Material (CSAM). The people doing the auditing (the detectives) often cannot legally or ethically access examples of CSAM to train their detection tools. They are stuck with a blank spot in their knowledge.

The paper first tests the current "gold standard" detective tools, known as Shadow Model attacks. Imagine these tools as a team of practice chefs who try to mimic the target chef. To do this, they cook many practice dishes using ingredients they are allowed to touch. The problem? If the banned ingredient is missing from their pantry, they never learn what it tastes like. When they try to guess if the target soup has it, they get completely confused. The paper shows that in this "unseen" situation, these fancy shadow models fail miserably. They perform no better than a simple guess, often falling to near-zero success rates. It's like a chef trying to identify a spice they've never seen; they just end up guessing randomly.

However, the paper discovers a surprisingly simple and effective alternative: Quantile Regression attacks. Instead of trying to mimic the whole cooking process with practice chefs, this method acts more like a smart thermometer. It learns to predict a "safety threshold" for the flavor of the soup based on patterns it can see. Even though it has never tasted the banned spice, it learns to recognize the subtle, general "vibe" or features that appear when a secret ingredient is present, regardless of what that ingredient actually is.

The results are striking. In tests using image data (like the CIFAR-100 dataset), the new quantile regression method was able to find the hidden "ingredient" with up to 11 times more success than the shadow models when the target class was completely unseen. On other datasets, like text or tabular data, it still outperformed the old methods significantly, sometimes doubling or sextupling the success rate. The authors also ran simulations on massive datasets like ImageNet, showing that even if 990 out of 1,000 classes were missing from the training data, the quantile method could still spot the missing ones better than random guessing.

The paper doesn't just show numbers; it offers a theoretical explanation for why this works. It suggests that because the quantile method learns general features that connect to the "score" of a sample, it can transfer its knowledge from the classes it knows to the classes it doesn't. It's like learning that "spicy" is a general feeling; even if you've never tasted a specific new pepper, you can still recognize the heat.

In short, the paper proves that the most popular tools for AI safety audits break down when you can't access the specific data you're looking for. But it also offers a lifeline: a simpler, more robust method that works even when the "banned ingredient" is completely invisible to the auditor. This is a crucial finding for anyone trying to make AI safer in the real world, where legal and ethical rules often prevent us from seeing the very things we need to detect.

Technical Summary: Membership Inference Attacks for Unseen Classes

Problem Definition: The Unseen Class Setting
Membership Inference Attacks (MIAs) are critical tools for data auditing, allowing practitioners to determine if specific data points were used to train a black-box model. Standard state-of-the-art (SoTA) MIAs, particularly those based on shadow models, rely on a fundamental assumption: the adversary possesses an auxiliary dataset drawn from the same distribution as the target model's training data, including all target classes.

This paper identifies a critical failure mode in real-world auditing scenarios where this assumption is violated. In domains such as child safety (e.g., detecting Child Sexual Abuse Material or CSAM), medical record auditing, or proprietary data analysis, auditors often face legal, ethical, or logistical constraints that prevent them from accessing representative samples of the target class. The authors formalize this as the "unseen class" setting: a data access model where the auditor must perform membership inference on a target class for which they possess zero training samples to train the attack model.

Methodology and Analysis
The paper evaluates three primary classes of MIA methods under the unseen class constraint:

Marginal Baseline (LOSS): A simple global threshold attack based on loss or logit scores.
Shadow Model-Based Attacks (e.g., LiRA, RMIA): The current SoTA, which trains multiple "shadow" models mimicking the target architecture to learn the distribution of scores for members vs. non-members.
Quantile Regression Attacks: An approach that directly learns a function mapping input examples to score thresholds (specifically, predicting the $\alpha$ -quantile of the non-member score distribution) using pinball loss.

The authors demonstrate that shadow model attacks fail catastrophically in the unseen class setting. Because shadow models are trained on a subset of classes excluding the target, they assign zero probability to the unseen class labels. Consequently, the confidence scores for the true label on unseen samples are zero, causing the attack to fail to distinguish members from non-members effectively, often performing no better than random guessing or the marginal baseline.

In contrast, the paper proposes that quantile regression attacks offer a robust alternative. Unlike shadow models, which rely on comparing target scores against reference distributions of specific classes, quantile regression learns features that correlate with the score distribution across the entire background distribution. This allows the attack to generalize to unseen classes by identifying membership signals that are invariant to the specific class label.

Key Contributions and Results
The paper provides both empirical and theoretical evidence supporting the efficacy of quantile regression in this setting:

Empirical Performance:
- Image Data (CIFAR-100/CINIC-10): In the 1% False Positive Rate (FPR) regime, quantile regression attacks achieve up to 11× higher True Positive Rate (TPR) compared to shadow model-based approaches (LiRA/RMIA) on unseen classes. On ImageNet, even with 990 out of 1000 classes missing from the training set, quantile regression maintains a TPR of ~3.8% at 1% FPR, significantly outperforming the marginal baseline.
- Tabular and Text Data: On the Texas tabular dataset and 20 Newsgroups text classification, quantile regression achieves up to 2× and 6× higher TPR, respectively, compared to shadow models under class dropout.
- Data Scarcity: The method remains effective even when the auditor has access to only a small fraction of samples per class (e.g., 10 samples per class on ImageNet).
Theoretical Model:
The authors introduce a "transferability" theorem for quantile regressors. They prove that if the feature representation of the training distribution (seen classes) and the test distribution (full distribution including unseen classes) share a linear density ratio relationship, the false-positive rate guarantee of the quantile regressor trained on seen classes transfers to the full distribution. Empirical estimates using Gaussian mixture models on embeddings suggest this condition holds well for large, diverse datasets like ImageNet, explaining the method's robustness.

Significance and Impact
The paper argues that the "unseen class" setting represents a significant, previously unaddressed gap in the MIA literature. It highlights that the gold-standard shadow model approach is not universally applicable and fails precisely in the high-stakes, resource-constrained scenarios where auditing is most critical (e.g., CSAM detection).

The work serves as a cautionary tale for practitioners attempting to apply existing MIA tools directly to real-world safety audits without verifying data access constraints. By identifying quantile regression as a computationally efficient and generalizable alternative, the paper provides a viable path forward for auditing models trained on sensitive or restricted data, enabling law enforcement and model providers to verify training data provenance even when direct access to harmful content is legally prohibited.

Technical Summary: Membership Inference Attacks for Unseen Classes

More like this