💻 computer science

Privacy Auditing with Zero (0) Training Run

This paper introduces Zero-Run, a post-hoc privacy auditing framework that enables empirical evaluation of differential privacy in large-scale models without retraining by using causal inference to correct for distribution shifts between known member and non-member datasets.

Original authors: Tudor Cebere, Mathieu Even, Linus Bleistein, Aurélien Bellet

Published 2026-05-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tudor Cebere, Mathieu Even, Linus Bleistein, Aurélien Bellet

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Black Box" Mystery

Imagine a massive, super-smart AI model (like a giant brain) that has been trained on millions of private documents, photos, or medical records. We want to know: Did this AI accidentally memorize specific private secrets?

In the past, to check this, security experts had to play a game of "re-do the experiment." They would say, "Let's train the AI again, but this time without that one secret document. Then let's train it with the document. If the AI acts differently, we know it memorized the secret."

The Problem: This is impossible for modern AI. Training these models takes thousands of supercomputers running for weeks. You can't just "re-run" the training every time you want to check for privacy leaks. It's like asking a chef to cook a 10-course meal from scratch every time you want to taste the salt.

The New Solution: "Zero-Run" Auditing

This paper introduces a new way to check for privacy leaks called Zero-Run Auditing.

Instead of re-cooking the meal, the auditor simply tastes the finished dish (the released AI model) and compares it to a list of ingredients they know were used (the training data) and a list of ingredients they know were not used (data that was never seen).

The Catch: In the old method, the "ingredients" (data) were randomized, like shuffling a deck of cards. In this new method, the "known used" ingredients and "known unused" ingredients might look very different from each other.

Example: Imagine the AI was trained on photos of poodles (members). The auditor brings in photos of golden retrievers (non-members) to test it.
If the AI guesses "Poodle!" for the poodles and "Dog!" for the retrievers, is that because the AI memorized the poodles? Or is it just because poodles and retrievers look different?
The paper calls this Distribution Shift. It's a "confounding factor"—a distraction that makes the AI look like it's leaking secrets when it might just be reacting to the obvious differences in the data.

The Analogy: The Detective and the Suspects

Think of the AI as a Detective and the data points as Suspects.

The Old Way (Interventional): The detective is given a lineup where suspects are randomly assigned to be "Guilty" or "Innocent." If the detective picks the "Guilty" ones correctly, we know they have inside information (leakage).
The New Way (Observational/Zero-Run): The detective is given a lineup where the "Guilty" suspects are all tall and the "Innocent" suspects are all short.
- If the detective picks the tall people as "Guilty," did they use inside info? Or did they just guess based on height?
- The paper says: We need to correct for the height.

How the Paper Fixes It: The "Propensity Score"

The authors use a concept from statistics called a Propensity Score. In our detective analogy, this is a "Height Calculator."

Before the detective makes their guess, the auditor calculates: "Based purely on how tall this person is, what is the probability they are actually guilty?"

If a suspect is very tall, the calculator says, "There's a 90% chance they are guilty just because they are tall."
If the detective still guesses "Guilty" for this person, the auditor says, "Okay, you got that right, but 90% of that credit goes to the height, not your detective skills."
The auditor then discounts that guess. They only count the part of the guess that couldn't be explained by height alone.

The paper proposes two ways to do this math:

The Global Correction: A conservative approach that assumes the worst-case scenario for the "height" difference. It's safe but might miss some leaks.
The Pointwise Correction: A sharper approach that looks at each suspect individually. It calculates exactly how much "height" (distribution shift) influenced that specific guess and removes it. This gives a much more accurate picture of the actual privacy leak.

The Results: What They Found

The authors tested this on:

Fake Data: They created a scenario where they knew exactly how much the AI was leaking. They showed that without their correction, the audit would falsely accuse the AI of leaking way more than it actually did. With their correction, the audit was accurate.
Real Data (WildCam): They tested it on a real-world dataset of wildlife images. In the wild, "members" (animals seen during training) and "non-members" (animals seen later) naturally look different due to seasons or locations.
- Without correction, the audit failed or gave useless results because the "seasonal shift" looked like a privacy leak.
- With their Zero-Run correction, they successfully measured the actual privacy leakage, proving that even with messy, real-world data, you can audit an AI without retraining it.

The Bottom Line

This paper provides a toolkit for auditors to check if a massive AI model has leaked private secrets without needing to retrain the model.

It solves the problem of "unfair comparisons" by mathematically stripping away the obvious differences between the data the AI saw and the data it didn't see. This allows regulators and researchers to hold AI companies accountable for privacy, even when those companies refuse to let auditors touch their training pipelines.

In short: You don't need to rebuild the house to check if the windows are locked; you just need a better way to look at the light coming through them, accounting for the time of day and the weather.

Technical Summary: Privacy Auditing with Zero (0) Training Run

1. Problem Statement

Privacy auditing is essential for empirically establishing lower bounds on the differential privacy (DP) parameters of learning algorithms, complementing theoretical guarantees. However, existing auditing methods (e.g., Multi-Run and One-Run) rely on interventional access to the training pipeline. They require either retraining models multiple times with randomized data insertions or executing the algorithm once with randomized membership indicators.

This interventional requirement renders auditing infeasible for large-scale, deployed foundation models (e.g., Large Language Models) due to:

Computational Prohibitions: Retraining or re-executing training pipelines consumes thousands of GPU days.
Logistical Barriers: Access to internal data pipelines and the ability to randomize data inclusion are often unavailable in black-box deployments.
The Accountability Gap: The systems posing the greatest privacy risks are the most insulated from independent evaluation, forcing reliance on trust rather than verification.

Furthermore, a naive attempt to audit without intervention (using static datasets of known members and non-members) fails because it introduces confounding distribution shifts. If member and non-member datasets follow different underlying distributions ( $P_1 \neq P_0$ ), a membership inference attack (MIA) may succeed by exploiting these distributional differences rather than algorithmic leakage, leading to false positives and invalid privacy bounds.

2. Methodology: Zero-Run Auditing

The authors propose Zero-Run Auditing, a post-hoc framework that treats privacy auditing as an observational inference problem, analogous to causal inference. The framework operates without modifying the algorithm or the data pipeline, relying solely on the model's output and two fixed datasets:

$D_1$ (Members): A subset of the training data (or known pre-training data).
$D_0$ (Non-members): A dataset known to be excluded from training (e.g., post-training data or synthetic samples).

The framework proceeds in four steps:

Observation: The auditor observes the model output $\theta = A(D_{train})$ and the observational data $\{X_i\} = D_1 \cup D_0$ .
Membership Inference: A standard MIA generates raw guesses $T_i \in \{-1, 0, 1\}$ for each $X_i$ .
Propensity Estimation: The auditor estimates a propensity score $\hat{\pi}(X_i) \approx P(S_i=1 | X_i)$ , representing the probability that a data point is a member based solely on its features. This isolates the structural selection likelihood from the algorithmic signal.
Correction: The framework applies corrections to disentangle distribution shift from algorithmic leakage using two complementary approaches.

Correction Approaches

The paper formalizes two methods to correct for the confounding effect of distribution shift:

Global Correction (Adaptive Composition):
- Models the observed leakage as the adaptive composition of two mechanisms: a distribution-shift mechanism ( $M_{DS}$ ) and the audited algorithm ( $A$ ).
- Treats the total leakage as $f_{Tot} = f \otimes f_{DS}$ , where $\otimes$ is the tensor product of trade-off functions.
- Result: Produces conservative global bounds. It assumes worst-case distribution shift leakage across all data points, which can lead to a loss of statistical power if the shift is heterogeneous.
Conditional Correction (Pointwise Auditing):
- Conditions on the observed examples $X_i$ . It recognizes that the contribution of distribution shift to inference is local, defined by the log-odds $\epsilon_{DS}(x_i) = |\log \pi(x_i) - \log(1-\pi(x_i))|$ .
- Tampering Mechanism: To prevent overconfidence from distribution shift, the framework "tampers" the MIA guesses. For an example with a highly imbalanced propensity score (e.g., $\pi(x) \approx 1$ ), a correct guess is less indicative of algorithmic leakage. The framework retains such guesses with a lower probability $b_i$ , effectively randomizing abstention.
- Result: Yields sharper, instance-dependent bounds that are robust to distribution shifts without the severe penalties of the global worst-case approach.

Uncertainty Quantification

Since propensity scores must be estimated from data, the framework incorporates bootstrapping to quantify uncertainty. By resampling the propensity estimation dataset, the method produces empirical lower bounds that account for estimation error, ensuring the final audit remains valid with high probability.

3. Key Contributions

Formalization of Zero-Run Auditing: The paper defines a post-hoc auditing regime where membership is not randomized. It demonstrates that naive audits on such data overestimate leakage and explicitly connects this setting to observational causal inference, requiring an "overlap" assumption (positivity) between member and non-member distributions.
Leakage Correction Frameworks:
- A Composition Viewpoint that deconvolves distribution shift from algorithmic leakage via adaptive composition.
- A Conditional Viewpoint that adjusts pointwise membership guesses based on propensity scores, yielding robust, shift-dependent corrections.
Uncertainty Quantification: Integration of bootstrapping to provide statistically valid lower bounds that account for errors in propensity score estimation.
Scalability: Demonstration that privacy audits can be performed on large, deployed models where retraining is computationally or logistically impossible.

4. Experimental Results

The authors evaluate Zero-Run auditing on synthetic data and real-world benchmarks (iWildCam):

Synthetic Data: Under increasing distribution shift magnitudes, standard One-Run audits (ignoring shift) produce invalid bounds. Global corrections rapidly lose statistical power due to worst-case penalties. In contrast, the propensity-aware conditional correction maintains informative, relatively flat empirical bounds across shift levels.
Real-World Data (iWildCam): In the presence of severe natural distribution shifts (geographic/environmental), global and propensity-unaware methods yield non-positive (useless) bounds. The propensity-aware conditional correction successfully extracts a valid privacy lower bound ( $\hat{\mu} = 0.645$ ) for out-of-distribution (OOD) data, closely matching the bound achieved by an ideal IID split ( $\hat{\mu} = 0.652$ ).
Cost: The Zero-Run approach incurs a penalty of approximately $\approx 1$ unit in the $\mu$ -GDP metric compared to ideal One-Run auditing, attributed to the cost of propensity score estimation and uncertainty quantification.

5. Significance and Claims

The paper claims that post-hoc privacy auditing is possible but must be treated as an observational inference problem. The key significance lies in:

Bridging the Accountability Gap: Providing regulators and researchers with tools to verify privacy guarantees for foundation models without requiring access to training infrastructure.
Causal Analogy: Successfully applying causal inference principles (propensity scores, confounding adjustment) to the domain of privacy auditing.
Practicality: Enabling privacy evaluation in scenarios where theoretical guarantees are opaque and experimental re-execution is infeasible.

The authors remain modest, acknowledging limitations:

Propensity Estimation: The framework relies on accurate propensity score estimation, which is challenging in high-dimensional, finite-sample settings.
Inherited Limitations: Zero-Run auditing inherits the finite-sample inefficiencies of One-Run methods and is dependent on the baseline strength of the underlying MIA.
Falsifiability: While the audit is valid, the propensity score bounds themselves cannot be formally certified; they are computationally falsifiable through adversarial testing but rely on empirical validation.

In conclusion, the paper establishes that distribution shift is a distinct source of membership information that must be quantified and corrected. By using propensity scores, Zero-Run auditing converts these confounding factors into valid empirical lower bounds on algorithmic privacy leakage.