🤖 machine learning

Scalable Differentially Private Data Compression via Diffusion and Stochastic Codes

This paper introduces DP-DiPP, a scalable differentially private image compression framework that combines stochastic codes with diffusion models to achieve significantly higher compression rates (10–30 times better) while maintaining strong privacy guarantees and utility for high-dimensional data.

Original authors: Gergely Flamich, Oykü Sıla Güner, Yanxiao Liu, Deniz Gündüz

Published 2026-07-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gergely Flamich, Oykü Sıla Güner, Yanxiao Liu, Deniz Gündüz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very valuable, high-resolution photo of your family. You want to share it with a researcher so they can study it, but you are terrified that if they see the original, they might figure out who you are.

To protect your identity, you decide to add a layer of "digital fog" (noise) to the photo. This is called Differential Privacy. It's like blurring the faces just enough that no one can tell who is who, but the overall shape of the photo remains useful for study.

The Problem: The "Fog" is Too Heavy
Here is the catch: When you add this privacy fog to a high-resolution image, the file becomes huge and messy.

The Old Way (Privatize then Compress): Imagine you take your photo, cover it in thick, random static (the privacy noise), and then try to zip it up to save space. Because the static is completely random, it's impossible to compress it efficiently. It's like trying to fold a piece of paper covered in random glitter; it just won't fit in a small envelope. You end up with a massive file that is still hard to send.
The Dilemma: If you try to compress it before adding the noise, you lose the privacy. If you add the noise first, the file is too big.

The Solution: DP-DiPP (The "Smart Blender")
The authors of this paper created a new tool called DP-DiPP. Instead of doing the privacy protection and the compression as two separate steps, they blended them into one smooth process.

Think of it like this:

The Diffusion Model (The Artist): Imagine an artist who can take a blurry, noisy sketch and slowly turn it into a clear picture, step-by-step. This is a "diffusion model." Usually, this artist works with "Gaussian noise" (a specific type of math fog).
The Switch (The Privacy Guard): The researchers realized that the artist's "Gaussian fog" wasn't strong enough to guarantee strict privacy. So, they swapped the artist's fog for a different kind called Laplace noise. This new fog is mathematically guaranteed to protect your identity better, like switching from a light mist to a thick, impenetrable wall of fog.
The Stochastic Code (The Smart Shrinker): This is the magic ingredient. Instead of just saving the noisy picture, the system uses a "stochastic code."
- Analogy: Imagine you and a friend both have the same secret deck of cards (shared randomness). You want to tell your friend a specific card you picked, but you can't just say the name. Instead, you use a special rule to pick a card from your shared deck that looks exactly like the one you picked, but you only send a tiny note saying "Card #42." Your friend, using their identical deck and the same rule, pulls out Card #42, which happens to be the exact card you wanted to show them.
- This allows them to send the "privacy-protected" image using a tiny bit of data (a short note) rather than the whole messy file.

How It Works Together
DP-DiPP works like a conveyor belt:

It takes your image and starts "denoising" it (making it clearer) step-by-step.
At every single step, instead of just saving the image, it uses the "Smart Shrinker" (Stochastic Code) to encode that step using the "Privacy Guard" (Laplace noise).
Because the shrinking and the privacy protection happen at the exact same time, the system doesn't waste space on the random noise. It only sends the essential information needed to reconstruct the image.

The Results
The team tested this on a dataset of 10,000 small images (CIFAR-10). They compared their new method against the old "add noise then zip" method.

The Win: DP-DiPP was 10 to 30 times more efficient. It could send the same amount of useful, privacy-protected information using a fraction of the data.
The Trade-off: It was slightly less efficient than a version that didn't care about privacy at all, but it was vastly better than the old way of doing things.
The Outcome: A computer could still learn to recognize the images (like telling a cat from a dog) just as well with the DP-DiPP files as it could with the old, massive files.

In a Nutshell
The paper presents a way to shrink high-resolution images that have been scrambled for privacy. By using a clever mathematical trick (Stochastic Codes) combined with a specific type of privacy noise (Laplace) inside a step-by-step image reconstruction process (Diffusion), they managed to make the files 10–30 times smaller without losing the ability to use the data or the guarantee of privacy. It turns a "communication-privacy-accuracy trilemma" (where you usually have to sacrifice one for the others) into a solution where you get all three.

Technical Summary: Scalable Differentially Private Data Compression via Diffusion and Stochastic Codes

1. Problem Statement

The rapid growth of personal data collection necessitates robust privacy frameworks, with Differential Privacy (DP) serving as the standard for formal guarantees. While DP has achieved practical success in tabular data and low-dimensional settings, scaling it to high-dimensional data (e.g., images, audio, video) remains a significant challenge.

Two primary obstacles hinder high-dimensional DP:

Storage and Communication Costs: Releasing uncompressed privatized data incurs substantial storage and bandwidth overhead.
Utility Degradation: The noise required to satisfy privacy guarantees often severely degrades the utility of high-dimensional data.

Existing approaches typically follow a "privatize-then-compress" pipeline: adding noise to data and then applying lossless compression (e.g., PNG). This approach is inefficient because privacy mechanisms often introduce random structures that are difficult to compress losslessly, leading to a "communication-privacy-accuracy trilemma." Furthermore, no effective scheme currently exists to jointly compress high-resolution data while maintaining strict privacy guarantees.

2. Methodology: DP-DiPP

The authors propose DP-DiPP (Differentially Private DiffC with Poisson Processes), a pipeline that jointly privatizes and compresses data by combining Stochastic Codes with Diffusion Models.

2.1. Theoretical Foundation: Approximate Poisson Private Representation (PPR)

The core theoretical contribution is an extension of the Poisson Private Representation (PPR) introduced by Liu et al. (2024).

Challenge: Standard PPR allows for exact simulation of any $\epsilon$ -LDP mechanism with optimal codelength but suffers from super-exponential runtime scaling with the information content (specifically, the density ratio $\|r(\cdot|x)\|_\infty$ ). For diffusion models, this ratio can be unbounded, making exact PPR infeasible.
Solution: The authors introduce a step-limited PPR. This variant imposes a budget $n$ on the number of samples examined during encoding. If the budget is exceeded, the algorithm terminates and returns the best index found.
Guarantees: The step-limited PPR provides:
- Approximation: The output distribution is close to the target in Total Variation distance.
- Privacy: It retains a formal LDP guarantee. Specifically, if encoding a pure $\epsilon$ -LDP mechanism, the output is $2\alpha\epsilon$ -LDP, where $\alpha > 1$ is a tunable parameter trading off privacy and codelength.
- Efficiency: The runtime is fixed and finite, allowing for parallelization.

2.2. Integration with Diffusion Models (DiffC)

The method integrates the step-limited PPR with DiffC, a variable-rate, diffusion-based lossy compression framework.

The Privacy Mechanism: In standard DiffC, the denoising process is modeled using Gaussian distributions. However, Gaussian mechanisms do not yield pure $\epsilon$ -LDP certificates (they typically satisfy $(\epsilon, \delta)$ -DP).
Moment-Matched Laplace Approximation: To achieve pure LDP, the authors replace the Gaussian targets and proposals in the denoising process with moment-matched Laplace distributions.
- This substitution ensures the density ratio between the target and proposal distributions is bounded, a prerequisite for the PPR code to function without unbounded runtime.
- Crucially, the authors empirically find that this approximation does not significantly degrade the denoiser's performance or downstream utility.
Compression Pipeline:
1. The diffusion model generates a conditional posterior (modeled as a Laplace distribution) at each denoising step.
2. The step-limited PPR encodes samples from this distribution.
3. The process is repeated across a schedule of timesteps, decomposing the compression problem into manageable steps.

3. Key Contributions

Extension of PPR: The paper extends the Poisson Private Representation to an approximate, step-limited version. It proves that this variant maintains a $2\alpha\epsilon$ -LDP certificate and achieves near-optimal codelengths while ensuring finite, parallelizable runtime.
Privacy-Preserving Diffusion Compression: The authors modify the DiffC framework by replacing Gaussian denoising targets with moment-matched Laplace distributions. This transforms the reverse diffusion process into a pure $\epsilon$ -LDP mechanism, enabling the application of PPR.
DP-DiPP System: The integration of the approximate PPR code with the modified DiffC process yields a scalable, differentially private image compressor.
Empirical Validation: Experiments on the CIFAR-10 dataset demonstrate that DP-DiPP significantly outperforms the "privatize-then-compress" baseline.

4. Experimental Results

The authors evaluated DP-DiPP on privatized image classification using CIFAR-10, comparing it against:

A naive "privatize-then-compress" baseline (adding Laplace noise and compressing with PNG).
The original Gaussian DiffC (which lacks a pure LDP certificate).

Key Findings:

Compression Efficiency: DP-DiPP achieves a 10–30 times improvement in compression rate (bits per pixel) compared to the baseline while maintaining comparable classification accuracy and privacy guarantees.
Privacy-Utility Tradeoff: The method consistently reduces bitrate by a factor of 10–30 while retaining comparable pure LDP guarantees ( $\epsilon$ ) and utility (measured by classification accuracy).
Approximation Impact: Replacing Gaussian channels with Laplace channels and using a strict privacy parameter ( $\alpha=2$ ) in the step-limited PPR introduces only a negligible penalty in utility, with a roughly 2x increase in bitrate compared to the unprivatized Gaussian DiffC (which serves as an upper bound on performance but lacks formal privacy).

5. Significance and Claims

The paper claims that DP-DiPP resolves the "communication-privacy-accuracy trilemma" for high-dimensional data by moving away from sequential privatization and compression. Instead, it offers a joint compression and privatization framework.

The authors emphasize that their work provides a principled way to compress high-resolution data with strong formal privacy guarantees (pure LDP) without sacrificing downstream utility. By leveraging stochastic codes and diffusion models, they demonstrate that it is possible to achieve significant storage and communication savings in privacy-preserving machine learning tasks, a feat previously considered elusive for high-dimensional data.

The work is modest in its scope, focusing specifically on pure $\epsilon$ -LDP mechanisms and image data (CIFAR-10). The authors note that extending these techniques to $(\epsilon, \delta)$ -LDP mechanisms remains a direction for future theoretical work.