💻 computer science

Only Whats Necessary: Pareto Optimal Data Minimization for Privacy Preserving Video Anomaly Detection

This paper introduces "Only What's Necessary," a privacy-by-design framework for video anomaly detection that employs breadth- and depth-based data minimization to suppress personally identifiable information while preserving detection utility, using Pareto analysis to identify optimal trade-off points between privacy and performance in compliance with GDPR.

Original authors: Nazia Aslam, Abhisek Ray, Thomas B. Moeslund, Kamal Nasrollahi

Published 2026-03-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nazia Aslam, Abhisek Ray, Thomas B. Moeslund, Kamal Nasrollahi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you run a security camera system in a busy city square. Your goal is to spot trouble—like a fight breaking out or someone dropping a package—so you can call for help. This is Video Anomaly Detection.

However, there's a catch. To spot the trouble, your cameras record everything. They capture faces, license plates, tattoos, and who is standing next to whom. While this helps the computer see the "fight," it also violates people's privacy. In fact, under laws like the EU's GDPR, you are only allowed to collect data that is strictly necessary for the job. You can't hoard private details just in case they might be useful later.

This paper introduces a clever solution called "Only What's Necessary." It's a framework that acts like a smart filter, stripping away everything the security system doesn't need to do its job, while keeping the critical clues.

Here is how it works, broken down into simple concepts:

1. The Two Ways to "Trim the Fat"

The authors realized you can reduce privacy risks in two different ways, which they call Breadth and Depth. Think of it like editing a movie:

Breadth-Based Minimization (The "Skip-a-Frame" Method):
Imagine watching a movie where you skip every other frame. You still see the action (the fight), but you miss the subtle details of the actors' expressions.
- In the paper: They simply show the computer fewer frames per second. If a fight happens, the computer sees the movement, but it doesn't have enough continuous video to clearly identify a face.
Depth-Based Minimization (The "Blur-and-Mask" Method):
Imagine taking a photo of a crowd and putting a giant blur over everyone's face, or painting over their clothes so you only see their silhouette.
- In the paper: They use techniques like blurring (making faces fuzzy), masking (hiding people entirely), or downsampling (making the image grainy). This keeps the motion (someone running) but destroys the identity (who is running).

2. The Balancing Act: The "Goldilocks" Zone

The tricky part is finding the sweet spot.

If you show the computer raw video, it's great at spotting fights (high utility), but it's a privacy nightmare (high risk).
If you blur everything and skip frames, it's super safe for privacy, but the computer might miss the fight because the video is too grainy or choppy (low utility).

The authors didn't just guess which method was best. They treated this like a tug-of-war between two goals:

Keep the system smart (detect anomalies).
Keep the system private (hide identities).

They tested every combination (e.g., "Blur + Skip 5 frames," "Mask + Skip 10 frames") and plotted the results on a graph. This graph is called a Pareto Frontier.

3. Finding the "Sweet Spot"

Imagine a map where the top-right corner is the "Dream Land" (Perfect detection + Perfect privacy). Obviously, that doesn't exist.

Instead, they found the Edge of the Cliff. This is the line where you can't improve privacy without making the detection worse, and you can't improve detection without risking privacy.

On this line, they looked for the "Sweet Spot"—the specific setting that gives you the most privacy protection for the least amount of drop in performance.

The Verdict: What Works Best?

After running thousands of tests, they found the winner:
Temporal Sampling (skipping frames) + Blurring.

Think of it like this: You are watching a security feed through a slightly foggy window, and you only get a quick glance every few seconds.

Is it enough to see a fight? Yes. The motion is still there.
Is it enough to recognize a face? No. The blur and the gaps in time make it impossible to identify who is involved.

Why This Matters

This paper proves that you don't have to choose between safety and privacy. By using a "Privacy-by-Design" approach, we can build security systems that are legally compliant (they only take what they need) and technically effective (they still catch the bad guys).

It's like hiring a security guard who is trained to look for actions (running, fighting) rather than identities (names, faces). They do their job perfectly, but they don't need to know your name to do it.

1. Problem Statement

Video Anomaly Detection (VAD) systems are critical for safety but require large datasets containing Personally Identifiable Information (PII), such as faces and demographic attributes. This creates a conflict with data protection regulations like the EU GDPR, specifically the principle of Data Minimization (Article 5(1)(c)), which mandates that personal data be limited to what is strictly necessary for a specified purpose.

Current challenges include:

Lack of Operational Definition: There is no clear mathematical formulation for "necessary" data in machine learning pipelines.
Trade-off Gap: Existing approaches often prioritize performance (utility) over privacy or treat privacy as a secondary concern, failing to quantify the specific trade-off between detection accuracy and privacy leakage.
Indirect Leakage: Even if data is not directly exposed, inference attacks can extract sensitive attributes from model inputs or outputs.

The paper aims to formalize data minimization as a controllable design mechanism to find the "sweet spot" where privacy leakage is minimized with only acceptable degradation in anomaly detection performance.

2. Methodology

The authors propose a Privacy-by-Design framework called "Only What's Necessary." The core approach treats data minimization as a multi-objective optimization problem involving two complementary dimensions: Breadth and Depth.

A. The Minimization Framework

The framework transforms raw video input $x$ into a minimized representation $z$ using a configurable module $T_\theta$ :
$z = T_\theta(x)$

The minimization strategies are applied as follows:

Breadth-based Minimization (Temporal): Reduces the amount of data processed over time.
- Implementation: Temporal sampling (skipping frames).
- Mechanism: Increasing the stride $s$ (e.g., $s=5, s=10$ ) reduces the number of frames retained, removing redundant motion information.
Depth-based Minimization (Spatial/Appearance): Reduces the information content within each retained frame.
- Implementation:
  - Downsampling: Reducing spatial resolution ( $\times2, \times4$ ).
  - Masking: Suppressing human regions (detected via YOLO) to a binary representation.
  - Blurring: Applying Gaussian blur to human regions to obscure identity while retaining motion cues.
  - Background Removal: Subtracting static background to retain only moving foreground.

B. Evaluation Metrics

Each minimization configuration is evaluated against two downstream objectives:

Utility (Anomaly Detection): Measured by the Area Under the Curve (AUC) of a VAD model ( $f_\phi$ ). Higher AUC is better.
Privacy Leakage: Measured by the performance of a dedicated Privacy Inference Model ( $g_\psi$ $g_{ψ}$ ) attempting to predict sensitive attributes (e.g., gender, face, skin tone) from the minimized video.
- Metrics: cMAP (class-wise mean Average Precision) and F1-score.
- Goal: Lower values indicate stronger privacy preservation (less leakage).

C. Pareto-Optimal Selection

Since improving utility often increases privacy leakage (and vice versa), the authors use Pareto Analysis to identify non-dominated solutions.

Pareto Frontier: Identifies configurations where no other setting is strictly better in all three metrics (AUC, cMAP, F1).
Selection Strategies: To choose a single "sweet spot" from the Pareto frontier, three strategies are proposed:
- Distance-to-Ideal Point: Selects the configuration closest to the theoretical ideal (Max AUC, Min Leakage).
- Weighted Aggregation: Combines normalized metrics into a single score based on predefined weights ( $w_A, w_F, w_C$ ).
- Constraint-Based: Selects the highest utility model that satisfies specific privacy thresholds ( $\tau_F, \tau_C$ ).

3. Key Contributions

Formalization: Defines video data minimization as a privacy-utility trade-off problem with explicit mathematical objectives.
Dual-Dimension Framework: Introduces a systematic approach combining breadth-based (temporal) and depth-based (spatial/appearance) reduction strategies.
Selection Methodology: Proposes three distinct strategies (Distance, Weighted, Constraint) to identify Pareto-optimal operating points for real-world deployment.
Comprehensive Evaluation: Constructs a utility-privacy Pareto frontier across multiple public datasets (UCSD Ped1/2, CUHK Avenue, PAHMDB) to demonstrate the effectiveness of the framework.

4. Experimental Results

The framework was evaluated on standard VAD datasets (UCSD Ped1, UCSD Ped2, CUHK Avenue) and a privacy-annotated dataset (PAHMDB).

Raw Data: Achieves the highest AUC (e.g., 79.10% on UCSD Ped1) but the highest privacy leakage (cMAP 70.20%).
Breadth-Only (Temporal Sampling): Reduces AUC moderately but offers limited privacy gains.
Depth-Only:
- Masking: Provides the strongest privacy protection (lowest cMAP/F1) but causes significant drops in AUC (e.g., ~57% on Ped1).
- Blurring: Offers a superior balance, maintaining relatively high AUC while significantly reducing privacy leakage.
- Background Removal: Results in the largest utility degradation without providing the best privacy protection, highlighting the importance of scene context for VAD.
Combined Strategies:
- Temporal Sampling + Blurring ( $s=5$ + Blur): Identified as the optimal "sweet spot." It consistently appears on the Pareto frontier, offering a high AUC (e.g., ~71-72%) with substantially reduced privacy leakage compared to raw video.
- Temporal Sampling + Masking: Achieves the lowest privacy leakage but at the cost of significant utility loss.

Key Finding: The combination of Temporal Sampling (stride 5) and Blurring provides the most favorable trade-off, effectively satisfying GDPR's "necessary" requirement without rendering the anomaly detection system useless.

5. Significance

Regulatory Compliance: Provides a concrete, mathematically grounded method for organizations to comply with GDPR's data minimization principles in AI systems.
Privacy-Utility Balance: Moves beyond the binary choice of "privacy vs. performance" by quantifying the trade-off and identifying specific configurations where both can be optimized simultaneously.
Practical Deployment: The proposed "sweet spot" configurations (specifically blurring + temporal sampling) offer a ready-to-deploy solution for safety-critical environments (e.g., surveillance) where privacy is paramount but anomaly detection accuracy cannot be compromised.
Generalizability: The framework is model-agnostic and can be applied to various VAD architectures and privacy-sensitive attributes.