💻 computer science

PrivHAR-Bench: A Graduated Privacy Benchmark Dataset for Video-Based Action Recognition

This paper introduces PrivHAR-Bench, a graduated multi-tier benchmark dataset and evaluation toolkit designed to standardize the assessment of the privacy-utility trade-off in video-based Human Activity Recognition by applying a spectrum of visual privacy transformations to a diverse set of action videos.

Original authors: Samar Ansari

Published 2026-04-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Samar Ansari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize what people are doing in a video—like telling the difference between someone "jumping jacks" and someone "brushing their teeth." This is called Human Activity Recognition (HAR).

The problem is, if you show the robot the raw video, it can also see the person's face, their clothes, and their surroundings. This is a huge privacy risk. You don't want the robot to know who the person is, only what they are doing.

So, researchers have tried to "blur" or "scramble" the video to hide the person's identity. But here's the catch: How much blurring is enough?

If you blur it a little, the robot might still recognize the person (bad for privacy).
If you blur it too much, the robot can't tell what action is happening anymore (bad for utility).

Until now, there was no standard way to measure this balance. Most studies just compared "Clear Video" vs. "One Specific Blur." It's like trying to learn how to drive by only testing your car on a perfectly smooth road and a single pothole. You don't know how it handles the whole range of bumps in between.

Enter PrivHAR-Bench: The "Privacy Gym"

The authors of this paper, led by Samar Ansari, have built a new tool called PrivHAR-Bench. Think of it as a gym with 9 different levels of difficulty designed specifically to test how well a robot can recognize actions while keeping people anonymous.

Here is how it works, using simple analogies:

1. The Graduated Spectrum (The 9 Levels)

Instead of just "Clear" or "Blurred," this dataset offers a sliding scale of privacy, like turning a dimmer switch on a light:

Level 1 (The Soft Blur): Imagine looking at a person through a foggy window. You can see their shape and movement, but their face is fuzzy. The robot can still guess the action easily, but it's harder to identify the person.
Level 2 (The Outline): Imagine a "connect-the-dots" drawing or a shadow puppet. All the colors and textures are gone; only the edges of the body remain. The robot has to rely purely on the shape of the movement.
Level 3 (The Puzzle Scramble): Imagine taking a photo of a person, cutting it into a grid of small squares (like a puzzle), and then shuffling those squares randomly. The person's face and body are now a chaotic mess of pixels. The robot has to figure out the action just by looking at the "noise."

The dataset tests three different sizes of these puzzle pieces (Big, Medium, and Tiny). The smaller the pieces, the harder it is for the robot to see anything, but the safer the person's identity is.

2. The "No-Background" Trick (Stopping Cheating)

Here is a sneaky problem: If you blur a person's face but leave the background clear, a smart robot might cheat.

Example: If the robot sees a boxing ring in the background, it might guess "Boxing" without even looking at the person.
The Fix: PrivHAR-Bench includes a special "No-Background" version. It cuts out the person entirely and makes the rest of the screen pitch black. This forces the robot to prove it actually understands the person's movement, not just the room they are in.

3. The "Fair Play" Rules

To make sure everyone is playing by the same rules, the authors created a strict rulebook:

Same Videos: Every single video is processed through all 9 levels.
No Cheating: The videos are saved in a lossless format (like high-quality PNG images) so the "scrambling" isn't accidentally fixed by video compression.
Standardized Tests: Everyone uses the same split of training and testing data, so you can directly compare your results with others.

What Did They Find?

The authors tested a standard AI model (R3D-18) on this new benchmark:

Clear Video: The model was great at 88.8% accuracy.
The Blur/Edge Levels: Accuracy dropped to around 66%. This is a good balance; the robot still works, but the person is much harder to identify.
The Scrambled Levels: Accuracy dropped further to around 63%.
The "No-Background" Scramble: Accuracy dropped to 53.5%. This proves that the background was helping the robot cheat by about 10-15%.

They also checked if the "Puzzle Scramble" actually hid faces. They found that even with the largest puzzle pieces, 89% of faces became unrecognizable to face-recognition software.

Why Does This Matter?

Think of PrivHAR-Bench as a standardized ruler for privacy.
Before this, researchers were all using different rulers (some inches, some centimeters, some made of rubber). Now, they all have the same ruler.

This allows developers to say: "My new privacy method is better because it keeps the robot's accuracy at 70% while hiding the face completely, whereas the old method dropped accuracy to 50%."

It helps us build a future where we can use smart cameras in hospitals, gyms, and public spaces to help people, without sacrificing their right to privacy. It's the bridge between "Smart Technology" and "Safe Technology."

1. Problem Statement

Video-based Human Activity Recognition (HAR) has achieved high accuracy using deep learning, but deploying these systems in privacy-sensitive environments (e.g., healthcare, public spaces) creates a conflict between recognition utility and individual privacy.

Current Limitations: Existing research typically evaluates privacy methods using a binary paradigm (clear video vs. a single transformed version). This approach fails to capture the nuanced trade-off between privacy strength and recognition accuracy.
Evaluation Flaws: Current benchmarks suffer from:
1. Lack of Graduated Spectrum: No intermediate reference points to measure how much utility is lost for a specific gain in privacy.
2. Context Bias: Models often classify actions based on background context (e.g., a kitchen implies cooking) rather than human motion. If privacy transformations only obscure the person, the background remains a "shortcut" for the model, invalidating the privacy test.
3. Non-Comparable Protocols: Inconsistent data splits, resolutions, and metrics across studies prevent direct comparison of methods.

2. Methodology: PrivHAR-Bench Design

The authors introduce PrivHAR-Bench, a standardized, multi-tier benchmark dataset designed to evaluate the Privacy-Utility Trade-off (PUT) under controlled conditions.

A. Dataset Composition

Source: A curated subset of 15 action classes from the UCF101 dataset, selected for diverse human articulation and minimal reliance on background objects.
Scale: 1,932 source video clips, each processed into 9 parallel tiers.
Format: All data is distributed as lossless PNG frame sequences (224×224 resolution, 32 frames per clip) to prevent codec artifacts from corrupting cryptographic transformations.
Splits: Fixed, group-based train/test splits (75%/25%) to prevent identity leakage.

B. The Graduated Privacy Spectrum (9 Tiers)

The dataset applies increasing levels of visual privacy transformations to the Region of Interest (ROI):

Tier 1 (Spatial Obfuscation): Gaussian Blur ( $\sigma=15$ ). Destroys fine-grained features (faces, texture) but preserves body outline.
Tier 2 (Structural Abstraction): Canny Edge Detection. Removes all texture/color, retaining only structural contours.
Tier 3 (Cryptographic Block Permutation): AES-CTR based block scrambling. The ROI is divided into $B \times B$ $B \times B$ blocks and permuted. Three granularities are provided:
- $B=16$ (Coarse)
- $B=8$ (Medium)
- $B=4$ (Fine-grained)
NoBG Variants: For every Tier 3 variant, a background-removed (NoBG) version is generated where non-ROI pixels are set to black. This isolates the model's reliance on the transformed human motion from environmental cues.

C. Generation Pipeline

Deterministic: Uses fixed random seeds, pinned dependencies, and SHA-256 verified weights (YOLOv8n-Pose) to ensure exact reproducibility.
ROI Handling: Uses YOLOv8n-Pose to detect bounding boxes and keypoints. Only the largest detected person is processed.
Encryption: Uses AES-CTR with a unique nonce per frame to ensure no temporal block correspondence (preventing models from tracking block trajectories across time).

3. Key Contributions

First Graduated Privacy Benchmark: Unlike previous works offering binary comparisons, PrivHAR-Bench provides a continuous spectrum of privacy strength, allowing researchers to plot accuracy degradation curves.
Systematic Context Bias Control: The inclusion of NoBG variants forces models to prove they recognize actions from the transformed human region alone, eliminating the "background shortcut" loophole.
Standardized Evaluation Toolkit: A released eval.py toolkit computes Top-1 accuracy, SSIM, PSNR, Face Detection Failure Rate, and a composite Privacy-Utility (PU) Score, ensuring all studies use identical metrics and splits.
Lossless Storage: By distributing PNG sequences rather than compressed video, the dataset preserves the mathematical integrity of cryptographic transformations (which lossy codecs would destroy).

4. Experimental Results

Baseline experiments were conducted using R3D-18 (a 3D ResNet-18 architecture).

Accuracy Degradation Curve:
- Original (Clear): 88.8% accuracy.
- Privacy Tiers (Tier 1-3): Accuracy drops to a range of 63.1% – 66.5%.
- NoBG Variant (Tier 3, B=8): Accuracy drops significantly to 53.5%.
Context Bias Impact:
- Removing the background caused a 10.9 percentage point drop in within-tier accuracy (64.4% $\to$ 53.5%).
- In cross-domain evaluation (trained on Clear, tested on NoBG), accuracy collapsed to 4.8% (near random chance), proving that standard models rely heavily on background context.
Privacy Efficacy (Face Detection):
- Using ArcFace, ~89% of faces became undetectable after block scrambling, regardless of block size ( $B=4, 8, 16$ ). This validates that the cryptographic tiers effectively destroy identity features.
Privacy-Utility (PU) Score:
- Tier 2 (Edge) achieved the highest PU score (0.713), balancing strong privacy (low SSIM) with competitive accuracy.
- Tier 1 (Blur) had the lowest PU score (0.177) due to weak visual privacy (high SSIM) despite good accuracy.

5. Significance and Future Work

Standardization: PrivHAR-Bench establishes a "gold standard" for comparing privacy-preserving HAR methods, removing the variance caused by ad-hoc evaluation protocols.
Insight into Model Robustness: The benchmark reveals that current models are brittle; they rely heavily on background context and fail to generalize when spatial structure is cryptographically scrambled.
Limitations:
- Resolution: Source videos are low resolution (320×240), which may overestimate the effectiveness of block scrambling compared to HD surveillance.
- Scope: Currently limited to single-person actions; multi-person interactions are not addressed.
- Temporal Privacy: The benchmark focuses on spatial privacy; temporal re-identification (gait) risks are not fully mitigated.

Conclusion: PrivHAR-Bench provides the community with the necessary infrastructure to rigorously evaluate and improve privacy-preserving video analytics, moving the field from binary "works/doesn't work" claims to a nuanced understanding of the privacy-utility trade-off. The dataset and code are publicly available via Zenodo and GitHub.