🤖 AI

CFE-PPAR: Compression-friendly encryption for privacy-preserving action recognition leveraging video transformers

This paper proposes CFE-PPAR, the first compression-friendly encryption method for privacy-preserving action recognition that utilizes key-transformed video transformers to maintain high recognition performance and visual quality even when encrypted videos are compressed.

Original authors: Haiwei Lin, Shoko Imaizumi, Hitoshi Kiya

Published 2026-05-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haiwei Lin, Shoko Imaizumi, Hitoshi Kiya

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a secret video of yourself doing a dance routine. You want to send this video to a cloud server so a smart computer can tell you what dance you're doing (action recognition), but you don't want the server to actually see you. You want to keep your identity and the visual details private.

This is the problem of Privacy-Preserving Action Recognition (PPAR).

The Old Problem: The "Fragile Secret"

Previously, people tried to solve this by scrambling the video pixels (encryption) or blurring them (quality reduction).

The Blurring Method: It's like putting a heavy fog over the video. The computer can still guess the dance, but it's hard to tell exactly what's happening, and you might still be recognizable.
The Scrambling Method: This is like taking a jigsaw puzzle of your video, cutting it into pieces, and shuffling them around. The computer can still solve the puzzle if it knows the secret shuffle pattern.

Here's the catch: These old scrambling methods were not "compression-friendly."
Think of video compression (like sending a video on WhatsApp or YouTube) as a way to shrink a file by removing tiny details that the human eye doesn't notice.

If you take a scrambled video and try to shrink it, the file breaks. The "shrinkage" process gets confused by the random shuffling.
The result? The computer fails to recognize the dance, and if you try to un-shuffle the video later to watch it, it comes out looking like a broken, unwatchable mess.

The New Solution: CFE-PPAR

The authors of this paper propose a new method called CFE-PPAR. Think of this as a "Magic Shrink-Proof Scramble."

How it Works (The Analogy)

Imagine your video is a giant mosaic made of small tiles.

The Scramble (Encryption): The client (you) takes the video and cuts it into small blocks. Inside each block, they shuffle the tiles, flip them upside down, or swap colors. This hides your face and body.
The Secret Key: You have a secret key (like a specific recipe) that tells you exactly how you shuffled the tiles.
The "Compression-Friendly" Trick: Unlike old methods that made the video look like static noise, this new method keeps the relationship between the tiles inside each block intact. It's like shuffling a deck of cards but keeping the cards in a neat stack. Because the internal structure is still orderly, standard video compression tools can shrink the file size without breaking it.
The Smart Computer (The Server): The server receives the scrambled, compressed video. It doesn't have your secret key, so it can't see you. However, the server has a special "mirror" version of its brain (the AI model). This brain has been pre-trained with the same secret recipe you used to scramble the video.
- Because the brain knows exactly how the tiles were shuffled, it can look at the scrambled video and say, "Ah, I know this pattern! That's a 'Dance'!"
- It does this without needing to un-scramble the video first.

The Results: What the Paper Claims

The researchers tested this on two famous video datasets (UCF101 and HMDB51) using standard compression formats (Motion-JPEG and H.264).

Accuracy: When the video was compressed and sent to the server, the new method (CFE-PPAR) recognized the actions almost as well as if the video were sent in its original, unencrypted form. The old methods failed miserably under compression, with accuracy dropping drastically.
Recovery: If an authorized person (who has the key) wants to see the original video later, they can un-scramble it. Even after the video was compressed and shrunk, the new method allowed them to recover a clear, watchable video. The old methods resulted in a "catastrophic" loss of quality where the video was unrecognizable.
Security: The paper tested if a hacker could guess the original video just by looking at the scrambled, compressed version (a "ciphertext-only attack").
- If the video used a single key for the whole clip, a hacker could partially reconstruct it (like solving a jigsaw puzzle).
- However, if the video used different keys for different blocks (a variation called V2), the hacker couldn't make sense of it at all. The video remained secure.

Summary

CFE-PPAR is a new way to hide video content so that:

The video can be shrunk (compressed) for easy transmission without breaking the privacy.
A smart computer can still understand what is happening in the video without seeing the actual people.
The original video can be perfectly restored later if you have the key.

It solves the specific problem that previous "scrambled" videos were too fragile to be compressed, making them useless for real-world applications where data needs to be sent over the internet.

Technical Summary: CFE-PPAR

Problem Statement
Privacy-preserving action recognition (PPAR) aims to enable machines to classify human activities in videos without exposing sensitive visual content. While encryption-based PPAR methods offer strong privacy protection and high recognition performance on uncompressed data, they face a critical limitation: they are not "compression-friendly." When videos encrypted by existing methods (such as perceptual encryption or homomorphic encryption) undergo standard video compression (e.g., Motion-JPEG or H.264), recognition accuracy suffers a catastrophic drop. Furthermore, the decryption process often fails to recover faithful visual content after compression, rendering these methods impractical for real-world scenarios where data transmission and storage require compression.

Methodology
The authors propose CFE-PPAR (Compression-Friendly Encryption for Privacy-Preserving Action Recognition), a novel framework designed to maintain high recognition accuracy and visual recoverability even after lossy compression. The method integrates three core components:

Compression-Friendly Encryption (CFE):
Unlike previous encryption methods that disrupt the spatial correlations necessary for compression algorithms, CFE employs a block-wise encryption strategy that preserves intra-block correlation. The process involves:
- Preprocessing: Resizing video frames to match the transformer's input dimensions.
- Block Division: Dividing each frame into Main-Blocks (MBs) matching the patch size of the Video Transformer (VT), and further subdividing MBs into Sub-Blocks (SBs).
- Sub-block Transformations: Applying five specific transformations (rotation, flipping, pixel value inversion, RGB-channel shuffling, and sub-block scrambling) to each SB using a secret key ( $K_{ST}$ ).
- Main-block Scrambling: Permuting the order of MBs within a frame using a separate secret key ( $K_{MS}$ ).
  This approach ensures that while visual information is obscured, the statistical properties required for effective compression remain intact.
Key-Dependent Domain Adaptation (KDDA):
To allow a Video Transformer to recognize encrypted videos without retraining or architectural changes, the model parameters are transformed to match the encryption. Specifically, the 3D convolution kernel ( $E$ ) of the cube embedding layer and the positional embeddings ( $E_{pos}$ ) are modified using the same keys ( $K_{ST}$ and $K_{MS}$ ) used for encryption. This creates a "domain-adaptation VT" that can directly infer on encrypted inputs.
Threat Model:
The system assumes an untrusted public server that handles encrypted, compressed data but possesses no secret keys. The adversary is assumed to have access to the encrypted data and the algorithm but not the keys, limiting them to Ciphertext-Only Attacks (COAs).

Key Contributions

First Compression-Friendly PPAR: This work introduces the first encryption method specifically designed to be compatible with standard video compression codecs (MJPEG and H.264) while preserving action recognition capabilities.
Dual Optimization: The method simultaneously achieves high recognition accuracy on encrypted, compressed data and allows for the faithful reconstruction of original video frames by authorized clients.
Training-Free Inference: The approach utilizes a pre-trained Video Transformer with transformed parameters, eliminating the need for fine-tuning the model on encrypted data or modifying the network architecture.

Experimental Results
The authors evaluated CFE-PPAR on the UCF101 and HMDB51 datasets under Motion-JPEG and H.264 compression (with and without inter-frame prediction) across various bitrates.

Recognition Accuracy: Under uncompressed conditions, CFE-PPAR achieved accuracy comparable to plain (unencrypted) videos. Under lossy compression, it significantly outperformed existing methods (LCVE and BDQ). Notably, under H.264 with inter-frame prediction, CFE-PPAR maintained accuracy nearly on par with plain videos even at mid-to-high bitrates, whereas previous methods suffered severe degradation.
Visual Recovery: While previous encryption methods resulted in decrypted videos with low PSNR (often below 20 dB) after compression, CFE-PPAR effectively mitigated this degradation, allowing for reliable reconstruction of original frames.
Attack Resistance: The paper evaluated resistance against Ciphertext-Only Attacks using an Extended Jigsaw Puzzle Solver (EJPS). The results indicated that the variant using different keys for each Main-Block (CFE-PPAR V2) was robust against reconstruction, whereas the variant using a single key for all blocks (V1) was vulnerable when uncompressed. However, compression itself significantly hindered reconstruction attempts for both variants.

Significance and Claims
The paper claims that CFE-PPAR addresses a fundamental gap in privacy-preserving computer vision: the incompatibility between strong encryption and efficient data compression. By demonstrating that encrypted videos can be compressed without sacrificing recognition performance or visual recoverability, the authors argue that their method enables the practical deployment of PPAR systems in cloud and edge environments where bandwidth and storage are constrained. The work verifies that it is possible to achieve strong privacy protection, high utility (accuracy), and compression efficiency simultaneously, a combination previously unattainable with existing encryption-based strategies.