💻 computer science

Face anonymization preserving facial expressions and photometric realism

This paper proposes a feature-preserving face anonymization framework that extends DeepPrivacy with dense landmarks and post-processing modules to generate realistic images that effectively conceal identity while maintaining high fidelity in facial expressions, lighting, and skin tone, addressing a critical gap in existing privacy-preserving methods.

Original authors: Luigi Celona, Simone Bianco, Raimondo Schettini

Published 2026-03-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Luigi Celona, Simone Bianco, Raimondo Schettini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a photo of yourself smiling at a birthday party. You want to share it with the world so people can see your joy and the lighting of the room, but you don't want anyone to know it's you.

This is the problem of Face Anonymization.

The Old Way: The "Blurry Sticker"

In the past, if you wanted to hide your identity, you'd use a digital "sticker" to cover your face, or you'd blur it out like a paparazzi photo.

The Problem: This is like putting a giant, fuzzy smudge over your face. Sure, no one knows who you are, but they also can't see if you're happy, sad, or angry. They can't tell if the sun is shining or if it's raining. The photo becomes useless for anything other than "hiding."

The New Way: The "Digital Mask Maker"

Recently, scientists started using AI (specifically something called GANs) to generate a new face that looks real but isn't yours. Think of it like a high-tech mask maker.

The Problem with Early AI Masks: The early AI masks were good at hiding your identity, but they were terrible at keeping the "vibe" of the photo.
- If you were smiling, the AI might give you a blank, stone-faced stare.
- If you were lit by warm sunset light, the AI might make your new face look like it was under cold, blue hospital lights.
- If you had a warm skin tone, the AI might make the new face look pale or grey.

This made the photos useless for things like studying human emotions or testing how cameras handle different lighting.

The Solution: "The Perfect Digital Doppelgänger"

The paper you shared introduces a new method that creates a perfect digital doppelgänger. It doesn't just hide your identity; it keeps all the important details that make the photo useful.

Here is how they did it, using simple analogies:

1. The "Detailed Blueprint" (Dense Landmarks)

Imagine trying to draw a face based on a sketch that only has 7 dots (eyes, nose, ears). It's hard to get the expression right.

What they did: Instead of 7 dots, they gave the AI a blueprint with 68 dots.
The Result: The AI can now see the tiny crinkles around your eyes when you laugh or the slight furrow of your brow when you are thinking. It preserves your facial expression perfectly, so the new face looks just as happy or serious as the original.

2. The "Lighting Transfer" (The Shadow Copy)

Imagine you take a photo of a statue in a sunny park. If you swap the statue's face with a new one, you have to make sure the new face has the same shadows and highlights, or it will look like a sticker pasted on.

What they did: They used a clever trick called "Intrinsic Decomposition." Think of this as separating the photo into two layers:
- Layer A (The Paint): The actual skin color and texture (which changes).
- Layer B (The Light): The shadows and brightness (which stays the same).
The Result: They took the "Light" layer from your original photo and wrapped it around the new AI face. Now, the new face has the exact same lighting and shadows as the original. If you were in the sun, the new face is in the sun.

3. The "Skin Tone Match" (The Color Filter)

Sometimes AI makes people look like they have a different race or skin tone, which is unfair and inaccurate.

What they did: They added a final step where they check the "color statistics" of the original photo and gently adjust the new face to match.
The Result: The new face has the same skin tone as the original person. It looks natural and fair, without accidentally changing the person's demographic appearance.

Why Does This Matter?

Think of this technology as a privacy-preserving translator.

Before: You had to choose between Privacy (hiding your face) and Utility (keeping the photo useful). You couldn't have both.
Now: You can have a photo where your identity is completely erased (so no one can track you), but the emotion, the lighting, and the skin tone remain exactly the same.

Real-world examples:

Medical Research: Doctors can study how people's faces change when they are in pain or happy, without violating patient privacy.
Self-Driving Cars: Engineers can test how cars react to different lighting conditions (sunset, night, rain) using real human faces, without exposing the drivers' identities.
Social Media: You can share your vacation photos with the world, and the AI will swap your face with a realistic stranger's face that still looks like you in that specific lighting and mood.

The Bottom Line

This paper is about teaching AI to be a better "body double." It's not just about hiding the actor; it's about making sure the body double performs the scene with the same emotion, under the same lights, and wearing the same "costume" (skin tone) as the original actor. This makes the data safe for privacy but still incredibly useful for science and technology.

1. Problem Statement

The widespread use of facial images in social media and large-scale datasets raises significant privacy concerns regarding biometric identification. While face anonymization aims to conceal identity while retaining data utility, existing methods suffer from critical limitations:

Loss of Utility: Traditional methods (masking, blurring, pixelation) destroy structural and photometric information, rendering data useless for downstream tasks like expression analysis or relighting.
Generative Shortcomings: Current Generative Adversarial Network (GAN) approaches (e.g., DeepPrivacy) focus primarily on identity removal and general image realism. They often neglect photometric consistency (lighting direction, shadows, skin tone) and facial expressions.
Consequence: Anonymized datasets often exhibit distorted shadows, shifted skin tones, and flattened expressions, introducing bias and reducing effectiveness in applications such as affective computing, medical analysis, and fair recognition systems.

2. Methodology

The authors propose a feature-preserving anonymization framework that extends the DeepPrivacy architecture. The method consists of three core components:

A. Enhanced Conditioning with Dense Landmarks

Modification: The original DeepPrivacy uses a sparse set of 7 landmarks (ears, eyes, nose, shoulders) derived from Mask R-CNN.
Improvement: The proposed method replaces this with a dense set of 68 facial landmarks (using the DLib implementation).
Impact: This provides the generator with finer geometric priors, enabling the synthesis of subtle facial deformations and preserving the subject's affective state (expression) more faithfully.

B. Photometric Post-Processing

To ensure the anonymized face matches the original in lighting and color without reintroducing identity, two lightweight post-processing modules are applied after GAN generation:

Relighting via Intrinsic Decomposition:
- The method assumes an image $I$ can be factorized into Albedo ( $A$ , reflectance) and Shading ( $S$ , illumination): $I \approx A \cdot S$ .
- Using a Retinex-style bilateral filter, the shading map ( $S_{Io}$ ) is extracted from the original image.
- The anonymized face's albedo ( $A_{Ia}$ ) is combined with the original's shading: $I_{relight} = A_{Ia} \cdot S_{Io}$ .
- Refinement: A Laplacian pyramid blending technique is used to merge the relit image with the original across different spatial frequencies, ensuring smooth transitions and preserving high-frequency details.
Color Transfer in YCbCr Space:
- To preserve skin tone statistics, a channel-wise color transfer is performed in the YCbCr color space.
- The luminance channel ( $Y$ ) is kept unchanged to preserve structure.
- The chrominance channels ($Cb, Cr$) are adjusted to match the mean and standard deviation of the original image, ensuring skin color consistency without leaking identity information.

3. Key Contributions

Architectural Enhancement: Integration of dense 68-point facial landmarks into the DeepPrivacy conditioning mechanism to significantly improve expression retention.
Photometric Consistency Modules: Introduction of model-agnostic, lightweight post-processing (Intrinsic Decomposition + Laplacian Pyramid + Color Transfer) to restore lighting direction and skin tone, addressing a gap in existing GAN-based anonymization.
Comprehensive Evaluation Framework: Development of new metrics specifically for expression fidelity, lighting consistency (Si-MSE, Si-L2), and skin color fidelity ( $\Delta E_{2000}$ ), complementing standard metrics like FID and re-identification rates.

4. Experimental Results

The method was evaluated on the CelebA-HQ dataset against eight state-of-the-art baselines (including DeepPrivacy, DeepPrivacy2, G2Face, RiDDLE, and AIDPro).

Image Quality: Achieved competitive FID scores (FID $_{ImageNet}$ : 10.54, FID $_{CLIP}$ : 8.27), ranking second-best overall, indicating high perceptual realism.
Privacy: Successfully obscured identity, with FaceNet re-identification similarity scores comparable to top-tier methods (e.g., 0.2125 vs. 0.0326 for RiDDLE, balancing privacy with utility).
Expression Preservation: Achieved the highest emotion classification accuracy (75.00%) and the second-lowest landmark error, significantly outperforming DeepPrivacy (53.29%) and G2Face (72.00%).
Lighting Consistency: Achieved the best Si-L2 (0.0100) and near-best Si-MSE scores, outperforming even G2Face (specifically designed for geometry), proving superior preservation of shading and illumination.
Skin Color: Achieved a low perceptual color difference ( $\Delta E_{2000} = 8.93$ ), nearly matching the best-performing method (G2Face at 7.83) while maintaining lower variance.

Ablation Study: Confirmed that the dense landmarks improved expression accuracy from 53% to 72%, while the Intrinsic Decomposition relighting significantly reduced lighting errors compared to learned relighting networks (SfSNet).

5. Significance and Impact

Utility for Downstream Tasks: By preserving expressions, lighting, and skin tone, the anonymized data remains viable for affective computing, relighting, medical analysis, and fairness-aware recognition.
Bias Mitigation: Preserving skin tone statistics helps prevent the amplification of demographic biases often introduced by generative models that shift skin colors.
Paradigm Shift: The paper argues that effective anonymization must move beyond simple identity suppression to feature-aware preservation. This ensures that privacy-preserving datasets are not just "safe" but also scientifically valid and ethically robust for sensitive domains like healthcare and social robotics.
Efficiency: The use of intrinsic decomposition and color transfer avoids the heavy computational cost of training complex relighting networks, making the pipeline suitable for real-time or large-scale applications.