💻 computer science

HonestFace: Towards Honest Face Restoration with One-Step Diffusion Model

The paper introduces HonestFace, a one-step diffusion model for face restoration that leverages an identity embedder, masked face alignment, and a new multi-reference dataset to achieve high-fidelity, authentic reconstructions with superior identity consistency compared to state-of-the-art methods.

Original authors: Jingkai Wang, Wu Miao, Jue Gong, Zheng Chen, Xing Liu, Hong Gu, Yutong Liu, Yulun Zhang

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Jingkai Wang, Wu Miao, Jue Gong, Zheng Chen, Xing Liu, Hong Gu, Yutong Liu, Yulun Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fix a blurry, scratched-up photo of your best friend taken at a crowded concert. You want to make it look crisp and clear again, but you have a problem: the original photo is so damaged that the computer doesn't know what your friend's eyes actually look like, or if they have a tiny mole near their mouth. If the computer just guesses, it might give your friend blue eyes when they have brown ones, or smooth out their skin until it looks like plastic. This is the world of "face restoration," a branch of computer science where machines try to repair damaged images. For years, scientists have built tools to sharpen these photos, but there's a catch. Many of these tools are too eager to please; they invent new details that look nice but aren't true to the person in the picture. They might make a face look perfect, but it stops looking like your friend and starts looking like a generic, airbrushed stranger. The big question is: how do we fix a broken photo without lying about what the person actually looks like?

Enter HonestFace, a new method developed by researchers at Shanghai Jiao Tong University and vivo that aims to be the "truth-teller" of photo repair. Think of most current face-restoration AI as a creative writer who, when asked to finish a story, makes up a happy ending that doesn't fit the characters. HonestFace, on the other hand, is like a strict historian who refuses to invent facts. It uses a special "one-step" process to look at a blurry photo and a collection of clear, high-quality photos of the same person (like a reference album) to reconstruct the face. Instead of guessing, it carefully extracts the real identity features—like the exact shape of the eyes and the texture of the skin—from the good photos and blends them with the blurry one. The result is a restored image that looks incredibly sharp and natural, but most importantly, it stays "honest" to the original person, keeping their unique wrinkles, eye color, and even small blemishes, rather than smoothing them away into a fake, plastic look.

The Problem: When "Perfect" Means "Fake"

Face restoration is like trying to solve a puzzle where half the pieces are missing or melted. When a photo is low-quality (LQ)—maybe it's blurry, noisy, or compressed—the computer loses the fine details that make a face unique. Existing tools try to fill in the gaps using "generative" models, which are like artists that have seen millions of faces and try to paint a new one based on what they know.

The problem is that these artists often get too creative. They might smooth out skin so much that it looks like a doll, or they might change the hair texture to look too uniform, losing the messy, natural strands of real hair. This is called "over-smoothing." Other times, they might shift the colors, making a person's skin look too pink or their eyes a different shade. While these photos look "high quality" in terms of sharpness, they fail the "honesty" test: they don't look like the real person anymore. They look like a generic, perfect version of a human, which is great for a movie character but bad for identifying your friend in a crowd.

The Solution: The "Honest" Detective

The authors propose HonestFace, a system designed to fix these issues by being "honest" about what it sees. It operates on a simple but powerful idea: if you want to fix a blurry photo of a specific person, you shouldn't just guess; you should look at other clear photos of that same person to guide the repair.

Here is how HonestFace works, broken down into its three main "superpowers":

1. The Identity Detective (Identity Embedder)
Imagine you are trying to describe a friend to a painter, but you can only show them a blurry photo. You also have a folder of clear photos of that friend. A normal AI might just look at the blurry photo and guess. HonestFace, however, acts like a detective. It has two special tools:

The Feature Finder: It zooms in on the most consistent parts of the face, like the eyes, to grab specific details like eye color and the shape of the eyelids from the clear reference photos.
The Identity Keeper: It looks at the whole face to understand the "big picture" of who the person is.
By combining these two, it creates a "prompt" (a set of instructions) that tells the AI exactly who it is painting, ensuring the eyes and overall look match the real person, not just a random guess.

2. The Spotlight Guide (Masked Face Alignment)
Even with the right identity, an AI might still mess up the texture. It might make the skin look too smooth or the hair too perfect. To fix this, HonestFace uses a "spotlight" technique. It creates a map (a heatmap) that highlights the most important parts of the face for human perception—like the eyes, the mouth, and the texture of the skin around wrinkles.
Think of it like a teacher grading a test who only looks at the most important questions. The AI focuses its energy on making these specific areas look real and textured, rather than wasting effort on less important parts. This ensures that the restored face has natural-looking pores, fine wrinkles, and messy hair, rather than a plastic, airbrushed finish.

3. The One-Step Magic
Usually, fixing a photo with these advanced AI models takes many steps, like peeling an onion layer by layer, which takes a long time. HonestFace uses a "one-step diffusion model." This is like a magician who can pull a rabbit out of a hat in a single snap instead of doing a long, complicated trick. It takes the blurry photo and the reference photos and produces the final, high-quality result in one go. This makes it incredibly fast, taking only about 0.13 seconds to process an image, which is fast enough to use in real-time apps.

The New Proof: A Real-World Test

To prove that their method actually works, the researchers didn't just use fake, computer-generated bad photos. They created a new dataset called MultiRefCeleb-Test. This is a collection of over 9,000 real photos of more than 400 celebrities taken in real-world conditions (like concerts, as shown in the paper's Figure 1). These photos have real problems: bad lighting, motion blur, and compression artifacts.

The results showed that HonestFace outperformed all the other top methods.

Visual Quality: In side-by-side comparisons, HonestFace produced faces that looked more natural. While other methods made faces look like smooth plastic or changed eye colors, HonestFace kept the skin texture realistic and the colors accurate.
Identity: It was much better at keeping the person's identity. If you took a blurry photo of a celebrity, HonestFace restored a face that looked like that celebrity, whereas others often looked like a different person or a generic model.
Speed: Despite doing a more complex job, it was just as fast as the fastest existing one-step methods.

What It Doesn't Do (and What It Rules Out)

The paper is very clear about what it is not. It argues against the idea that "smoother is better." It explicitly rejects methods that produce over-smoothed, "plastic" faces, even if those faces look sharp. It also argues against methods that rely on a single reference photo or average multiple photos together, suggesting that these approaches miss out on the rich details needed for a truly "honest" restoration.

The authors are confident in their results based on extensive testing on both synthetic (computer-made) and real-world datasets. They measured their success using standard metrics like how close the restored image is to the original (PSNR, SSIM) and how "real" it looks to human judges (LPIPS, MANIQA). The data shows that HonestFace consistently scores higher in these areas than previous methods, suggesting that it is a significant step forward in making face restoration both high-quality and truthful.

In short, HonestFace is a new way to fix photos that says, "Let's make it look great, but let's make sure it still looks like you." It combines the speed of modern AI with a careful, detail-oriented approach that refuses to sacrifice the truth for the sake of a pretty picture.

Technical Summary: HonestFace

Problem Statement

Face restoration aims to reconstruct high-quality (HQ) facial images from low-quality (LQ) inputs suffering from severe degradations such as blur, noise, and compression artifacts. While recent advancements in Transformers, GANs, and diffusion models have improved restoration quality, a critical challenge remains: achieving "honest" restoration. Current generative models often produce results that are sharp but lack authenticity, exhibiting issues such as over-smoothing (loss of skin pores and wrinkles), repetitive or artificial hair textures, and significant color shifts. Furthermore, maintaining strict identity fidelity is difficult when LQ inputs lack sufficient identity cues, leading to restored faces that deviate from the subject's true appearance. Existing reference-based methods often fail to fully exploit multiple high-quality references or struggle with diverse real-world degradation levels.

Methodology

The authors propose HonestFace, a novel one-step diffusion model designed to balance identity consistency with realistic texture generation. The framework operates within a latent diffusion space and incorporates three primary technical components:

1. Identity Embedder (IDE)

To address identity fidelity, the IDE employs a dual-path architecture to extract and preserve crucial identity features from both the LQ input and multiple high-quality reference images ( $R$ ):

Facial Feature Extractor: Recognizing that specific attributes like eye color and skin texture are highly consistent across images of the same individual, this module selects the most geometrically similar reference image (based on affine landmark distance, $d_{A-LD}$ ). It then crops eye patches from this optimal reference and processes them through a pre-trained VGG16 network to extract color and structural embeddings.
Face Identity Encoder: This component processes the entire set of aligned reference images using a pre-trained face recognition model (ArcFace) to generate global identity embeddings. These are aggregated via an attention-based module to produce a robust identity representation.
Fusion: The features from the facial feature extractor, the face identity encoder, and a Visual Representation Embedder (VRE) derived from the LQ input are fused via a trainable attention layer to form the final prompt embedding ( $p$ ) for the diffusion model.

2. Masked Face Alignment (MFA)

To enhance perceptual naturalness and avoid over-smoothing, the authors introduce MFA. This method generates a heatmap-based attention mask from the LQ input using a pre-trained landmark detection model.

The heatmap highlights perceptually critical regions (e.g., eyes, mouth, wrinkles) while de-emphasizing less critical areas.
This mask is applied to the alpha channel during the loss calculation, blending the ground truth and restored image against a neutral background.
A localized perceptual loss ( $L_{MFA}$ ) is computed using LPIPS on these masked regions, forcing the model to focus computational capacity on reconstructing fine textures and natural colors in salient areas.

3. One-Step Diffusion Framework with Adversarial Distillation

HonestFace utilizes a one-step inference strategy for efficiency. The model is trained using an adversarial distillation strategy that combines:

Variational Score Distillation (VSD): To guide the one-step generation.
GAN Discriminator: An adversarial loss is employed to jointly optimize the generator and discriminator, ensuring the generated distribution aligns with real image distributions.
Loss Functions: The total objective includes the adversarial loss, an identity preservation loss ( $L_{ID}$ ) using an ensemble of face extractors, a global perceptual loss ( $L_{Per}$ ) based on EA-DISTS, and the proposed local perceptual loss ( $L_{MFA}$ ).

Key Contributions

Identity Embedder (IDE): A novel module combining a facial feature extractor (focusing on eye patches from optimal references) and a face identity encoder to robustly preserve both fine-grained details and global identity consistency.
Masked Face Alignment (MFA): A perceptual loss strategy that uses heatmap-derived masks to strategically concentrate restoration efforts on human-perception-critical regions, ensuring authentic textures and reducing color shifts.
MultiRefCeleb-Test Dataset: The introduction of a large-scale, real-world benchmark containing over 400 identities and 9,000+ images. Unlike synthetic benchmarks, this dataset features authentic degradations and provides multiple high-quality references per identity.
HonestFace Model: A one-step diffusion model that integrates multiple references to achieve state-of-the-art performance in both visual quality and identity fidelity, effectively mitigating common issues like over-smoothing and repetitive artifacts.

Experimental Results

The authors evaluated HonestFace on synthetic benchmarks (CelebHQRef-Test, Reface-Test) and the new real-world MultiRefCeleb-Test dataset.

Quantitative Performance: HonestFace achieved state-of-the-art results across most metrics. On synthetic datasets, it outperformed reference-based methods (e.g., ASFFNet, DMDNet, FaceMe) and no-reference methods (e.g., CodeFormer, DAEFR) in perceptual metrics (LPIPS, DISTS, C-IQA, M-IQA) and identity consistency metrics (Deg., LMD). On the real-world MultiRefCeleb-Test, it demonstrated superior performance in FID and perceptual scores compared to existing SOTA methods.
Qualitative Performance: Visual comparisons showed that HonestFace produces more natural skin textures, preserves unique features (e.g., moles, wrinkles), and maintains accurate eye and skin colors, avoiding the "plastic" or over-smoothed appearance common in diffusion-based baselines like DiffBIR or FaceMe.
Efficiency: Despite adding modules for multi-reference handling, the model maintains competitive inference complexity, requiring only one step and comparable parameters/MACs to other one-step diffusion models.

Significance

The paper posits that "honest" face restoration requires a paradigm shift from merely generating sharp images to accurately reflecting the original subject's characteristics without introducing artifacts. By integrating multi-reference guidance with a one-step diffusion framework and perceptually guided loss functions, HonestFace addresses the trade-off between fidelity and realism. The authors claim that their approach sets a new standard for face restoration in real-world scenarios, particularly for applications where identity consistency and natural texture are paramount, such as smartphone portrait enhancement. The release of the MultiRefCeleb-Test dataset is highlighted as a significant contribution for future research into authentic, degradation-aware face restoration.