Arc2Morph: Identity-Preserving Facial Morphing with Arc2Face

This paper introduces Arc2Morph, a novel deep learning-based facial morphing technique leveraging the Arc2Face foundation model that achieves identity-preserving morphing attacks with effectiveness comparable to traditional landmark-based methods, thereby posing a significant challenge to current face recognition systems.

The Gist

The Digital Doppelgänger: How "Arc2Morph" Creates the Ultimate Fake ID

Imagine you and your best friend decide to create a digital hybrid. You don't just take a photo of you and a photo of your friend; you blend them together so perfectly that the result looks like a single, real person who is somehow both of you at the same time.

This is exactly what the paper "Arc2Morph" is about. It introduces a new, super-smart way to create these "face hybrids" (called morphs) that can trick both human guards and computer security systems.

Here is the breakdown of how it works, why it's scary, and why the researchers are actually doing this to help us.

---

1. The Problem: The "Two-Face" Passport Scam

Think of an electronic passport (like the ones with chips inside) as a digital lock. When you apply for one, a photo is taken and stored in that chip. Later, when you travel, a machine scans your face and checks: "Does this live person match the photo in the chip?"

The Attack:
Two people (let's call them Alice and Bob) want to share one passport.

1. They create a morphed photo that looks 50% Alice and 50% Bob.
2. They trick the human officer at the airport into thinking this hybrid photo is a real, single person.
3. They get the passport issued with this hybrid photo.
4. Later, Alice can use the passport to travel. The machine says, "Hey, that looks like the photo!" (because it's half Alice).
5. Then, Bob uses the same passport. The machine says, "Hey, that looks like the photo!" (because it's half Bob).

They have successfully hacked the system, sharing one identity between two people.

2. The Old Way vs. The New Way

For years, hackers used two main methods to make these hybrids:

• The "Ruler and Glue" Method (Landmark-based): Imagine taking a photo of Alice and Bob, drawing dots on their eyes, noses, and mouths, and then using software to stretch and warp their faces until the dots match, then smearing the pixels together. It's like using a ruler to draw a line between two pictures. It works well, but if you look closely, the eyes might look weird or the skin texture might look "smudged."
• The "AI Painter" Method (Deep Learning): This uses Artificial Intelligence to "dream up" a new face that is a mix of the two. The problem with early AI painters was that they often forgot who the people were. The result might look like a smooth, pretty face, but it didn't look enough like Alice or Bob to trick the security cameras.

3. The New Hero: "Arc2Morph" (The Identity Chef)

The authors of this paper created a new method called Arc2Morph. They used a special AI tool called Arc2Face.

Think of Arc2Face as a master chef who is obsessed with the "flavor" of a person's identity.

• The Ingredients: Instead of just looking at the pixels of the face, the AI looks at the "mathematical DNA" (called embeddings) that defines who Alice and Bob are.
• The Recipe: The AI takes the "identity DNA" of Alice and Bob, mixes them together in a special mathematical bowl (the CLIP latent space), and then asks the chef to bake a brand new face based on that mixture.
• The Secret Sauce: The chef is also given a "mold" (a 3D map of the face) to ensure the new face looks exactly like a passport photo (straight on, white background, no funny shadows).

Why is this special?
Previous AI methods were like a chef who forgot the recipe and just made a generic cake. Arc2Morph is a chef who remembers exactly how Alice and Bob taste, mixes them perfectly, and serves a cake that tastes like both of them simultaneously.

4. Did It Work? (The Taste Test)

The researchers put their new "hybrid faces" to the test against the best security systems in the world.

• The Score: They used a metric called MAP (Morphing Attack Potential). Think of this as a "Success Score." A score of 100% means the fake face fooled the system every single time.
• The Result: Arc2Morph scored incredibly high (often near 99% or 100%).
• The Shock: It didn't just beat the other AI methods; it beat the old "Ruler and Glue" methods too. For a long time, people thought the old methods were the hardest to detect. Arc2Morph proved that the new AI method is actually more dangerous because the faces look so real and preserve the identity so well.

5. Why Are They Doing This? (The Good Guys)

You might be thinking, "Wait, isn't this helping criminals?"

Actually, the researchers are the firefighters, not the arsonists.

• You can't build a better fire alarm if you don't understand how fire spreads.
• By creating the most dangerous "fake faces" possible, they are showing security companies: "Look! Your current alarms are broken. Here is a face that fools them. You need to build a better detector to catch this."

They are releasing their code and data to the public so that security experts can study these fakes and build stronger defenses for our passports and ID cards.

Summary Analogy

Imagine a locksmith (the security system) trying to keep a vault safe.

• Old Hackers used a bump key (the old "Ruler and Glue" method). It worked, but the locksmith could eventually learn to spot the scratches.
• New Hackers (using Arc2Morph) are using a 3D-printed key that is a perfect, invisible copy of the original.
• The Researchers are the locksmiths who built the 3D printer. They aren't trying to rob the bank; they are building the 3D printer to show the bank, "Hey, your locks are too weak for this new key. You need to upgrade to a biometric scanner that can't be fooled by 3D prints!"

The Bottom Line: This paper shows that AI is getting so good at faking faces that our current ID systems are in serious danger. But by exposing this weakness, the authors are helping us build a future where our digital identities are actually safe.

Technical Summary

1. Problem Statement

Face morphing attacks represent a critical vulnerability in electronic identity documents (e.g., passports). These attacks involve creating a single facial image that blends the features of two individuals.

• The Threat: If a morphed image is accepted during the enrollment phase (often without live capture supervision), it can be stored in a document's chip. Later, both original individuals can use the same document to authenticate, effectively sharing a single identity.
• The Challenge: A successful attack must satisfy two conditions:
  1. Human Deception: The image must look realistic and resemble both subjects to a human examiner, with no visible artifacts.
  2. Machine Deception: The image must successfully match against both contributing subjects when processed by Face Recognition Systems (FRSs).
• Current Limitations: Traditional landmark-based methods (geometric warping + texture blending) have historically been the most effective and hardest to detect. Deep learning-based methods often struggle to preserve identity information as effectively or generate images with the specific constraints (pose, background) required for ISO/ICAO compliance.

2. Methodology: Arc2Morph

The authors propose Arc2Morph, a novel deep learning-based framework that leverages Arc2Face, an identity-conditioned face foundation model. The pipeline is designed to generate high-quality, ISO/ICAO-compliant morphed images while maximizing attack potential.

Core Pipeline Steps:

1. Identity Extraction:
   • Two input images ($I_A, I_B$) are processed by a pre-trained ArcFace encoder ($E_{id}$) to extract compact 512-dimensional identity embeddings ($e_A, e_B$).
2. CLIP Space Projection:
   • The ArcFace embeddings are injected into a fixed textual prompt ("photo of a ⟨id⟩ person") and processed by a modified CLIP text encoder ($E_t$).
   • This maps the identity embeddings into the CLIP latent space, producing semantic representations ($p_A, p_B$). This step bridges the gap between biometric identity features and the generative model's conditioning space.
3. Identity Interpolation:
   • The two CLIP representations are interpolated to create a hybrid identity representation ($p_M$).
   • The paper evaluates two interpolation strategies: Linear Interpolation (lerp) and Spherical Linear Interpolation (slerp).
   • The interpolation is controlled by a factor $\alpha$ (default 0.5 for an equal blend).
4. Image Synthesis (Arc2Face):
   • The interpolated latent vector $p_M$ is fed into the Arc2Face generator.
   • Pose/Expression Control: To ensure ISO/ICAO compliance, a ControlNet module conditions the generation on a 3D facial normal map ($I_P$) extracted from one of the input images using EMOCAv2. This ensures the output has a neutral expression and frontal pose.
5. Post-Processing:
   • The BEN2 background removal network is applied to isolate the face and replace the background with a uniform white color, meeting strict document photo standards.

3. Key Contributions

• Novel Architecture: Introduction of a deep learning-based morphing method that outperforms existing State-of-the-Art (SOTA) deep learning techniques and rivals traditional landmark-based methods in attack potential.
• Identity Preservation Strategy: A unique approach of projecting ArcFace embeddings into the CLIP latent space before interpolation, leveraging CLIP's rich semantic structure to blend identities more effectively than direct feature interpolation.
• Compliance & Control: Explicit control over pose, expression, and background to generate images that strictly adhere to ISO/ICAO standards, making the attacks more realistic for real-world enrollment scenarios.
• New Datasets: Release of two new morphed face datasets (derived from FEI and ONOT) created using the proposed method for benchmarking.
• Reproducibility: Public release of the implementation code.

4. Experimental Results

The authors evaluated the method using the Morphing Attack Potential (MAP) metric, which measures the probability of a morphed image successfully matching multiple probe images across multiple FRSs.

• Datasets: Evaluation was conducted on four datasets: FEI Morph v2 (real images), MONOT (synthetic), EINMorph-HQ v2, and EINMorph-MQ v2.
• Performance vs. Competitors:
  • FEI Morph v2: Arc2Morph achieved a MAP of 99.9% (1 probe, 1 FRS) and 98.7% (1 probe, 3 FRSs), outperforming all 7 competing SOTA algorithms (both landmark and deep learning-based).
  • MONOT (Synthetic): The method achieved the highest attack capability against a single FRS (>97%). While some landmark methods showed slightly higher generality against 3 FRSs, Arc2Morph remained highly competitive.
  • EINMorph Datasets: The method demonstrated superior robustness and generality curves compared to all competitors, including landmark-based techniques (C01, C02, etc.), which were previously considered the hardest to detect.
• Ablation Study:
  • The study compared interpolation in the ArcFace identity space vs. the CLIP latent space, and lerp vs. slerp.
  • Best Configuration: Interpolating in the CLIP latent space using spherical linear interpolation (slerp) yielded the highest average MAP. The authors hypothesize that CLIP's higher dimensionality and semantic structure allow for finer identity blending than the ArcFace space alone.

5. Significance and Implications

• Security Threat: The paper demonstrates that deep learning-based morphing has caught up to, and in some metrics surpassed, traditional landmark-based attacks. This suggests that current detection systems relying on the assumption that deep learning morphs are "easier to detect" may be insufficient.
• Benchmarking: By releasing new datasets and code, the authors provide the community with essential tools to develop more robust detection mechanisms.
• Future Defense: The findings highlight the need for advanced liveness detection and morphing detection algorithms that can handle identity-preserving generative models, particularly those that can control pose and background to mimic official document photos.
• Ethical Stance: The research is conducted strictly for defensive purposes (improving detection systems) using public and synthetic data, with no collection of new biometric data or interaction with human subjects.

In conclusion, Arc2Morph establishes a new benchmark for the severity of deep learning-based face morphing attacks, proving that foundation models like Arc2Face can generate highly deceptive, identity-preserving morphs that satisfy strict regulatory standards.