💻 computer science

Discovering Geometric Biases in 3D Face Reconstruction: A Curvature-Aware Spectral Framework for Fairness Evaluation

This paper proposes a novel curvature-aware spectral framework using the Laplace-Beltrami Operator to detect, quantify, and visualize systematic demographic biases in 3D face reconstruction models, demonstrating that this geometric approach offers a more perceptually accurate evaluation of fairness than traditional Euclidean distance metrics.

Original authors: Veronika Shilova, Emmanuel Malherbe, Giovanni Palma, Panagiotis-Alexandros Bokaris, Laurent Risser, Jean-Michel Loubes

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Veronika Shilova, Emmanuel Malherbe, Giovanni Palma, Panagiotis-Alexandros Bokaris, Laurent Risser, Jean-Michel Loubes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a master sculptor who is famous for making incredibly realistic clay faces. This sculptor has spent years studying a specific group of people to learn how to shape clay. Now, imagine this sculptor tries to make a face for someone who looks very different from the people they studied.

According to this paper, the sculptor might get the general shape right, but they miss the tiny, unique details that make that person's face truly theirs. They might smooth out wrinkles that should be there, or flatten a nose that should have a specific curve.

Here is a breakdown of what the researchers discovered and how they found it, using simple analogies:

The Problem: The "One-Size-Fits-All" Mold

The paper focuses on 3D Face Reconstruction, which is the technology used to turn a flat photo into a 3D model of a face. Most of these systems use a "template" called a 3D Morphable Model (3DMM).

Think of a 3DMM like a master cookie cutter. It was made by studying a limited number of faces (the training data). Because the cookie cutter was made from a specific set of cookies, it works perfectly for those shapes. But if you try to use it to cut a cookie with a very different shape (like a different age, gender, or ethnicity), the cutter forces the dough into a shape that fits the mold, not the dough's natural form.

The researchers found that these digital cookie cutters have biases. They work well for the people they were trained on, but they "smooth out" or distort the faces of people who are older, or from different ethnic backgrounds.

The Old Way of Checking: Measuring with a Ruler

Previously, scientists checked if these 3D models were accurate by measuring the distance between the real face and the digital model using a simple ruler (Euclidean distance).

The Flaw: Imagine you have a bumpy road and a flat road. If you measure the distance between them at just a few points, they might look close. But if you drive a car over them, the bumpy one feels very different.
The old "ruler" method only checked the overall distance. It missed the texture and the curves. It couldn't tell the difference between a face that was slightly too big and a face that had lost all its unique wrinkles and folds.

The New Way: The "Curvature Compass"

The authors invented a new way to check the models, which they call a Curvature-Aware Framework.

Instead of just measuring distance, they used a mathematical tool called the Laplace-Beltrami Operator. Think of this as a sensitive compass that doesn't just measure how far apart two points are, but how curved the surface is at every single point.

How it works: It looks at the "hills and valleys" of the face. Is the nose bridge sharp? Are the cheekbones bumpy? Are the wrinkles around the eyes deep?
The Result: This new compass creates a "heat map" of errors. It can spot tiny, localized mistakes—like a nose that is too round or a forehead that is too flat—that the old ruler method completely ignored.

The Discovery: Who Gets the "Smoothed" Treatment?

Using this new compass, the researchers tested the models on a diverse group of people and found some clear patterns:

The Age Bias: As people get older, their faces develop more complex details (wrinkles, sagging, deeper folds). The 3D models, which are based on simple math, struggle with this complexity. They tend to "smooth out" older faces, making them look younger and less detailed than they really are. The researchers found a strong link: the older the person, the more the model failed to capture their true shape.
The Ethnicity Bias: When looking at people from different ethnic backgrounds (specifically African and Asian subjects in their study), the models tended to force their faces toward the "average" shape of the template.
- Example: For African subjects, the models often made the nose wings look unnatural or the nose tip too round and lifted, failing to capture the specific geometric nuances of their features.
The Gender Bias: They also found hints that the models treated men and women differently, particularly regarding how age affected the reconstruction of male faces in one of the models.

Why This Matters (According to the Paper)

The researchers didn't just find these errors; they proved that human eyes agree with their new compass.

They showed pairs of 3D reconstructions to real people and asked, "Which one looks more like the real person?"

The old "ruler" method was basically guessing (about 50% accuracy).
The new "curvature compass" method matched human judgment 73% of the time.

The Bottom Line

This paper argues that we cannot trust the old ways of measuring 3D face models anymore. If we want these tools to be fair and accurate for everyone—whether they are young, old, or from any background—we need to stop just measuring "distance" and start measuring "shape and curve."

The authors have made their code and data available so others can use this new "compass" to check if their own 3D face tools are biased, ensuring that future technology treats every face with the same geometric precision.

Technical Summary: Discovering Geometric Biases in 3D Face Reconstruction

Problem Statement

3D Morphable Models (3DMMs) serve as the standard parametric shape priors for state-of-the-art 3D face reconstruction algorithms. However, because these models are derived from finite training datasets, they inherit morphological biases that limit their generalizability across diverse global populations. Current evaluation metrics, which predominantly rely on global Euclidean distances (e.g., RMSE, NMSE), fail to capture subtle surface nuances such as local topology, undulations, and high-frequency geometric details. Consequently, these metrics often overlook systematic reconstruction failures that may disproportionately affect specific demographic groups (age, gender, ethnicity). This gap is critical given the increasing categorization of facial analysis as a high-risk AI application, necessitating rigorous accuracy metrics as mandated by regulations like the EU AI Act.

Methodology

The authors propose a Curvature-Aware Spectral Framework to analyze, quantify, and visualize geometric biases in 3D face reconstruction. The methodology consists of three primary components:

1. Curvature-Based Error Metric (CRE)

Instead of measuring point-to-point Euclidean distances, the framework utilizes Mean Curvature ( $H$ ), a second-order differential property, to characterize local surface geometry.

Laplace-Beltrami Operator (LBO): The authors estimate mean curvature using the LBO, defined as $H = -\frac{1}{2}\langle \vec{n}, \Delta \vec{p} \rangle$ . To handle piecewise linear triangular meshes, they employ a numerical estimation via local quadratic regressions. For each vertex, a quadratic bivariate surface is fitted to its $R$ -ring neighborhood to approximate the Laplacian, making the estimation robust to noise while preserving high-frequency details.
Error Mapping: A curvature-based error map $\epsilon_H$ is generated by computing the difference between the mean curvature of the ground-truth mesh ( $M$ ) and the reconstructed mesh ( $M'$ ) at corresponding points.
Global Metric: A scalar error metric $\mathcal{E}_{CRE}$ is derived from the $L_1$ norm of the error map, weighted by the barycentric area of vertices.

2. Spectral Decomposition and Bias Discovery

To identify systematic failure modes, the framework leverages the spectral properties of the LBO.

Shape Harmonics: The LBO eigenfunctions (shape harmonics) form a spectral basis tailored to the specific geometry of the 3DMM template. Lower eigenvalues correspond to global deformations, while higher eigenvalues capture local details.
Spectral Projection: Curvature error maps are projected into this spectral space, reducing dimensionality from the number of vertices ( $|V'|$ ) to a lower frequency count ( $K$ ).
Clustering: The projected spectral errors are clustered using K-means. This allows the identification of recurring patterns of geometric bias (failure modes) across a dataset of reconstructions. The cluster centroids are then reconstructed back into spatial error maps to visualize specific failure patterns.

3. Experimental Protocol

Dataset: The REALY benchmark is used, providing 100 globally aligned, high-fidelity 3D face scans with demographic labels (age, gender, ethnicity) and anatomical region masks (nose, mouth, cheek, forehead).
Models Evaluated: The study evaluates the intrinsic representativeness of the Basel Face Model (BFM) and FLAME 3DMMs. Additionally, it assesses state-of-the-art 2D-to-3D reconstruction algorithms (Deep3D, 3DDFA-v3, MGCNet, DECA, MICA) that utilize these priors.
Anatomical Segmentation: A Topological Voting Procedure is employed to map anatomical zones from the REALY dataset to the vertices of the 3DMM mean shape, ensuring consistent regional analysis.

Key Contributions

Curvature-Aware Evaluation Framework: Introduction of a novel framework using the Laplace-Beltrami Operator to quantify and visualize surface discrepancies that are invisible to standard Euclidean metrics.
Perceptual Validation: Demonstration through a user study that the proposed curvature-based error metric aligns significantly better with human qualitative judgment than traditional point-wise distances.
Systematic Bias Discovery: The first comprehensive study to expose morphological biases in 3D face reconstruction, revealing substantial fidelity gaps across demographic factors.

Results

Perceptual Validation

A user study involving 19 annotators compared the proposed CRE metric against standard NMSE (Normalized Mean Square Error) variants.

Performance: The CRE metric achieved a pairwise accuracy of 73.63% in matching human preferences, significantly outperforming NMSE-based benchmarks which hovered near random chance (approx. 49–53%).
Conclusion: Local surface character, captured by curvature, is a more salient indicator of reconstruction quality than global geometric distances.

Demographic Bias Analysis

Age Bias: A strong positive correlation was found between subject age and reconstruction error (Pearson $r \approx 0.77$ –$0.88$ for cheek regions in BFM/FLAME). As facial morphology becomes more complex with age (folds, sagging), the linear basis of 3DMMs fails to capture high-frequency details, effectively "smoothing out" structural markers. This bias is propagated from the 3D prior to the final 2D-to-3D reconstruction outputs.
Ethnicity Bias: Clustering of error maps revealed distinct systematic failure patterns for non-Caucasian populations.
- BFM: African and Asian subjects were concentrated in the first error cluster, characterized by unnatural creases on the nose wings and a rounded, drawn-up nose tip.
- FLAME: Similar concentration in the first and second error clusters.
- Interpretation: The linear subspaces of these 3DMMs lack the expressive capacity to capture ethnic-specific geometric nuances, leading to a systematic "flattening" of features toward the model mean.
Gender Bias: Preliminary evidence suggests gender-related biases, with age-related degradation notably intensified in males within the BFM basis.

Significance and Claims

The paper asserts that this work represents the first comprehensive evaluation of demographic bias in 3D face reconstruction. The authors argue that relying on Euclidean distances is insufficient for detecting subtle, localized geometric failures that have significant demographic implications.

By adopting curvature-aware evaluation protocols, the research highlights the necessity of moving beyond global fidelity metrics to ensure demographic fairness and geometric precision. The findings suggest that current 3DMMs and reconstruction algorithms systematically fail to represent older individuals and specific ethnic groups, a limitation that could have discriminatory impacts in high-risk applications. The proposed framework is method-agnostic and generalizes to any template-based 3D surface reconstruction, offering a robust tool for future fairness research.