AutoFFS: Adversarial Deformations for Facial Feminization Surgery Planning

The paper introduces AutoFFS, a novel data-driven framework that utilizes adversarial free-form deformations to generate quantitative, counterfactual skull morphologies for objective and reproducible preoperative planning in Facial Feminization Surgery.

The Gist

The Big Picture: What is this paper about?

Imagine you are a sculptor trying to reshape a block of stone. In the real world, Facial Feminization Surgery (FFS) is exactly that: a surgeon reshapes the bones of a transgender woman's face to make them look more feminine.

Currently, surgeons have to guess how much to shave off the forehead, how to round the jaw, or how to change the nose. They rely on their experience and "eyeballing" it. It's like trying to paint a portrait without a reference photo.

AutoFFS is a new computer program that acts like a "What-If" simulator. It answers the question: "If this specific skull belonged to a woman, what would it look like?" It doesn't just guess; it mathematically calculates the exact bone changes needed to transform a "masculine" skull shape into a "feminine" one, giving surgeons a precise 3D blueprint before they ever pick up a scalpel.

---

How Does It Work? (The Magic Trick)

The researchers used a clever mix of AI and digital deformation. Here is the step-by-step process using a simple analogy:

1. The "Judges" (The Classifiers)

First, the team trained a panel of AI "judges" (neural networks) to look at skull scans and say, "That's a man" or "That's a woman."

• The Analogy: Imagine you have a panel of 8 different art critics. They have all studied thousands of faces and are very good at spotting the subtle differences between male and female bone structures (like a heavy brow ridge or a square jaw).

2. The "Digital Clay" (Free-Form Deformation)

The computer takes a 3D scan of a patient's skull. Instead of cutting the bone digitally, it treats the skull like a soft, digital lump of clay.

• The Analogy: Imagine the skull is wrapped in a invisible 3D grid of rubber bands. The computer can pull, push, and stretch these rubber bands to warp the shape of the skull without breaking it.

3. The "Adversarial Attack" (The Game of Cat and Mouse)

This is the core of the method. The computer wants to warp the skull until the "Judges" (the AI panel) are tricked into thinking it is a female skull.

• The Analogy: Think of it like a game of whack-a-mole or a chameleon.
  • The computer nudges the digital clay (pulls the jaw back, rounds the forehead).
  • It asks the Judges: "Is this a woman yet?"
  • The Judges say: "No, still looks a bit like a man."
  • The computer nudges the clay again, but this time in a slightly different way.
  • It repeats this thousands of times, getting smarter with every try, until the Judges are 100% convinced, "Yes, this is definitely a woman!"

4. The "Smooth Operator" (Regularization)

If the computer just tried to trick the judges as fast as possible, it might warp the skull into a weird, alien shape (like melting the face like a Salvador Dalí painting).

• The Analogy: To stop this, the researchers added "physics rules" (called regularization). It's like telling the digital clay: "You can stretch, but you must stay smooth. You can't turn the nose into a spiral." This ensures the changes look like natural human bone growth, not a glitchy video game error.

---

What Did They Find?

The team tested this on real medical scans (from patients with Multiple Sclerosis, where the skull shape is normal, but the brain has issues—so the skull data is safe to use).

1. The AI was fooled: When they ran the "male" skulls through the system, the AI judges started seeing them as "female" with high confidence.
2. Humans were fooled too: They showed the results to 11 human volunteers.
   • When shown real skulls, humans guessed the gender correctly about 81% of the time.
   • When shown the transformed skulls, humans guessed the new gender correctly about 63% of the time.
   • The Takeaway: The computer successfully changed the "vibe" of the face. Even though it only changed the front part of the face (forehead, chin, cheekbones), humans could clearly tell the difference.

Why Does This Matter?

• For Surgeons: It moves surgery from "guessing" to "precision." Instead of saying, "I think I should shave 3mm off the forehead," the computer says, "Here is the exact 3D map of how the bone needs to move to achieve the desired look."
• For Patients: It offers a clearer path to gender affirmation. It helps visualize the outcome before the surgery happens, reducing anxiety and improving results.
• For Science: It proves that we can use "adversarial attacks" (usually used to trick AI) for something good: helping people feel more comfortable in their own bodies.

In a Nutshell

AutoFFS is a digital time machine for your face. It takes a skull, asks a team of AI experts what a female version of that skull would look like, and then gently reshapes the digital bone until it matches that vision. It turns a complex, subjective surgical planning process into a data-driven, objective science.

Technical Summary

1. Problem Statement

Facial Feminization Surgery (FFS) is a critical procedure for gender affirmation, aiming to reshape craniofacial structures to align with a female morphology. However, current preoperative planning relies heavily on subjective clinical assessment and surgeon experience.

• Limitations: Existing methods lack quantitative, reproducible anatomical guidance. They often focus on isolated regions (e.g., just the mandible) rather than global morphological interdependencies across the entire skull.
• Goal: The authors propose a data-driven framework to answer the counterfactual question: "What would this skull look like if it had the opposite sex?" to provide objective, personalized surgical planning.

2. Methodology: AutoFFS

The proposed framework, AutoFFS, generates counterfactual skull morphologies by performing a deformation-based targeted adversarial attack on an ensemble of pre-trained binary sex classifiers.

A. Core Concept

Instead of generating new images from scratch (like GANs), the method takes an existing skull scan ($X$) and applies a spatial deformation field ($\Phi$) to transform it into a target sex ($y_{target}$). The deformation is optimized such that the transformed skull ($X'$) is misclassified by the sex classifiers as the target sex.

B. Technical Components

1. Classifier Ensemble:

   • Multiple pre-trained binary sex classifiers (ResNets and SE-ResNets of varying depths) are trained on a dataset of skull MR scans.
   • Ensembling: Using multiple architectures ensures the generated deformations are robust and capture population-level sexual dimorphism rather than overfitting to a single model's bias.

2. Free-Form Deformation (FFD):

   • The deformation field is parameterized as a 3D cubic B-spline.
   • A regular control lattice ($n_x \times n_y \times n_z$) is defined over the scan domain.
   • Control points have associated offset vectors ($\Delta P$) that define the displacement.
   • The continuous displacement field is interpolated using cubic B-spline basis functions, ensuring smooth, differentiable transformations.

3. Adversarial Optimization:

   • Objective: Optimize the control point offsets to maximize the probability of the target class label while minimizing a loss function.
   • Loss Function: A Smooth Worst-Case Margin (SWC) loss is used. It approximates the worst-case logit across the ensemble (using a log-sum-exp operator) to ensure the deformation fools all classifiers, not just the easiest one.
   • Symmetry: To maintain anatomical realism, the method applies the classifier to both the deformed image and its midsagittal mirror image.

4. Regularization:
   To prevent unrealistic, noisy, or non-smooth deformations, two regularization terms are added to the loss:

   • Smoothness Regularizer: Penalizes the Jacobian of the displacement field (preventing folding or extreme stretching).
   • Bending Energy Regularizer: Penalizes the Hessian of the displacement field (encouraging smooth curvature changes).
   • Constraint: Deformations are restricted to the anterior half of the skull (facial features) by fixing control points in the posterior region to zero.

C. Algorithm Flow

1. Initialize control point offsets to zero (identity transform).
2. Compute displacement field via B-spline interpolation.
3. Apply deformation to the input skull.
4. Pass the deformed skull (and its mirror) through the classifier ensemble.
5. Calculate the SWM loss and regularization terms.
6. Backpropagate gradients to update control point offsets (using Adam optimizer).
7. Repeat for a set number of iterations.

3. Experimental Setup

• Dataset: 444 MR scans from Multiple Sclerosis (MS) patients (152 male, 292 female). MS lesions are confined to the brain/spinal cord, leaving cranial bone morphology unaffected, making this a valid surrogate for skull shape analysis.
• Preprocessing: Bone segmentation, reorientation to RAS+, and cropping to the frontal half ($256 \times 256 \times 128$).
• Training: Classifiers trained with data augmentation (flipping, affine transforms) and masking strategies.
• Evaluation:
  • Classifier Performance: Hold-out test set accuracy, F1, and AUROC.
  • System Performance: Measuring class probability shifts after transformation.
  • Human Perceptual Study: 11 raters classified 80 skulls (20 real male, 20 real female, 20 transformed male-to-female, 20 transformed female-to-male).

4. Key Results

• Classifier Performance: The ensemble of classifiers achieved high accuracy (up to 93.9% for ResNet34) and AUROC scores consistently above 0.93, confirming robust learning of sexual dimorphism.
• Transformation Efficacy:
  • Quantitative: After transformation, the probability of the target sex label increased significantly. For male-to-female transformations, the mean class probability shifted from ~0.24 to 0.76.
  • Ensembling Impact: Using the ensemble strategy resulted in more consistent classification shifts compared to using a single best classifier.
  • Qualitative: Visualizations show deformations concentrated in known dimorphic regions: chin, brow ridges, forehead, and cheekbones.
  • Necessity of Regularization: Without B-spline formulation and regularization, the method produced noise-like perturbations, confirming the necessity of smoothness constraints.
• Human Evaluation:
  • Raters correctly identified real skulls with 81% accuracy.
  • For transformed skulls, raters perceived the intended target sex in ~63% of cases (a 44% improvement over chance), demonstrating a systematic perceptual shift.
  • Inter-rater agreement was lower for transformed skulls, reflecting the partial modification of features (only anterior face changed).

5. Key Contributions

1. Novel Framework (AutoFFS): A data-driven approach for FFS planning that generates counterfactual skull morphologies using adversarial deformations rather than generative synthesis.
2. Objective Guidance: Moves surgical planning from subjective assessment to quantitative, reproducible anatomical guidance.
3. Robustness via Ensembling: Demonstrates that attacking an ensemble of diverse classifiers yields more reliable and generalizable morphological changes.
4. Validation: Provides both quantitative (classifier confidence) and qualitative (human perceptual study) evidence that the generated morphologies exhibit target sex characteristics.

6. Significance and Future Work

• Clinical Impact: This work addresses a largely overlooked patient group (transgender and gender-diverse individuals) by providing a tool to visualize potential surgical outcomes before surgery.
• Generalizability: While focused on FFS, the method is applicable to Facial Masculinization Surgery (FMS).
• Future Directions:
  • Building a large-scale, diverse database covering various ethnicities and ages.
  • Integrating generated morphologies into clinical workflows.
  • Deriving specific surgical cutting guides directly from the counterfactual deformations.

The paper concludes that data-driven planning for FFS is a viable and promising research direction with meaningful clinical potential.