🤖 AI

PriorProbe: Recovering Individual-Level Priors for Personalizing Neural Networks in Facial Expression Recognition

The paper introduces PriorProbe, a novel MCMC-based framework that successfully recovers fine-grained individual-level cognitive priors to significantly enhance the personalization and predictive accuracy of neural networks in facial expression recognition tasks.

Original authors: Haijiang Yan, Nick Chater, Adam Sanborn

Published 2026-02-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haijiang Yan, Nick Chater, Adam Sanborn

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to read human faces. The robot is already very good at this; it can look at a clear, happy smile and say, "That's happiness!" with 99% certainty.

But here's the problem: human faces aren't always clear. Sometimes a face is a mix of emotions, or the expression is very subtle. In these "fuzzy" moments, the robot gets confused. Why? Because the robot doesn't know your personal history or your specific way of seeing the world.

Think of it like this: If you see a shadow in the dark, you might think it's a friendly dog because you love dogs. Your neighbor, who is afraid of dogs, might think it's a monster. The shadow (the data) is the same for both of you, but your prior beliefs (what you expect to see) are totally different.

This paper introduces a new tool called PriorProbe to fix this. Here is how it works, broken down into simple steps:

1. The Problem: The Robot's "One-Size-Fits-All" Brain

Current AI models are trained on massive groups of people. They learn an "average" way of seeing faces. But humans aren't averages. When a face is ambiguous, the robot fails to predict what you specifically will think, because it doesn't know your unique "mental filter."

2. The Solution: PriorProbe (The "Mind-Reading" Game)

The researchers created a game to figure out exactly how you think about emotions, without changing the robot's brain.

The Setup: They built a special "Face Factory" (called DistFace). Imagine a machine that can mix and match faces. It can take the face of a stranger and put a "happy" expression on it, or a "scared" expression on it, without changing who the person is. This ensures the robot isn't getting confused by who the person is, only how they look.
The Game (The MCMC Loop): Instead of just asking you, "Is this face happy or sad?", the researchers play a back-and-forth game with you:
1. The Face Round: They show you a specific emotion (like "Anger") and ask, "Which of these two faces looks most like Anger?" You pick one.
2. The Category Round: They show you a face and ask, "Is this face 'Anger' or 'Sad'?" You pick one.
3. The Loop: They repeat this hundreds of times, using your answers to slowly "drill down" into your specific brain. It's like a detective narrowing down a suspect list by asking you specific questions until they know exactly what your mental map of emotions looks like.

3. The Result: Your Personal "Emotion Map"

By the end of the game, the researchers have a perfect map of your prior beliefs. They know exactly how likely you are to think a certain face is "happy" versus "sad," even before you see the face.

They call this your Individual-Level Prior.

4. The Test: Does it Help the Robot?

The researchers took these personal maps and gave them to a top-tier AI robot. They didn't retrain the robot or change its code; they just let the robot "consult" your map when it was unsure.

The Outcome: When the faces were clear, the robot did the same thing as before (no harm done). But when the faces were ambiguous (the tricky ones), the robot suddenly got much better at predicting your specific choice. It finally understood how you interpret the world, not just the average person.

The Big Takeaway

This paper shows that we don't need to rebuild AI from scratch to make it personal. Instead, we can use a clever game to "probe" what's inside your head, create a personal map of your biases, and hand that map to the AI. This makes the AI smarter at understanding you specifically, especially in those confusing moments where the answer isn't obvious.

In short: They built a way to ask your brain, "What do you expect to see?" and then taught the AI to listen to that answer, making the AI a much better personal assistant for reading faces.

Technical Summary: PriorProbe for Personalizing Neural Networks in Facial Expression Recognition

Problem Statement
Adapting artificial intelligence to individual users is critical for personalized decision-making, yet capturing individual-specific behavior remains a significant challenge. While state-of-the-art deep neural networks (DNNs) can predict human behavior on average, they struggle to capture individual perceptual judgments, particularly regarding subtle or ambiguous facial expressions where interpersonal variability is high.

Formally, facial expression recognition can be modeled as Bayesian inference: $P(e|f) \propto P(e)P(f|e)$ , where $P(e)$ is an individual's prior belief over emotional states and $P(f|e)$ is the likelihood of observing expression $f$ given emotion $e$ . When stimuli are ambiguous, the likelihood term becomes uninformative, and individual differences in priors $P(e)$ dominate the inference. Existing methods for personalizing DNNs face two fundamental hurdles:

Data Scarcity: Training on individual-level data is often costly and infeasible due to the need for large, representative datasets.
Model Degradation: Fine-tuning DNNs on sparse, noisy individual data risks overfitting and catastrophic forgetting of learned representations.

Furthermore, traditional cognitive science methods for eliciting priors often fail to uniquely identify them or introduce systematic biases by treating the likelihood function as a free parameter or an arbitrary mapping.

Methodology
The authors introduce PriorProbe, a novel elicitation approach grounded in Markov Chain Monte Carlo with People (MCMCP). Unlike parameter-level personalization (e.g., fine-tuning weights), PriorProbe recovers fine-grained, individual-specific cognitive priors and likelihoods to inform the inference space of a neural network without modifying its weights.

The methodology consists of three core components:

DistFace (Disentangled Affect-Identity Space):
To enable unbiased sampling, the authors constructed a controlled generative space where facial affect is disentangled from facial identity.
- Architecture: A hybrid Variational Auto-Encoder (VAE) comprising:
  - Encoder: Three pretrained branches extract features: ResEmoteNet (expression), FMAE-IAT (Action Units, identity-adversarial), and ArcFace (identity).
  - Latent Space: These features are mapped into distinct sub-latent spaces ( $V_{af}$ for affect, $V_{id}$ for identity).
  - Denoising: A denoising auto-encoder compresses the 128-dimensional affect space into a 16-dimensional pure affect space for efficient sampling.
- Training: The model was trained on the BU-4DFE dataset using reconstruction, identity-preservation, and affect-preservation losses to ensure the affect branch drives expression while the identity branch drives identity.
PriorProbe Elicitation Procedure:
PriorProbe operates within the 16-dimensional affect space using a block Metropolis–Hastings algorithm with a human-in-the-loop. It alternates between two trial types to jointly sample from an individual's likelihood $P(f|e)$ and prior $P(e)$ :
- Face Trials: Participants are shown an emotion category $e$ and two face options ( $f, f'$ ). They select the face best representing the category. To accelerate sampling, an Independent Metropolis–Hastings (IMH) scheme is used where a "Gatekeeper" (ResEmoteNet) proposes faces likely to belong to the target category.
- Categorization Trials: Participants are shown a face $f$ and two emotion categories ( $e, e'$ ). They select the best description. The proposal distribution for categories is also guided by the Gatekeeper.
- Recovery: The resulting Markov chain converges on the joint distribution $\pi(f, e)$ . Individual priors $P(e)$ are recovered by reweighting the sampled categories by $1/G_e(e|f)$ , where $G_e$ is the Gatekeeper's probability assignment, effectively correcting for the proposal bias.
Integration and Evaluation:
The recovered individual priors are integrated with a state-of-the-art DNN (ResEmoteNet) within the inference space. The system is evaluated on a task of classifying novel, ambiguous stimuli (images where the DNN's confidence is $<0.25$ ). The goal is to determine if incorporating individual priors improves the model's alignment with human classification on ambiguous stimuli without degrading performance on ground-truth labels.

Key Contributions

PriorProbe Framework: A general, interpretable method for recovering fine-grained, individual-specific mental representations (priors and likelihoods) efficiently, avoiding the need for massive individual datasets or weight modification.
DistFace Dataset/Model: A generative model that disentangles facial affect from identity, providing a "pure affect space" essential for unbiased prior elicitation in facial expression tasks.
Empirical Validation: A comprehensive human experiment demonstrating that neural networks informed by recovered priors better predict individual choices on ambiguous stimuli compared to the network alone or networks using alternative prior sources.

Results
In an experiment with 55 participants, the authors recovered individual priors over seven basic emotions.

Individual Differences: The recovered priors exhibited substantial heterogeneity across participants, confirming that individuals possess distinct cognitive biases regarding emotional categories.
Performance Gains: Integrating PriorProbe-derived priors into the neural network yielded substantial performance gains in predicting individual classifications on ambiguous stimuli.
Robustness: The personalization improved alignment with individual human judgments without degrading the model's inference performance on unambiguous (ground-truth) stimuli.

Significance and Claims
The paper claims to provide a general, interpretable, and robust framework for personalizing deep neural networks. By shifting the focus from parameter-level adaptation to the recovery of cognitive priors, PriorProbe addresses the limitations of data scarcity and catastrophic forgetting. The authors assert that this approach offers a principled route to developing digital systems that deliver genuinely personalized assistance, particularly in domains like facial expression recognition where individual perceptual differences are significant and standard models fail to capture them. The work demonstrates that individual-level cognitive priors can be accurately elicited and effectively utilized to enhance the predictive power of AI systems for specific users.