Face-selective responses correlate with deep networks that learn from environment feedback

This study demonstrates that a reinforcement learning model, which learns face perception through environmental feedback rather than ground-truth labels, accounts for human neural responses in face-selective regions as effectively as traditional supervised and unsupervised deep learning models.

The Gist

The Big Question: How Does the Brain Learn to Recognize Faces?

Imagine your brain is a massive, high-tech security camera system. Its job is to look at faces and figure out who they are, how they feel, and whether they are friends or foes.

For a long time, scientists tried to build computer programs (Deep Neural Networks) to mimic this system. They usually taught these computers in two ways:

1. The "Teacher" Method (Supervised Learning): The computer looks at a photo, and a human teacher says, "That's John," or "That's Mary." The computer memorizes the labels.
2. The "Self-Taught" Method (Unsupervised Learning): The computer looks at thousands of photos and tries to guess what they look like or group them by similarity without any labels.

The Problem: In real life, we don't have a teacher standing next to us whispering names into our ears every time we see a stranger. And we don't just look at pictures; we interact with people. We learn that Person A is nice (give them a smile, get a smile back) and Person B is grumpy (avoid them, get ignored).

This paper asks: What if we taught the computer to learn faces the way humans do—by interacting with the world and getting feedback?

The Experiment: A Digital "Approach or Avoid" Game

The researchers built a new type of computer model called a Reinforcement Learning (RL) model. Think of this model as a digital robot living in a video game.

• The Setup: The robot sees a face.
• The Choice: It has to decide: "Do I approach this person, or do I run away?"
• The Feedback:
  • If it approaches a "nice" person, it gets points (a reward).
  • If it approaches a "mean" person, it gets no points or a penalty.
  • If it avoids a "mean" person, it avoids the penalty.

The robot's only goal is to maximize its points. It has to learn which faces are safe to approach and which are dangerous, purely through trial and error, just like a baby learning who to trust.

The Test: Does the Robot Think Like a Human Brain?

To see if this robot was thinking like a human, the researchers compared its "brain" to the actual brains of 10 human patients.

• The Human Data: These patients had tiny electrodes implanted in their brains (for medical reasons) to monitor seizures. While the patients looked at pictures of faces, the electrodes recorded the electrical activity of their brain cells.
• The Comparison: The researchers used a tool called Representational Dissimilarity Matrices (RDMs).
  • The Analogy: Imagine you have a map of how similar different faces feel to your brain. If you see a picture of your mom and your sister, your brain says, "These are very similar." If you see a stranger, it says, "This is very different."
  • The researchers made these "similarity maps" for the human brains and for the computer robots. Then, they checked if the maps matched.

The Results: The Robot Caught Up!

Here is what they found:

1. The "Teacher" Robot (Supervised) and the "Self-Taught" Robot (Unsupervised) both did a great job matching the human brain maps. This we already knew.
2. The "Interactive" Robot (Reinforcement Learning) was the surprise.
   • When the robot used a standard computer architecture (called ResNet), it didn't do as well as the others. It was like a student who studied hard but didn't quite get the test.
   • However, when they gave the robot a smarter, more complex brain architecture (called VIB DenseNet), it performed just as well as the other robots.

The Takeaway: A computer that learns by interacting with the environment (getting rewards and punishments) can build a mental map of faces that is just as accurate as a computer taught by a human teacher.

Why Does This Matter? (The "Aha!" Moment)

This is a big deal because it suggests that our brains might not just be passive cameras.

• Old View: Our brains just take pictures and label them.
• New View: Our brains are shaped by our actions. We learn who is a friend or foe because we do things (approach or avoid) and see what happens. The "reward" we get from the world sculpts our brain's understanding of faces.

The study also found that the architecture (the physical design of the computer's brain) matters just as much as the task (what it's trying to learn). It's like saying: "You can be a great chef, but if you are cooking in a broken kitchen, you won't make a good meal. You need the right tools and the right recipe."

Summary in One Sentence

By teaching a computer to learn faces by "playing a game" of approaching friends and avoiding foes, the researchers proved that learning from real-world feedback is just as powerful as learning from a textbook, and it helps us understand how our own brains build social connections.

Technical Summary

1. Problem Statement

Deep neural networks (DNNs) have successfully modeled neural responses in the visual system, but existing approaches face two primary limitations regarding biological plausibility:

• Supervised Learning (SUP): Most models rely on ground-truth labels (e.g., identity classification) that are unavailable to human observers in real-world settings.
• Unsupervised Learning (UNSUP): While these models avoid ground-truth labels, they typically ignore the role of environmental feedback in shaping visual representations. Biological evidence suggests that visual representations are tuned by the behavioral tasks an animal performs and the rewards/punishments received from the environment (e.g., approach-avoidance behaviors).

The authors aim to bridge this gap by developing a Reinforcement Learning (RL) model of face perception that learns through environmental feedback (simulating social interactions) and evaluating its ability to account for human neural responses compared to standard SUP and UNSUP models.

2. Methodology

A. Neural Data Collection

• Participants: 11 patients undergoing intracranial electroencephalography (iEEG) for seizure localization (10 included in final analysis after excluding one for noisy data).
• Stimuli: Images from the Karolinska Directed Emotional Faces (KDEF) dataset, varying in identity, expression, and viewpoint.
• Task: Participants performed a gender classification task while face-selective electrodes were identified using a functional localizer.
• Data Processing: Local field potentials (LFP) were recorded from 24 face-selective electrodes (12 in the ventral stream, 8 in the lateral stream, 4 "other"). Representational Dissimilarity Matrices (RDMs) were computed for three temporal windows (125–175ms, 175–225ms, 225–275ms).

B. Computational Models

The study trained six Deep Convolutional Neural Networks (DCNNs) using the CelebA dataset. The models were categorized by Learning Mechanism and Encoder Architecture:

1. Learning Mechanisms:

   • SUP (Supervised): Trained to classify face identity (ground-truth labels).
   • UNSUP (Unsupervised): Trained to reconstruct the input image (Variational Autoencoder objective).
   • RL (Reinforcement Learning): Trained in a simulated environment to maximize reward. The model received a face image, predicted the expected reward of interacting with that identity, and chose to "approach" or "avoid."
     • Reward Structure: Each identity had an associated Gaussian reward distribution $N(\mu_i, \sigma_i)$.
     • Loss Function: Combined prediction error (if interacting) and "opportunity cost" (if avoiding), with an optimistic initial value to encourage exploration.

2. Encoder Architectures:

   • ResNet-18: A standard residual network widely used in neuroscience.
   • VIB DenseNet: A hybrid architecture combining DenseNet (for feature reuse and gradient flow) and Variational Information Bottleneck (VIB) (introducing stochasticity via a probabilistic bottleneck and KL-divergence regularization).

3. Multi-Task Model:

   • VIB UNSUP+RL: A model using the VIB DenseNet encoder with two decoders: one for image reconstruction and one for reward prediction, trained to minimize the sum of both losses.

C. Analysis

• Representational Similarity Analysis (RSA): The study compared the RDMs of the neural data with the RDMs of the model's internal representations (extracted from the final encoder layer).
• Metric: Kendall's $\tau$ rank correlation coefficient was used to measure the similarity between neural and model RDMs.
• Comparisons:
  • Model vs. Neural data across time windows.
  • Model vs. Model (to assess representational geometry differences).
  • Logistic regression to distinguish between ventral and lateral electrodes based on model correspondence patterns.
  • Semi-partial Kendall $\tau$ to isolate unique variance explained by each model.

3. Key Contributions

1. RL Modeling of Face Perception: Introduced a novel RL framework where agents learn face representations through approach-avoidance interactions rather than static labels or pure reconstruction.
2. Architecture-Task Interaction: Demonstrated that the success of RL models in matching neural data is heavily dependent on the encoder architecture. Specifically, the VIB DenseNet architecture was crucial for RL models to perform on par with supervised models.
3. Multi-Task Integration: Explored the feasibility of combining unsupervised reconstruction and RL objectives in a single network, showing that while performance on individual tasks drops, the combined model still captures neural variance effectively.
4. Ventral vs. Lateral Stream Differentiation: Showed that the pattern of correspondence between models and neural data can distinguish between ventral and lateral temporal regions, suggesting these streams may differ in how they are shaped by environmental feedback.

4. Key Results

• RL Performance: The RL models successfully learned to maximize rewards, significantly increasing their average reward scores post-training.
• Neural Correlation (ResNet): When using the standard ResNet-18 encoder, the SUP model correlated significantly better with neural responses than the UNSUP and RL models. The RL model showed the lowest correlation.
• Neural Correlation (VIB DenseNet): When using the VIB DenseNet encoder, the RL model achieved correlations comparable to both SUP and UNSUP models. There was no significant difference in performance between the three learning mechanisms with this architecture.
  • Statistical Significance: The VIB RL model showed significantly higher correlations with neural RDMs than the ResNet RL model ($p = 0.030$).
• Temporal Dynamics: The highest correspondence between all models and neural data occurred in the 125–175ms time window.
• Ventral vs. Lateral Streams:
  • Logistic regression could distinguish ventral from lateral electrodes with 75% accuracy based on the 18-dimensional pattern of model correlations.
  • Ventral electrodes generally showed higher correspondence with all models, particularly in early time windows.
  • The study hypothesizes that the lower correspondence with lateral electrodes is due to the use of static images in training, whereas lateral regions are known to be more responsive to dynamic stimuli.
• Unique Variance: Semi-partial analysis revealed that SUP models and VIB DenseNet architectures explained unique neural variance not captured by other models.
• Combined Model: The VIB UNSUP+RL model performed slightly worse on individual tasks (reconstruction and reward) compared to single-task models but maintained a level of neural correspondence equivalent to the single-task VIB models.

5. Significance

• Biological Plausibility: The findings suggest that reinforcement learning models, which learn from environmental feedback rather than static labels, are a viable and promising approach for understanding naturalistic face perception.
• Role of Architecture: The study highlights that the "learning objective" (task) and the "encoder architecture" jointly shape representational geometry. Specifically, the probabilistic nature of the VIB DenseNet (stochasticity and regularization) appears essential for RL models to align with biological neural representations.
• Future Directions: The results imply that future models aiming to simulate human vision should incorporate environmental feedback mechanisms. Furthermore, the discrepancy in lateral stream modeling suggests that future models should utilize dynamic stimuli (video) rather than static images to fully capture the functional roles of different visual streams.

In conclusion, this paper demonstrates that deep networks trained via reinforcement learning can account for face-selective neural responses as effectively as supervised models, provided they utilize an architecture (VIB DenseNet) that supports the necessary representational flexibility and stochasticity.