Visual Emotion Perception in a Deep Neural Network Model with Both Bottom-Up and Top-Down Connections

This paper introduces EmoFB, a biologically inspired deep neural network model that integrates intrinsic affective appraisal and external contextual steering to demonstrate how top-down feedback dynamically reshapes visual representations, improves recognition under ambiguity, and enhances alignment with human brain activity across key emotional and visual processing regions.

The Gist

Imagine your brain isn't just a camera that passively takes pictures of the world. Instead, think of it as a smart detective who is constantly guessing what they are seeing based on how they feel and what they are looking for.

This paper is about teaching a computer (a "Deep Neural Network") to act like that detective, specifically when it comes to emotions.

Here is the simple breakdown of what the researchers did and why it matters:

1. The Problem: The "Robot" vs. The "Human"

Most computer vision models today work like a robotic cashier. You hand them an item (a picture), they scan the barcode (process the image), and tell you what it is. They do this in a straight line: Input → Processing → Output. They don't care if you are happy, scared, or looking for something specific.

But humans are different. If you are scared, a shadow in the corner might look like a monster. If you are happy, that same shadow might look like a friendly dog. Our feelings and our goals change how we see the world. This is called emotional perception, and until now, computers haven't been very good at simulating this "top-down" influence (where your brain tells your eyes what to look for).

2. The Solution: Meet "EmoFB"

The researchers built a new AI model called EmoFB. Think of EmoFB not as a robot, but as a team of two people working together:

• The Eyes (Visual System): This part sees the raw picture.
• The Heart & Mind (Affective System): This part feels the emotion and knows the goal.

These two parts talk to each other using two special "walkie-talkies" (feedback signals):

1. Intrinsic Feedback (The Gut Feeling): This is the model's own emotional reaction to what it sees. "Oh, that looks scary!" It sends a signal back to the eyes to say, "Be careful, look closer at the scary parts."
2. External Steering (The Mission Briefing): This is like a boss giving instructions. "We are looking for a cat, not a dog." It tells the model, "Ignore the dog, focus on the cat."

3. The Experiment: The "Blurry Room" Test

The researchers tested EmoFB in three different scenarios, ranging from easy to very confusing:

• Single Image: A clear picture.
• Side-by-Side: Two pictures next to each other.
• Overlay: A messy picture where two images are smashed on top of each other (very hard to see).

The Result:
When the model had the "External Steering" (the mission briefing), it got much better at finding things, even in the messy, blurry pictures. It didn't just guess better; it actually reorganized its brain. It started grouping similar things together more clearly, just like how a human expert organizes their filing cabinet.

4. The "Aha!" Moment: It Thinks Like Us

The coolest part of the study is that when they compared EmoFB's "brain activity" to real human brain scans (fMRI), they found a perfect match.

• When the model used its emotional feedback, its internal wiring looked just like the wiring in the human visual cortex (where we see) and the amygdala (where we feel fear and emotion).
• It proved that by adding these "top-down" emotional signals, the machine started seeing the world the way humans do.

The Big Takeaway

This paper bridges the gap between Artificial Intelligence and Human Emotion.

It shows that to build truly smart machines, we can't just give them better cameras. We have to give them a "gut feeling" and the ability to listen to their own goals. By doing this, we not only make better AI, but we also learn how our own brains use emotions to help us see the world clearly.

In short: The paper teaches a computer to stop just "seeing" and start "feeling" its way through a picture, making it smarter and more human-like.

Technical Summary

Based on the abstract provided, here is a detailed technical summary of the paper "Visual Emotion Perception in a Deep Neural Network Model with Both Bottom-Up and Top-Down Connections":

1. Problem Statement

The core problem addressed is the limited understanding of the computational mechanisms through which distinct affective signals (emotions) modulate visual processing. While it is well-established in neuroscience that emotion reshapes perception via top-down feedback, most existing computational models treat emotional content perception as a static, feedforward task. These models fail to capture the dynamic interplay between:

• Internal affective states.
• External goals (contextual priors).
• Sensory input.

Consequently, there is a lack of mechanistic insight into how these top-down signals guide emotional perception in a way that aligns with biological systems.

2. Methodology: The EmoFB Model

The authors introduce EmoFB, a biologically inspired Deep Neural Network (DNN) designed to simulate emotional perception. The methodology distinguishes itself through the following architectural and experimental features:

• Architecture: The model integrates a dedicated affective system with a standard visual processing hierarchy.
• Dual Feedback Mechanisms: Unlike standard feedforward networks, EmoFB utilizes two functionally distinct top-down feedback signals:
  1. Intrinsic Feedback: Generated by the model's own affective appraisal of the perceptual input (self-generated emotional state).
  2. External Steering: Conveys contextual priors, such as task expectations or specific target categories, to guide processing.
• Experimental Design: The model was evaluated across three tasks designed to vary in perceptual ambiguity:
  1. Single Image: Standard recognition.
  2. Side-by-Side: Comparing two images.
  3. Overlay: Highly ambiguous conditions where images are superimposed.

3. Key Contributions

• Novel Framework: The proposal of EmoFB as a computational framework that explicitly models the dynamic loop between affective appraisal and visual processing, moving beyond static feedforward assumptions.
• Mechanistic Differentiation: The separation of top-down signals into intrinsic (internal state) and external (contextual) components, allowing for the isolation of their specific effects on perception.
• Neuro-AI Bridge: The creation of a model that directly bridges affective neuroscience and artificial intelligence, providing a testbed for neurocognitive theories of emotion appraisal.

4. Key Results

• Dominance of External Steering: The study found that external steering exerted the strongest influence on the model's performance. It significantly improved recognition accuracy, particularly under the most challenging conditions (high perceptual ambiguity).
• Representation Restructuring: Top-down feedback did not merely improve accuracy; it fundamentally restructured internal representations. Specifically, it sharpened category-specific clustering within the feature space, making the model's internal encoding of emotional categories more distinct.
• Neurobiological Alignment: Crucially, the inclusion of top-down feedback increased the representational similarity between the model and the human brain. The model's activity patterns showed stronger alignment with human fMRI responses across:
  • Early visual cortex.
  • Ventral visual areas.
  • The amygdala (a key structure in emotional processing).

5. Significance

This work provides a critical computational framework for testing theories regarding how emotional signals shape perception. By demonstrating that top-down feedback can restructure feature spaces and align artificial representations with biological data (fMRI), the paper offers a mechanistic explanation for emotional perception. It suggests that the brain's ability to perceive emotion is not just a result of bottom-up sensory input but is actively constructed through dynamic feedback loops involving internal states and external context. This advances the field of AI by creating more brain-like models and aids neuroscience by offering a simulatable hypothesis for the role of the amygdala and feedback pathways in visual processing.