Layer-specific spatiotemporal dynamics of feedforward and feedback in human visual object perception

By combining layer-specific fMRI and EEG recordings with deep neural network modeling, this study reveals that human visual object perception involves early feedforward signals encoding medium-to-high complexity features in the middle layers of visual cortex, followed by later feedback signals in superficial layers that refine representations to high complexity.

The Gist

The Big Picture: How We See Things

Imagine your brain is a massive, high-tech factory dedicated to recognizing objects. When you look at a coffee mug, a dog, or a car, information doesn't just sit still; it travels through the factory in two main directions:

1. The "Up" Conveyor Belt (Feedforward): This is the initial rush of raw data. It goes from the back of your eye (Early Visual Cortex) up to the higher-level processing areas (Lateral Occipital Complex or LOC) to figure out, "What is this?"
2. The "Down" Manager's Memo (Feedback): Once the factory starts guessing what the object is, it sends instructions back down to the beginning to say, "Hey, look closer at the handle," or "That's not a dog, it's a cat."

The problem for scientists has always been that these two processes happen incredibly fast and overlap like two people talking over each other in a crowded room. It's hard to tell who said what and when.

The New Tool: A Layered Cake with a Time Machine

This study used a special combination of tools to solve this puzzle:

• 7T MRI (The Microscope): They used a super-powerful MRI scanner that can see the brain in "slices" like a layered cake. Instead of just looking at the whole brain, they could look at the top layer (superficial), the middle layer, and the bottom layer (deep) of the brain's cortex.
• EEG (The Stopwatch): They used EEG (electrodes on the scalp) to measure brain activity with millisecond precision, acting like a high-speed stopwatch.
• AI (The Translator): They used a Deep Neural Network (an AI trained to recognize objects) to act as a dictionary. They compared the brain's activity to the AI's "layers" to see how complex the information was.

The Discovery: A Relay Race with a Twist

The researchers watched how the brain processed 24 different pictures of everyday objects. Here is what they found, broken down by location:

1. The Early Station (EVC - The Back of the Brain)

Think of this as the Receiving Dock.

• What happened: When the image appeared, the signal hit the middle layer of this area first (about 100 milliseconds later).
• The Twist: Shortly after, the signal appeared in the top and bottom layers.
• The Meaning: This looked like a standard relay race. The middle layer got the message first, then passed it around. However, the researchers couldn't definitively say if the "Manager" (feedback) was talking back here yet. It mostly looked like the initial "Up" conveyor belt doing its job.

2. The High-Level Factory (LOC - The Side of the Brain)

Think of this as the Quality Control & Design Center. This is where the brain decides, "Yes, that is a coffee mug."

• The First Wave (Feedforward): The signal hit the middle layer first (around 160ms).
  • What it meant: The brain was recognizing "medium-complexity" features. It knew it was seeing a "round object with a handle," but maybe not the specific brand yet.
• The Second Wave (Feedback): A huge gap of time passed, and then a new signal appeared in the top layer (around 400ms).
  • What it meant: This was the "Manager" coming back. The top layer started processing high-complexity features. It wasn't just seeing a "round object"; it was now seeing "a ceramic mug with a specific logo."

The "Aha!" Moment: Why the Layers Matter

The most exciting part of the study is what the "Top Layer" in the High-Level Factory was doing.

Imagine you are trying to identify a blurry photo of a friend.

1. Feedforward (Middle Layer): Your brain says, "It's a person with brown hair." (Medium complexity).
2. Feedback (Top Layer): Your brain sends a signal back and says, "Wait, look at the glasses. That's your friend, Dave!" (High complexity).

The study found that the top layer of the brain is where this "extra complexity" happens. The feedback signal doesn't just turn up the volume on the old signal; it actually adds new, detailed information that wasn't there in the first rush of data.

The Takeaway

This paper is like finally getting a blueprint of a factory that was previously a black box.

• Before: We knew information went up and down, but it was a blur.
• Now: We know that the "Up" signal (Feedforward) starts in the middle of the brain's layers and carries basic, medium-level info.
• The "Down" signal (Feedback) arrives later and lands specifically in the top layers, bringing in high-level, detailed context that helps us truly understand what we are seeing.

It's as if the brain has a specific "meeting room" (the top layer) where the final, detailed decisions are made after the initial rush of data has passed through the factory floor.

Technical Summary

1. Problem Statement

Visual object perception relies on bidirectional information flow (feedforward and feedback) along the ventral visual stream, spanning from Early Visual Cortex (EVC) to the Lateral Occipital Complex (LOC). While the anatomical basis of these flows is known (feedforward connections terminate in middle cortical layers; feedback targets superficial and deep layers), identifying their distinct spatiotemporal dynamics and functional roles during naturalistic vision remains a challenge.

• The Gap: Previous methods either rely on invasive animal studies (not applicable to humans) or indirect experimental contrasts (e.g., attention vs. non-attention) that confound feedforward and feedback signals.
• The Challenge: Feedforward and feedback signals are rapid and overlapping in time, making them difficult to disentangle using standard macro-scale neuroimaging (e.g., 3T fMRI).

2. Methodology

The authors employed a multimodal approach combining 7T layer-specific fMRI, EEG, and Deep Neural Networks (DNNs) to resolve these dynamics with high spatial and temporal precision.

• Participants & Stimuli: 10 human participants viewed 24 naturalistic object images on real-world backgrounds.
• 7T fMRI Acquisition:
  • Used Gradient-Echo (GE-BOLD) EPI sequences at 0.9 mm isotropic resolution.
  • Layer Segmentation: Cortical ribbons in EVC and LOC were manually segmented and divided into three equidistant layers: Deep, Middle, and Superficial.
  • Analysis: General Linear Models (GLM) were used to extract beta weights for each condition. Representational Dissimilarity Matrices (RDMs) were constructed for each layer and region.
• EEG Data: Utilized existing EEG data (N=32) from a previous study using the same 24 images with a backward masking paradigm (ISI = 600 ms) to ensure naturalistic processing without masking interference.
• EEG-fMRI Fusion (Representational Similarity Analysis - RSA):
  • Correlated time-resolved EEG-RDMs (-200 to 600 ms) with region-specific and layer-specific fMRI-RDMs using Spearman's rank correlation.
  • Partial Correlation: To control for macrovascular contamination (where superficial layers show inflated BOLD signals due to draining veins), partial correlations were used to remove the influence of underlying layers when analyzing superficial layers.
• Feature Complexity Analysis (Commonality Analysis):
  • Compared neural RDMs against RDMs derived from a Vision Transformer (DeiT) DNN.
  • DNN layers serve as a proxy for feature complexity (low layers = simple features; high layers = complex features).
  • This allowed the authors to map the "format" of neural representations (feature complexity) to specific spatiotemporal windows.

3. Key Contributions

1. Methodological Advancement: Successfully validated EEG-fMRI fusion at 7T for layer-specific analysis, extending previous 3T macro-scale findings to the mesoscale of cortical layers.
2. Spatiotemporal Disentanglement: Provided the first non-invasive, layer-specific evidence distinguishing the timing of feedforward vs. feedback signals in human EVC and LOC.
3. Functional Characterization: Demonstrated that feedback signals are not merely modulatory but actively contribute to increasing the complexity of visual feature representations.

4. Key Results

A. Macroscale Dynamics (Region Level)

• Temporal Hierarchy: Object processing in EVC peaked earlier (120 ms) than in LOC (180 ms).
• Validation: The temporal progression confirmed the hierarchical flow of information, validating the fusion approach for finer layer analysis.
• Feature Complexity: EVC representations aligned with low-to-mid DNN layers (low complexity), while LOC aligned with mid-to-high DNN layers (high complexity).

B. Mesoscale Dynamics (Layer Level)

• EVC (Early Visual Cortex):
  • Middle Layer: Showed the earliest peak (~100 ms), interpreted as the initial feedforward input.
  • Deep/Superficial Layers: Peaked later (~140 ms). However, feature complexity analysis showed no shift in complexity across layers (all aligned with low-mid DNN layers).
  • Interpretation: The late signals in EVC likely reflect feedforward propagation or lateral gain modulation rather than inter-areal feedback.
• LOC (Lateral Occipital Complex):
  • Middle Layer: Peaked early (~160 ms), representing the feedforward arrival. Representations here were of mid-to-high complexity.
  • Superficial Layer: Peaked significantly later (~400 ms). Crucially, these late signals showed a shift to high complexity (aligning with the highest DNN layers).
  • Interpretation: The late, superficial signal in LOC represents feedback from higher-order areas that actively refines and increases feature complexity.

C. Feature Complexity Shift

• In LOC, there is a distinct transition:
  • Early (Feedforward): Mid-complexity features.
  • Late (Feedback): High-complexity features.
• This suggests feedback does not just modulate existing signals but facilitates the emergence of more complex, abstract object representations.

5. Significance and Implications

• Mechanistic Insight: The study provides a functional temporal map for the "canonical microcircuit" model of the cortex, confirming that feedforward signals enter via middle layers and feedback signals exit via superficial layers in higher-order areas.
• Predictive Coding: The findings support predictive coding theories where sensory input (feedforward) and predictive signals (feedback) are transmitted via distinct temporal and laminar channels.
• Cognitive Function: The results suggest that feedback is essential for high-level object recognition, potentially originating from areas like the Dorsolateral Prefrontal Cortex (DLPFC).
• Future Directions: The framework offers a non-invasive tool to study how attention, expectation, and memory utilize feedback loops to shape perception. It also highlights the need for higher-resolution imaging to further separate vascular effects from true neural layer specificity.

Conclusion: By combining layer-resolved fMRI, high-temporal EEG, and DNN modeling, the authors successfully isolated feedforward and feedback signals, revealing that feedback in the human visual cortex is a late-arriving, superficial-layer process that actively elevates the complexity of object representations.