A cortical hierarchical mechanism for subjective visual experience during working memory

This study reveals that the subjective visual experience of working memory relies on a specific cortical hierarchy from the intraparietal sulcus to early visual cortex, where aphantasia arises not from a single region's failure but from a breakdown in feedback signals along this pathway that prevents the transition from visual to mnemonic codes.

The Gist

The Mystery of the "Mind's Eye"

Imagine your brain has a mental sketchpad. When you remember something, like the face of a friend or the route to your favorite coffee shop, you can usually "see" it in your mind's eye. This is called visual imagery.

But for some people, this sketchpad is blank. They can remember facts and details perfectly well, but they cannot "see" the image in their mind. This condition is called Aphantasia.

This study asked a big question: Why can't people with aphantasia "see" their memories, even though their memory is working fine? Is their "screen" broken, or is the "projector" that shines the image onto the screen broken?

The Experiment: Three Ways to Remember

The researchers put two groups of people in an MRI machine:

1. Typical Imagers: People who can see images in their minds.
2. Aphantasics: People who cannot.

They gave them three different memory games involving a moving light pattern (a Gabor patch):

• The "Perception" Game: They actually saw the light move. (Both groups saw it clearly).
• The "Illusion" Game: They saw a light move straight up and down, but its texture made it look like it was moving diagonally. (Both groups saw the same illusion).
• The "Imagination" Game: They saw a symbol (like an "L" or "R") and had to imagine the light moving on their own, without seeing it.

The Result:

• Performance: Both groups did the memory games almost equally well. They could remember the direction of the light accurately.
• Experience: The "Typical Imagers" felt like they were watching a movie in their heads. The "Aphantasics" felt like they were just holding a list of facts in their heads, with no visual movie.

The Discovery: The Broken Feedback Loop

The researchers looked inside the brain to see what was happening. They found a fascinating "broken loop" in the brains of people with aphantasia.

Think of your brain's visual system like a high-tech theater:

• The Screen (Early Visual Cortex): This is the back of your brain where images are projected.
• The Director (Parietal Cortex/IPS): This is the front part of your brain that decides what to show and sends instructions.
• The Projector Beam (Feedback): The signal that goes from the Director back to the Screen to paint the image.

Here is what they found:

1. The Screen Works Fine (At First): When the "Typical Imagers" and "Aphantasics" actually saw the light (in the Perception game), the "Screen" (Early Visual Cortex) lit up normally for both groups. The feed-forward signal (eyes to brain) was working perfectly.
2. The Director is Confused: When it was time to imagine the light (the Imagination game), the "Director" (Parietal Cortex) in the aphantasic brain didn't send the right "start" signal. It was like the director forgot to yell "Action!"
3. The Projector Beam is Weak: Even when the Director tried to send instructions to keep the image alive during the delay (the waiting period), the connection was weak. The "feedback beam" from the Director to the Screen was broken.
   • In Typical Imagers: The Director shouted instructions, the beam hit the Screen, and a bright, stable image appeared.
   • In Aphantasics: The Director sent a weak signal, or the beam got lost. The Screen received a fuzzy, unstable signal. The image never fully formed, so the person didn't "see" anything.

The "Bottleneck" and the Backup Plan

The researchers found a specific traffic jam in the brain called V3AB. It's a hallway between the Director and the Screen. In aphantasics, the feedback signal gets blocked here. It's like a bridge that is closed; the message can't get from the front of the brain to the back.

But wait, how did they pass the test?
The study found a clever backup plan. While the "main screen" (the side of the brain connected to the eyes) was dark and fuzzy, the opposite side of the brain (the ipsilateral side) was working hard.

• Think of this as a backup generator. The aphantasic brain was using a different, non-visual code to remember the direction. It was remembering the logic of the movement (e.g., "it went left") rather than the picture of the movement.
• This is why they could still answer the questions correctly, even though they felt like they were just guessing or using logic, not "seeing" the image.

The Big Takeaway

This study proves that seeing with your mind's eye isn't just about having a good memory. It requires a specific conversation between two parts of the brain:

1. The part that plans the image (Parietal Cortex).
2. The part that displays the image (Visual Cortex).

In people with aphantasia, the "memory" part of the brain is fine, but the conversation between the planner and the display screen is broken. The planner can't shout loud enough, or the signal gets lost in the hallway, so the screen stays blank.

In short: Aphantasia isn't a failure of memory; it's a failure of the brain's internal projector. The movie is being planned, but the projector bulb is dim, so the audience (consciousness) sees nothing but a blank screen.

Technical Summary

1. Problem Statement

Working memory (WM) allows for the temporary maintenance and manipulation of information, often involving subjective sensory experiences (e.g., "seeing" an image in one's mind). While neuroimaging has identified content-specific neural representations in sensory cortices (like the Early Visual Cortex, EVC) during WM, it remains unclear how the brain generates the subjective conscious experience of these internal simulations.

A critical gap exists in distinguishing between the objective maintenance of information and the subjective experience of it. The phenomenon of aphantasia—where individuals report an inability to generate voluntary visual imagery despite normal cognitive function—offers a unique natural experiment to dissociate these two processes. The study aims to determine whether the lack of visual imagery in aphantasia stems from a localized dysfunction in the EVC or a broader breakdown in the cortical hierarchy and feedback mechanisms connecting higher-order regions to sensory areas.

2. Methodology

Participants:

• Groups: 17 individuals with aphantasia (confirmed via Vividness of Visual Imagery Questionnaire and interviews) and 17 typical imagers.
• Screening: Aphantasics reported low VVIQ scores (~16 vs. ~62 for typical imagers) and a lack of visual imagery across modalities, though they retained normal memory performance.

Experimental Design:
Participants performed three distinct delayed-recall WM tasks inside an fMRI scanner, differing only in the sample period (stimulus delivery) but sharing the same delay and probe periods:

1. Perception Task: Participants viewed a moving Gabor patch (visual stimulus).
2. Imagery Task: Participants viewed a symbolic cue and voluntarily imagined the moving Gabor (no external visual input).
3. Illusion Task: Participants viewed a Gabor with drifting internal texture that induced a motion illusion (involuntary imagery).

Key Manipulations:

• Spatial Specificity: Stimuli were presented in the right visual field. Neural analyses focused on contralateral regions (engaged in subjective experience) vs. ipsilateral regions (non-imagery field).
• Temporal Resolution: Tasks included a sample period (2s), a delay period (8.5s), and a probe period.
• Behavioral Measures: Participants reported motion direction (recall error) and vividness ratings (post-scan).

Neuroimaging & Analysis Techniques:

• fMRI: 3T scanners with high temporal resolution (TR = 1500ms).
• Multivariate Pattern Analysis (MVPA): Support Vector Machine (SVM) decoding to track motion direction representations in specific Regions of Interest (ROIs): EVC (V1-V3), V3AB, IPS (intraparietal sulcus), and MT+.
• Time-Resolved Decoding: Analyzed neural activity at each time point to distinguish feedforward (sample) vs. feedback (delay) signals.
• Representational Similarity Analysis (RSA): Compared neural Representational Dissimilarity Matrices (RDMs) against "Visual" (stimulus-driven) and "Mnemonic" (memory-driven) models to track the transition from sensory to memory codes.
• Functional Connectivity: Generalized Psychophysiological Interaction (gPPI) to assess connectivity between IPS and EVC.
• Controls: Eye-tracking (DeepMReye and external eye-tracker) and univariate BOLD analysis to rule out eye movement or global activation differences.

3. Key Contributions & Results

A. Behavioral Dissociation

• Performance: Aphantasics successfully completed WM tasks but showed significantly higher recall errors and variability than typical imagers, suggesting that visual imagery aids precision.
• Subjective Experience: Aphantasics reported near-zero vividness across all tasks, whereas typical imagers reported high vividness.
• Illusion Intact: Aphantasics perceived the motion illusion (sample period) normally, indicating their involuntary visual processing is intact.

B. Neural Mechanisms: The Hierarchical Breakdown
The study identified a specific cortical hierarchy failure from the Intraparietal Sulcus (IPS) to the Early Visual Cortex (EVC):

1. Impaired Imagery Generation (IPS):

   • In typical imagers, the contralateral IPS showed "imagery dominance" (stronger decoding during imagery than perception) during the sample period.
   • In aphantasics, this imagery dominance signal in the IPS was absent, indicating a failure to generate voluntary imagery content at the source.

2. Weakened Feedback to EVC:

   • Sample Period: Aphantasics showed normal feedforward decoding in EVC during perception and illusion tasks.
   • Delay Period: Aphantasics exhibited significantly weaker and unstable motion path representations in the contralateral EVC compared to typical imagers across all tasks (perception, imagery, and illusion).
   • Timing: The divergence in EVC representations emerged earlier in imagery/illusion tasks than in perception, suggesting a general deficit in feedback signals, not just a maintenance failure.

3. The V3AB Bottleneck:

   • RSA revealed that typical imagers transition from visual codes (stimulus-driven) to mnemonic codes (memory-driven) along the hierarchy.
   • In aphantasics, this transition was blocked around area V3AB. They failed to shift to stable mnemonic codes, retaining unstable, task-dependent representations.

4. Functional Connectivity:

   • gPPI results: Typical imagers showed strong positive connectivity between contralateral IPS and EVC during the delay. Aphantasics showed minimal connectivity in this pathway, specifically during imagery and illusion tasks.

C. Compensatory Mechanisms

• Despite deficits in the contralateral (visual) pathway, aphantasics maintained intact representations in the ipsilateral IPS.
• These ipsilateral signals were non-visual (not tied to conscious imagery) but sufficient to support behavioral performance, explaining why aphantasics can perform WM tasks without "seeing" the image.

D. Control Analyses

• Differences were not due to overall BOLD signal intensity (univariate analysis showed no group differences).
• Differences were not driven by eye movements (eye-tracking decoding failed to predict motion paths).

4. Significance

• Redefining Aphantasia: The study moves beyond characterizing aphantasia as a simple deficit in voluntary imagery. It identifies a general deficit in top-down feedback along the cortical hierarchy (IPS $\to$ V3AB $\to$ EVC), affecting both voluntary and involuntary (illusion) processes.
• Mechanism of Consciousness: It provides evidence that subjective visual experience during WM relies on stable, feedback-driven representations in the EVC. Without this feedback, information is maintained in a "non-visual" format (likely in the IPS) that supports behavior but lacks conscious qualia.
• Cortical Hierarchy Model: The findings support a hierarchical model where the transition from sensory to mnemonic codes occurs at specific bottlenecks (V3AB). Disruption in this flow prevents the emergence of conscious visualization.
• Dissociation of Maintenance and Experience: The study demonstrates that WM maintenance (behavioral success) and subjective visual experience are neurally dissociable. The brain can maintain information using non-visual codes (ipsilateral IPS) when the visual feedback loop is broken.

In conclusion, the paper establishes that the cortical hierarchy from IPS to EVC is the critical substrate for conscious mental visualization. Aphantasia arises not from a lack of sensory storage, but from a failure in the feedback mechanisms required to transform and sustain those sensory representations into a conscious experience.