🧠 neuroscience

Inner visual experience biases object neural representational geometry during perceptual encoding

This study demonstrates that subjective imagery experience, as evidenced by the presence or absence of aphantasia, biases the geometry of neural object representations during perceptual encoding, specifically reducing visual-similarity alignment in the left lateral occipitotemporal cortex while preserving language-relation alignment, suggesting that perception and imagery rely on heterogeneous rather than unitary neural codes.

Original authors: Tian, S., Mao, X., Wang, D.-H., Wang, X., Bi, Y.

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tian, S., Mao, X., Wang, D.-H., Wang, X., Bi, Y.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine your brain as a super-advanced library. For decades, scientists have known that this library has a special "Imagination Wing" and a "Perception Wing" that look surprisingly similar. When you see a picture of a cat, your brain lights up in a specific way. When you close your eyes and try to picture a cat, those same lights often flicker on again. This has led many to believe that seeing and imagining are basically the same process, just running on different tracks. But here's the mystery: what if the "books" on the shelves in these wings are organized differently for different people? Some people have incredibly vivid mental movies when they imagine things, while others, a group called "aphantasics," report having a completely blank mind when they try to visualize. They can describe a cat perfectly, but they can't "see" it in their mind's eye. The big question is: does this difference in imagination change how their brains organize information even when they are just looking at the real thing? Or is their brain's library perfectly normal, and only the "Imagination Wing" is missing a few lights?

A team of researchers decided to peek inside the brain's library of people with and without mental imagery to solve this puzzle. They used a high-tech camera called fMRI to watch the brain while participants looked at pictures of animals and objects, and then tried to remember them. They didn't just look at where the brain was active; they looked at how the information was arranged. Think of it like checking if a librarian sorts books by color, by author, or by the story inside. The researchers compared the brain's "sorting style" against two different blueprints: one based on how things look (visual similarity) and one based on how things are described in words (language relations).

Here is what they found, and it's a bit of a plot twist. First, they confirmed that both groups could do the memory task just as well. Whether you can "see" a cat in your mind or not, your brain can still tell the difference between a cat and a boat perfectly fine. However, the way their brains organized the information was different. For people with vivid imagery, the brain's "Perception Wing" (specifically a region called the left lateral occipitotemporal cortex) was sorted primarily by how things looked. It was like a library where books are arranged by their cover art. But for the aphantasics, this "cover art" sorting was significantly weaker. Their brains were less focused on visual similarities when they were just looking at the objects.

Interestingly, both groups had a very strong "language sorting" system. Their brains organized objects based on how we talk about them and the words we use to describe them, regardless of whether they could imagine the object or not. This language-based organization was rock-solid in both groups, acting like a reliable backup system. The study suggests that the ability to have a vivid mental image isn't just a party trick that happens after you see something; it's actually tied to how your brain organizes the visual world while you are looking at it. If you don't have that inner movie, your brain might rely more on the "word catalog" and less on the "picture catalog" right from the start.

The researchers also looked at what happened during the imagination part of the task. They found that for people who could imagine, the strength of that "visual sorting" in their brain matched how vivid their mental images felt. The more vivid the image, the stronger the visual pattern. But for the aphantasics, this visual pattern was barely there. This suggests that the difference between seeing and imagining isn't just about a switch turning on or off in a specific "imagery room." Instead, the whole library's organization is subtly different. The brain of someone with aphantasia seems to build its object knowledge using a different blueprint—one that leans heavily on language and meaning, rather than the pure visual geometry that guides the brains of those with vivid mental imagery. It's a reminder that two people can look at the same world and process it in fundamentally different ways, even if they both get the job done perfectly.

Technical Summary: Inner Visual Experience Biases Object Neural Representational Geometry During Perceptual Encoding

Problem and Motivation
Decades of research have established substantial neural overlap between visual perception and visual mental imagery, particularly in the ventral occipitotemporal cortex (VOTC). The prevailing view posits that these processes rely on a common neural code. However, this overlap is often demonstrated through shared activation or cross-decoding accuracy, which remains agnostic to the specific content and geometry of the underlying neural representations. A critical question remains: does the subjective experience of imagery shape the organization of object representations even during the initial perceptual encoding phase?

To address this, the authors utilize congenital aphantasia—a condition characterized by the absence of conscious visual imagery—as a natural experiment to dissociate imagery experience from object knowledge. While previous studies suggested that aphantasia primarily affects imagery-specific processes (e.g., reduced perception-imagery similarity in specific nodes), emerging evidence hints that perceptual representations themselves might differ. The study investigates whether the absence of imagery experience alters the representational geometry of objects during perception, specifically testing if neural representations in aphantasics rely more heavily on language-derived semantic structures rather than visual-similarity structures.

Methodology
The study employed a retro-cue visual working memory paradigm using fMRI to compare two groups: individuals with congenital aphantasia ( $N=17$ ) and typical visualizers ( $N=22$ ). Participants viewed pairs of object images (perception phase), received a cue to visualize one of the objects (imagery phase), and then performed a recognition task.

The analytical pipeline involved three main stages:

Localization of Shared Representations: Whole-brain searchlight Support Vector Machine (SVM) decoding was performed to identify regions exhibiting successful cross-phase decoding (training on perception, testing on imagery, and vice versa) for object domains. This identified Regions of Interest (ROIs) in the bilateral VOTC, posterior middle temporal gyrus (pMTG), and right inferior frontal gyrus (IFG).
Representational Similarity Analysis (RSA): Within these ROIs, the authors used linear mixed-effects models to fit neural Representational Dissimilarity Matrices (RDMs) against reference geometries derived from a multimodal deep neural network (CLIP). The CLIP model provided two distinct reference spaces:
- Image-encoder RDM: Captures visual-similarity-aligned geometry.
- Text-encoder RDM: Captures language-relation-aligned geometry.
  The analysis controlled for the other modality to isolate unique contributions.
Model Comparison: The study compared the explanatory power of the multimodal CLIP encoders against unimodal counterparts (DINOv2 for vision, RoBERTa for text) across different network layers to assess the role of multimodal alignment in different cognitive states.

Key Results

Behavioral Performance: Both groups performed the working memory task with comparable accuracy and reaction times, indicating that the absence of visual imagery does not impair basic object recognition or working memory performance.
Perceptual Encoding Geometry:
- Visualizers: Showed significant unique alignment with the CLIP image-encoder (visual-similarity geometry) in the bilateral VOTC and left lateral occipitotemporal cortex (LOTC).
- Aphantasics: Showed significantly reduced unique alignment with the image-encoder geometry in the left LOTC during perception. However, both groups exhibited comparable and robust unique alignment with the CLIP text-encoder (language-relation geometry) in the bilateral fusiform gyrus (FG), pMTG, and right IFG.
- Interaction: A significant interaction between group and the image-encoder effect was found, confirming that the reduction in visual-aligned geometry is specific to the aphantasic group.
Imagery Phase Geometry: During the imagery phase, the unique alignment with the image-encoder geometry was not significantly different between groups at the group level. However, across all subjects, the strength of this alignment in the left LOTC positively correlated with subjective imagery vividness (VVIQ scores).
Model Comparison: During perception, unimodal models (DINOv2, RoBERTa) often explained neural variance as well as or better than their multimodal CLIP counterparts. Conversely, during the imagery phase, the multimodal CLIP encoders showed a relative advantage over unimodal models, suggesting that multimodal alignment becomes more critical when reconstructing representations without direct sensory input.

Significance and Claims
The paper claims that subjective imagery experience is not merely a byproduct of intact object representations but is fundamentally associated with the geometry of neural object representations during both perception and imagery.

Perception is Not Invariant: The study challenges the assumption that perceptual representations serve as a stable, uniform reference for imagery. Instead, the "visual" nature of perceptual encoding is modulated by imagery experience; individuals without imagery (aphantasics) exhibit a systematic reduction in visual-similarity-aligned geometry even before imagery generation begins.
Heterogeneity of Shared Codes: Regions traditionally defined as having "shared" perception-imagery codes (via cross-decoding) actually comprise heterogeneous representational geometries. Successful cross-decoding in aphantasics appears to be supported by preserved language-relation-aligned geometry, even when visual-similarity geometry is diminished.
Role of Multimodal Alignment: The findings suggest that the ventral visual stream encodes object knowledge through multiple organizational principles. While visual-similarity geometry is sensitive to imagery experience, language-relation geometry remains robust across populations and cognitive states. Furthermore, the reliance on multimodal (visual-language) alignment increases when representations must be internally maintained in the absence of sensory input.

In conclusion, the authors argue that the "shared neural code" between perception and imagery is not a unitary format but a composite of heterogeneous geometries, where the expression of visual-similarity structure is directly linked to the capacity for subjective visual imagery.

More like this