🧬 biology

Spatial organization of faces and bodies in single neurons

This study demonstrates that inferotemporal cortex neurons encode faces and bodies not as translation-invariant templates, but through spatially localized subunits that preserve the geometric arrangement of facial features, thereby supporting both object identity and configural processing.

Original authors: Margaret Livingstone, Akshay Jagadeesh, sohrab najafian, Binxu Wang, Michael Arcaro

Published 2026-08-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Margaret Livingstone, Akshay Jagadeesh, sohrab najafian, Binxu Wang, Michael Arcaro

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The human brain is a master of recognition. It can identify a friend's face in a crowd, a familiar tree in a forest, or a specific car in a parking lot, regardless of where those objects appear in our vision or how large they seem. For decades, scientists studying the part of the brain responsible for this high-level vision, known as the inferotemporal cortex, believed that the neurons here acted like perfect, position-invariant templates. The prevailing idea was that once a neuron learned what a face looked like, it would fire whenever that face appeared, whether it was in the center of your vision, off to the side, or upside down. This theory suggested that the brain's final stage of visual processing had smoothed out all the messy details of location to focus purely on identity.

However, recognizing a face is a delicate business. It relies not just on seeing the parts, like eyes or a mouth, but on understanding exactly how those parts are arranged relative to one another. If you scramble the features of a face, it becomes unrecognizable, even if every single part is still there. This tension between needing to know "what" an object is and "where" it is located has long puzzled researchers. If the brain truly ignores location to recognize objects, how does it preserve the precise geometry that makes a face a face? A new study from Harvard Medical School and the University of Pennsylvania challenges the old view, suggesting that the brain's face-recognition cells are not blind to location, but are instead deeply sensitive to it, preserving the spatial map of the face within the neuron itself.

To investigate this, the researchers turned to macaque monkeys, whose visual systems are remarkably similar to our own. They implanted tiny electrode arrays into the monkeys' brains, specifically targeting regions known as "face patches," where neurons are highly selective for faces. The team then conducted a series of experiments to see how these neurons reacted when the monkeys looked at images. In the first experiment, they showed the monkeys pictures of upright faces and the exact same pictures turned upside down. They flashed these images at different spots across the monkeys' field of view, moving them around in a grid pattern. If the neurons were truly position-invariant, the center of their sensitivity should have remained the same regardless of whether the face was upright or inverted.

The results were striking. When the researchers mapped the sensitivity of these neurons, they found that the center of the neuron's attention shifted depending on the orientation of the face. Specifically, when the monkeys looked at an upside-down face, the neurons seemed to be looking higher up in the visual field than when the monkey looked at an upright face. This shift was not random; it was a consistent, measurable movement. The researchers realized this was not because the neurons were confused by the inversion, but because they were zeroing in on a specific feature: the eyes. In an upright face, the eyes are at the top. In an upside-down face, the eyes are at the bottom. The neurons were firing most strongly when the eyes landed in a specific spot relative to the neuron's own position. When the face was flipped, the eyes moved, and the neuron's "sweet spot" appeared to move with them. This suggested that the neurons were not seeing the whole face as a single, unchanging blob, but were instead tracking the specific location of its parts.

To test this further, the team showed the monkeys isolated parts of the face and body, such as just an eye, just a mouth, a whole face, or a whole body, flashing them at various locations. They found a clear, orderly pattern in how the neurons responded. Neurons that preferred eyes tended to be most active when the eyes were in the upper part of the visual field. Neurons that preferred mouths or bodies were more active when those features were lower down. This created a kind of internal map within the neuron's receptive field, where the top was reserved for eyes and the bottom for mouths or bodies. It was as if the neuron contained a miniature, spatially organized model of a face, rather than just a generic detector for "face-ness."

The researchers then took this a step further by showing the monkeys thousands of natural photographs, ranging from landscapes to portraits, and used a sophisticated computer model to decode what the neurons were actually seeing. This model broke down the neuron's response into smaller, distinct components. Instead of finding one single reason why a neuron fired, the model revealed that each neuron was actually a combination of several smaller, specialized detectors working together. One part of the neuron might be tuned to look for eyes in the upper visual field, while another part looked for hands or bodies in the lower field. These components were distinct and spatially separated, confirming that the neuron's sensitivity was not a uniform blanket but a mosaic of specific feature-location pairs.

These findings fundamentally change how we understand high-level vision. The study shows that the brain does not discard spatial information to recognize complex objects. Instead, it preserves the geometry of what it sees. The neurons in the face patches are not translation-invariant templates that ignore where an object is; they are spatially localized feature detectors that maintain the relationship between parts. This means that the brain's ability to recognize a face holistically likely emerges from the combined activity of many neurons, each holding a piece of the spatial puzzle. The brain does not need to throw away the "where" to know the "what"; it keeps both, weaving them together to create the seamless perception we experience every day. This work suggests that the brain's strategy for recognizing complex objects is far more intricate and spatially precise than previously thought, relying on a structured, map-like organization that mirrors the very shapes it seeks to understand.

Technical Summary: Spatial Organization of Faces and Bodies in Single Neurons

Problem Statement
The inferotemporal (IT) cortex is widely regarded as the substrate for position-invariant object recognition, where neurons encode complex objects like faces regardless of their retinal location. However, face perception is fundamentally holistic, relying on the specific spatial configuration of facial parts (e.g., eyes above the mouth). A tension exists between the canonical view of IT neurons as translation-invariant templates and the behavioral reality that face recognition is sensitive to configural changes (e.g., inversion). While previous studies have shown that IT neurons can be sensitive to local features or exhibit some position tolerance, it remains unresolved whether high-level feature selectivity and spatial tuning are separable or if they are intrinsically coupled within single neurons. Specifically, it is unclear if IT face cells act as global, position-invariant detectors or if they comprise spatially localized subunits that preserve the geometry of faces.

Methodology
The authors conducted three experiments using chronically implanted multielectrode arrays in the face-patch network (posterior, central, and anterior IT) of macaque monkeys.

Experiment 1 (Stimulus-Dependent Receptive Field Mapping): Researchers mapped receptive fields (RFs) using 6°×6° upright and inverted faces presented at multiple locations across a grid. Because upright and inverted faces are pixel-identical except for orientation, any shift in the estimated RF center between the two conditions would indicate that the RF estimate depends on the spatial arrangement of diagnostic features.
Experiment 2 (Feature-Specific Spatial Profiling): To determine if different features drive distinct spatial responses, the authors flashed isolated stimuli (eyes, mouths, hands, bodies, and whole faces) across a spatial grid. They analyzed the spatial response profiles to test for systematic vertical offsets between feature categories.
Experiment 3 (Factorized Encoding Model): The authors recorded responses to 1,000 natural images centered on the population RF. They fitted a low-rank, spatially factorized neural network encoding model (based on ResNet-50 features). This model decomposed each neuron's response into a sum of $K$ components, where each component paired a learned spatial mask with a learned feature weighting. This approach tested whether neurons are best described by a single spatial field or by multiple, spatially distinct feature subunits.

Key Results

Stimulus-Dependent RF Shifts: Receptive field centroids mapped with inverted faces were consistently displaced vertically upward compared to those mapped with upright faces. This shift was not due to a change in retinotopy but reflected the retinal position of the eyes. When the model was constrained to fit only the eyes, the vertical shift disappeared; when fitted to the mouth or full face, the shift remained. This indicates that the apparent RF location is dominated by the spatial preference for specific features (primarily the eyes) rather than a global, invariant template.
Systematic Feature Topography: Spatial response maps for isolated features showed orderly vertical offsets. Eyes and whole faces drove responses in the upper portion of the receptive field, while mouths and bodies drove responses in lower portions. This "faciotopic" or homuncular organization was observed across the population and in single units, with eye preferences consistently located above mouth preferences.
Factorized Subunits in Natural Images: The factorized encoding model revealed that most neurons (83% of channels in Experiment 2) were better described by multiple components ( $K > 1$ ) than by a single spatial field. These components were largely uncorrelated and tuned to distinct feature-location combinations. For example, a single neuron might possess an upper-field subunit tuned to eyes/faces and a lower-field subunit tuned to hands/bodies.
Semantic Decoding: When applied to natural image responses, the model recovered complementary factors. Factors concentrated in the upper visual field frequently corresponded to semantic labels of "eyes" or "face," while lower-field factors corresponded to "hands" or "body." The model accurately predicted held-out responses, confirming that single neurons integrate multiple, spatially segregated feature detectors.

Key Contributions

Refutation of Strict Translation Invariance: The study demonstrates that IT face cells are not translation-invariant templates. Instead, their feature selectivity varies systematically with retinal position.
Discovery of Spatio-Featural Subunits: The authors provide evidence that IT neurons function as mosaics of spatially localized subunits. Each subunit is tuned to a specific feature (e.g., eyes) at a specific retinal location (e.g., upper field), preserving the geometric layout of faces and bodies.
Methodological Advancement: The paper highlights that receptive field mapping in high-level vision cannot be adequately captured by a single generic probe. It necessitates sampling across both space and feature content to define a neuron's operational receptive field.
Modeling Framework: The introduction of a spatially factorized DNN encoding model successfully decomposes complex IT responses into interpretable spatio-featural components, bridging the gap between population-level fMRI maps and single-unit physiology.

Significance and Claims
The authors claim that their findings resolve the tension between holistic face perception and local feature coding. They argue that holistic perception does not require IT neurons themselves to be position-invariant holistic detectors. Instead, holistic perception likely emerges from downstream integration of spatially organized feature signals provided by IT.

The study posits that IT face patches are not simple detectors of whole faces but comprise spatio-featural subunits whose organization mirrors the geometry of faces in natural vision. This joint encoding of "where" and "what" provides a neural basis for preserving both identity and configural information. The authors suggest that this structured organization, where subunit heterogeneity preserves configurational information rather than promoting tolerance, represents a shift in the functional role of subunits at higher stages of the visual hierarchy compared to earlier areas like V1.

The paper concludes that these results constrain theories of face processing and refine models of IT, suggesting that future models must adopt spatio-featural descriptions rather than purely spatial (Gaussian) receptive fields to account for the inseparability of spatial and feature tuning in high-level vision.

More like this