Mid-superior temporal sulcus encodes spatial context and behavioral state in freely moving macaques

This study demonstrates that in freely moving macaques, the mid-superior temporal sulcus (mSTS) encodes a joint representation of spatial context and behavioral state, dynamically shifting between allocentric and body-centric coordinates while tracking the sequential structure of natural behaviors.

The Gist

The Big Idea: Breaking the "Lab Rat" Mold

For decades, scientists have studied the brains of monkeys (and humans) by keeping them still, staring at screens, and pressing buttons. It's like trying to understand how a car engine works by only watching it idle in a garage. You learn a lot about the engine, but you miss how it handles a bumpy road, a steep hill, or a sudden turn.

This paper asks: What is the brain actually doing when an animal is free to run, climb, and explore?

The researchers focused on a specific part of the monkey brain called the mid-superior temporal sulcus (mSTS). In the past, we thought this part of the brain was like a "Social Media Feed"—it only cared about watching other people, faces, and social interactions. But the researchers suspected that if you let the monkey move freely, this brain region might be doing something much more complex.

The Experiment: A Monkey Gym with a Camera Drone

To test this, the scientists built a giant, hexagonal "monkey gym" (about 8 feet tall) with platforms, ladders, and a ceiling. They put two monkeys inside to play, climb, and forage for nuts.

Instead of taping wires to their heads (which would stop them from climbing), the researchers used wireless brain implants. It's like giving the monkey a tiny, invisible Fitbit that records brain activity. At the same time, 30 high-speed cameras tracked the monkey's body in 3D space, creating a digital "ghost" of the monkey moving around the room.

The Discovery: The Brain is a "Context-Aware GPS"

When the monkeys were free to move, the mSTS didn't just watch the world; it became a master of context. Here are the four main things they found:

1. The "Where" is as important as the "What"

In the old "still monkey" experiments, this brain region cared mostly about what the monkey was looking at. But in the free-moving gym, the brain cared most about where the monkey was standing.

• Analogy: Imagine you are walking through a house. If you are in the kitchen, your brain knows "I am near the stove." If you are on the roof, your brain knows "I am high up." The mSTS acts like a GPS that knows exactly which room you are in, even if you aren't looking at the walls.

2. The Brain Changes Its "Map" Depending on the Height

This is the coolest part. The brain uses different "coordinate systems" depending on what the monkey is doing.

• On the floor: The brain uses an allocentric map (like a map on a wall). It knows, "I am 5 feet from the north wall."
• Climbing the ceiling: The brain switches to a body-centric map. It stops thinking about the room and starts thinking about its own body. "My arm is reaching up," "My legs are hanging down."
• Analogy: Think of a video game. When you are walking on flat ground, the game uses a world map. But when you are climbing a ladder or hanging from a rope, the game switches to a "first-person" view where your body is the center of the universe. The monkey's brain does this switch automatically!

3. The Brain Predicts the Next Move

The researchers found that the brain doesn't just react to what the monkey is doing right now; it predicts what it will do next.

• Analogy: Imagine a dance partner who knows the next step before you even lift your foot. The brain activity in the mSTS starts shifting toward the "next move" (like climbing up) before the monkey actually starts climbing. It's a pre-race warm-up for the brain.

4. The Same Move Looks Different in Different Places

If a monkey reaches for a nut on the floor, it's a different "neural event" than if it reaches for a nut while hanging from the ceiling. Even though the hand movement looks similar, the brain treats them as totally different tasks because the context (gravity, balance, location) is different.

• Analogy: Think of the word "Run." If you say "Run" in a race, it means sprinting. If you say "Run" in a computer program, it means executing code. The word is the same, but the context changes the meaning. The monkey's brain treats the same physical action differently depending on where it is happening.

Why Does This Matter?

For a long time, we thought the mSTS was a specialized "Social Brain" that only cared about other monkeys. This paper suggests that the brain is more versatile than we thought.

The mSTS might be a general-purpose "Situation Awareness" center. It helps the animal understand:

1. Where am I? (Spatial context)
2. What am I doing? (Behavioral state)
3. How do these two fit together? (Context)

The Takeaway:
You can't understand how a brain works by keeping it in a cage. Just like you can't understand a car by only looking at it in a showroom, you have to let it drive. When the monkey was free to move, its brain revealed a sophisticated, dynamic system that constantly updates its map of the world and its own body, preparing for the next move before it even happens. This region isn't just for watching others; it's for navigating the world.

Technical Summary

1. Problem Statement

Traditional primate neuroscience has largely relied on constrained paradigms (e.g., head restraint, controlled stimuli, trial-based tasks), which may obscure neural computations that only emerge during unconstrained, natural behavior. The mid-superior temporal sulcus (mSTS) is historically characterized as a hub for social perception, biological motion, and action observation based on these restrained studies. However, its role during free movement remains untested.

• Hypothesis: The mSTS carries richer, joint representations of spatial context and behavioral state during natural behavior, extending beyond the observer-centric models derived from restrained paradigms.
• Challenge: Recording from the deep sulcal cortex of freely moving primates is technically difficult due to the inaccessibility of the sulcus to surface arrays and the need to dissociate visual inputs from kinematic self-motion signals.

2. Methodology

The study employed a multi-modal approach combining wireless neurophysiology, high-resolution 3D pose tracking, and computational ethology.

• Subjects & Environment: Two adult rhesus macaques (M1, M2) freely explored a large, hexagonal 3D arena (2.4m tall, 1.5m per side) containing multi-level structures (floor, benches, ceiling) to encourage climbing and suspension.
• Neural Recording:
  • System: Wireless depth-electrode recordings using 128-channel NeuroNexus arrays implanted bilaterally in the mSTS (64 channels on the ventral bank, 64 on the dorsal bank).
  • Data: 1,404 single-unit (SUA) and multi-unit (MUA) recordings across five sessions.
• Behavioral Tracking:
  • Pose Estimation: A 30-camera motion capture system tracked 17 anatomical landmarks at 40 Hz with sub-pixel precision (median reprojection error: 0.35 px).
  • Feature Extraction: Derived 3D center-of-mass (COM) trajectories, head orientation, and kinematic variables (velocity, angular velocity).
  • Behavioral Segmentation: Used unsupervised learning (dimensionality reduction + clustering) on pose dynamics to identify 12 discrete behavioral syllables (e.g., rest, forage, climb, scan).
• Analytical Framework:
  • Encoding Models: Ridge regression with blocked temporal cross-validation to predict firing rates from feature groups: Visual proxies (VIS), Body kinematics (BODY), Head kinematics (HEAD), and 3D Position (POS).
  • Variance Decomposition: Quantified unique variance ($\Delta R^2$) explained by each feature group using block-shuffling.
  • Reference Frame Analysis: Compared encoding in Allocentric (world-fixed), Body-centric, and Head-centric coordinates.
  • Population Manifold Analysis: Used CEBRA (Contrastive Learning for Behavioral and Neural Analysis) to create low-dimensional embeddings of population activity, supervised by spatial and behavioral labels.

3. Key Contributions

1. First Wireless mSTS Recordings: Successfully recorded from the deep sulcal cortex of freely moving macaques, overcoming previous technical barriers.
2. Joint Coding Discovery: Demonstrated that mSTS encodes a joint representation of spatial context (where the animal is) and behavioral state (what the animal is doing), challenging the view of mSTS as purely a social/visual observer region.
3. Context-Dependent Reference Frames: Revealed that the reference frame for encoding is not fixed but shifts dynamically based on vertical context (allocentric on the floor, body-centric during climbing).
4. Temporal Structure: Identified that mSTS populations encode discrete behavioral syllables with sustained temporal structure and anticipatory dynamics.

4. Key Results

A. Spatial Position Dominates Encoding

• Variance Decomposition: Static 3D position in the arena (POS) explained the most unique variance in mSTS firing rates (median $\Delta R^2 = 0.063$), exceeding visual features (0.026), body kinematics (0.004), and head kinematics (0.003).
• Multiplexing: 73% of encoding-positive units carried significant unique variance for both spatial position and visual features, indicating multiplexed coding rather than segregated subpopulations.
• Decoding: Population activity could decode the animal's vertical tier (floor/bench/ceiling) with ~59% accuracy (raw) and ~51% accuracy even after regressing out visual and kinematic covariates, proving the spatial signal is distinct from sensory/motor confounds.

B. Flexible Reference Frames

• Overall Preference: Allocentric (world-fixed) coordinates were the best fit for 50% of units, outperforming body-centric (37%) and head-centric (13%) frames.
• Contextual Shift: The reference frame preference shifted monotonically with vertical height:
  • Floor: Strongly allocentric-dominant (median RF-index = +0.60).
  • Bench: Neutral/Mixed (+0.02).
  • Ceiling: Shifted toward body-centric dominance (-0.24).
• Implication: The brain adapts its coordinate strategy based on the behavioral demands of the environment (e.g., navigating open space vs. climbing).

C. Behavioral Syllable Encoding

• Sustained Representation: mSTS populations decoded discrete behavioral syllables with accuracy significantly above chance (19–25%).
• Temporal Generalization: A classifier trained at one time point around a syllable transition generalized broadly to other time points (off-diagonal accuracy in Temporal Generalization Matrices), indicating a sustained state representation rather than just a transient onset response.
• Sequence Structure: Neural decoding confusion correlated with behavioral transition probabilities; syllables that frequently follow one another were more often confused by the neural decoder.

D. Joint Population Manifold

• Context-Dependent Coding: Using CEBRA embeddings, the study showed that the same behavioral syllable occupied different regions of the neural population space depending on the animal's vertical location.
  • Metric: The distance between centroids of the same syllable at different heights was 1.33–3.73 times larger than the distance within the same context.
• Anticipatory Drift: Neural trajectories drifted toward the centroid of the upcoming syllable before the behavioral transition occurred (pre-ramp dynamics), suggesting prospective coding of imminent state changes.

5. Significance

• Redefining mSTS Function: The findings suggest that mSTS is not merely a passive observer of social cues but a region performing domain-general computations useful for monitoring self-motion, spatial context, and behavioral state. These computations are likely recruited during social interactions to track both self and others in a shared environment.
• Neuroethology Advancement: This study validates the "primate neuroethology" approach, demonstrating that unconstrained behavior reveals computational patterns (like joint spatial-behavioral coding and flexible reference frames) that are systematically missed in restrained paradigms.
• Theoretical Implications: The results support the hypothesis that brain regions associated with social cognition perform computations that are broadly useful for navigating complex, dynamic environments, regardless of whether the agent is interacting with another individual or exploring alone.

In summary, the paper establishes that the mSTS in freely moving macaques acts as a dynamic integrator of where the animal is and what the animal is doing, utilizing flexible coordinate systems and anticipatory population dynamics to support real-time navigation and action.