An Efficient Computing Theory of Prefrontal Structured Working Memory Representations

This paper proposes an efficient computing theory for prefrontal cortex that resolves conflicting neural coding schemes in structured working memory tasks by demonstrating that contextual and compositional representations are optimal extremes on a spectrum determined by the statistical correlations of task elements.

The Gist

The Big Idea: The Brain's "Energy Bill"

Imagine your brain is a massive, high-tech city. For decades, scientists have known that the sensory parts of the city (like the eyes and ears) are incredibly efficient. They act like smart power grids: they only send electricity (neural signals) where it's absolutely needed, compressing information so nothing is wasted. This is called "Efficient Coding."

But what about the brain's "CEO office"—the Prefrontal Cortex? This is where we hold thoughts in our heads (working memory), like remembering a phone number or planning a sequence of moves. Scientists have been confused here. Sometimes, they see neurons acting like specialized librarians (each one only knows one specific book). Other times, they see neurons acting like generalist mixologists (each one knows a little bit about everything, mixing them together).

Why the difference? Is the brain broken? No. This paper argues that the brain is actually a frugal accountant. It doesn't just care about what information it holds; it cares about how much energy it costs to run the calculations needed to use that information.

The New Theory: "Efficient Computing"

The authors propose a new theory called Efficient Computing. Instead of just asking, "How do we store this data cheaply?" they ask, "How do we do the math on this data cheaply?"

Think of it like this:

• Old View (Efficient Coding): "Let's pack our suitcase as tightly as possible so we can carry it."
• New View (Efficient Computing): "Let's pack our suitcase so that when we get to the hotel, we can unpack it, find our socks, and put them on without tripping over our shoes."

The brain optimizes for the task, not just the storage.

The Two Coding Styles: The Library vs. The Mixer

The paper explains two famous, seemingly contradictory ways the brain stores sequences (like remembering a list of items):

1. The "Contextual" Code (The Specialized Librarian):

   • What it looks like: A specific neuron only fires if you are remembering "Step 1 of the 'Push-Turn-Pull' dance." It doesn't care about "Step 1 of the 'Eat-Sleep-Code' routine."
   • When it happens: When the tasks are predictable and repetitive. If you always do the same dance, the brain creates a dedicated "shortcut" neuron for that specific routine. It's like having a specific key for a specific door.
   • Analogy: If you only ever drive the same route to work, you don't need a GPS; you just memorize the turns. Your brain builds a direct, hard-wired path.

2. The "Compositional" Code (The Mixologist):

   • What it looks like: One neuron represents "Item A," another represents "Item B." To remember "A then B," the brain just turns on both neurons at the same time.
   • When it happens: When the tasks are random and diverse. If you have to remember a random list of groceries every day, you can't build a shortcut for every possible list. You need a flexible system where you can mix and match "Apples" and "Milk" neurons to create any list.
   • Analogy: If you are a chef who has to cook a different meal every night, you don't build a separate kitchen for "Pasta Night." You have a pantry of ingredients (neurons) and you just grab what you need for the specific recipe.

The "Spectrum" Discovery

The paper's biggest "Aha!" moment is that these aren't two different brain types. They are just two ends of a spectrum.

Imagine a dimmer switch for a lightbulb.

• Turn it all the way down (Low Diversity): The tasks are very similar (highly correlated). The brain saves energy by using Contextual codes (specialized shortcuts).
• Turn it all the way up (High Diversity): The tasks are totally different (independent). The brain saves energy by using Compositional codes (flexible mixing).

The "dimmer" is simply how much the items in the sequence depend on each other.

• Example: If you are remembering a phone number, the digits are random. You need a Compositional system.
• Example: If you are remembering a dance routine where "Step 2" is always "Spin," you need a Contextual system.

Why Do Some Memories Fade Faster?

The paper also explains why, in some experiments, the memory of the first item in a list is stronger than the last item.

Think of the brain's memory as a conveyor belt in a factory.

• If the factory has to grab items off the belt at different times, it's most efficient to have the "grabbing station" (the readout) right next to the newest item.
• To save energy, the brain shrinks the "storage space" for items that are far away on the belt. It's like packing a suitcase: you put the heavy, important stuff (the item you need now) in the easy-to-reach pocket, and the lighter stuff (the item you need in 5 seconds) in the back.

This explains why, in some monkey experiments, the neurons for the first item in a sequence are "louder" (larger subspace) than the neurons for the third item. The brain is optimizing its energy bill based on when the information is needed.

The "Algorithm" Detective Work

Finally, the authors use this theory to solve a mystery. In some experiments, scientists saw data that could be explained by two different "algorithms" (ways the brain moves information):

1. The Gating Method: You put Item A in Box 1, Item B in Box 2. They stay there until you need them.
2. The Conveyor Belt Method: You put Item A in Box 1, then it slides to Box 2, then Box 3, waiting for the "Go" signal.

The data looked the same for both. But the authors' theory predicted that if the brain is using the Conveyor Belt, the "boxes" (subspaces) would have to be different sizes to save energy. When they re-analyzed the data, they found the boxes were different sizes.

Conclusion: The brain was using the Conveyor Belt method! The theory allowed them to deduce the invisible "software" running in the brain just by looking at the "hardware" energy usage.

Summary

This paper tells us that the brain isn't just a passive storage unit. It's an active, energy-saving engineer.

• If your life is predictable, the brain builds specialized shortcuts (Contextual).
• If your life is chaotic, the brain builds flexible mixers (Compositional).
• And it arranges everything to use the least amount of electricity possible while still getting the job done.

By understanding the "energy bill" of the brain, we can finally make sense of why it looks so different in different experiments. It's not broken; it's just doing the math efficiently.

Technical Summary

1. Problem Statement

The Efficient Coding Hypothesis has successfully explained sensory representations (e.g., retinal ganglion cells, V1) as optimal encodings of natural stimuli statistics. However, its application to cognitive representations in the prefrontal cortex (PFC) has been limited and less successful.

A specific puzzle in PFC research concerns structured working memory tasks (e.g., recalling a sequence of items). Experimental literature reports two seemingly contradictory coding schemes for identical or similar tasks:

1. Compositional Coding: Neurons are tuned to specific sequence elements (e.g., "Item A") regardless of the sequence context. The total activity is a sum of independent subspaces (e.g., Xie et al.).
2. Contextual Coding: Neurons are tuned to specific sequences or contexts (e.g., "The sequence A-B-C"). Single neurons fire only for a specific order, acting as "splitter cells" (e.g., Shima & Tanji).

The authors argue that the failure of previous normative theories to unify these findings stems from focusing solely on encoded variables while ignoring the computations the neural circuits must perform. They propose that these distinct coding schemes are not contradictory but are extremes of a spectrum determined by task statistics and computational constraints.

2. Methodology: The Efficient Computing Framework

The authors develop a normative theory of efficient computing that models neural activity as the most energy-efficient implementation of a specific algorithm required to solve a task.

A. Task Formalism

They define a family of Structured Working Memory Tasks comprising:

• Latent States ($s$): The internal state of the agent (e.g., position in a sequence).
• Actions ($a$): Transitions between states (e.g., "move forward").
• Stimuli ($o$): Observations attached to states that must be recalled.
• Environment ($e$): The specific mapping of stimuli to states (e.g., which port corresponds to which letter in a sequence).

B. The Optimization Problem

The theory seeks a neural representation vector $r(s, e)$ (firing rates) that minimizes an energy loss function subject to three constraints:

1. Functional Constraint (Memory): The representation must allow for the accurate decoding of the current stimulus via a fixed readout matrix $W_{out}$:
   $$W_{out}r(s, e) + b_{out} = o(s, e)$$
2. Computing Constraint (Dynamics): The representation must update correctly based on actions to track the latent state. This is enforced by action-specific weight matrices $W_a$:
   $$W_a r(s, e) + b_a = r(s + a, e)$$
   This ensures the network implements an internal model of the task structure (path integration).
3. Biological Constraint (Efficiency): The system minimizes energy consumption, defined as the sum of squared firing rates (activity) and squared weight norms (synaptic transmission costs):
   $$L = \langle ||r(s, e)||^2 \rangle + \lambda \left( ||W_{out}||^2_F + \sum_a ||W_a||^2_F \right)$$

C. Analytical Approach

The authors perform both mathematical analysis (deriving necessary subspace structures) and numerical optimization (using gradient descent on the loss function) to find the optimal $r(s, e)$, $W_{out}$, and $W_a$ for various task statistics.

3. Key Contributions and Theoretical Findings

A. The Subspace Structure

The authors prove that satisfying the functional and computing constraints necessitates that the neural representation is composed of a set of linear subspaces.

• Each subspace encodes the stimulus at a specific "offset" (action distance) from the current state.
• The dynamics of the task (actions) act as a "shuffling" mechanism, moving activity between these subspaces.
• This structure unifies the "slots" observed in Recurrent Neural Networks (RNNs) and the subspace dynamics observed in PFC recordings.

B. The "Axis of Task Diversity"

The core theoretical contribution is the demonstration that task statistics (specifically the correlations between sequence elements) determine the geometry of these subspaces:

• Independent Stimuli (High Diversity): When sequence elements are sampled independently (maximal entropy), the optimal solution is Compositional Coding. Subspaces are orthogonal to minimize interference and energy. This matches data where neurons are tuned to specific items regardless of order.
• Correlated Stimuli (Low Diversity): When sequence elements are highly correlated (e.g., knowing the first item predicts the second), the optimal solution is Contextual Coding. Subspaces become aligned (non-orthogonal). This allows the network to "whiten" the data, reducing the total firing energy required. This matches data where neurons fire only for specific sequences.

C. Subspace Sizes and "Distance to Readout"

The theory predicts that the size (magnitude of activity) of a subspace depends on its "distance" from the readout subspace:

• Subspaces encoding items to be recalled immediately (distance 0) are largest to minimize readout weights.
• Subspaces encoding items further in the future are smaller to minimize firing energy, provided the recurrent weights can scale them up later.
• This explains discrepancies in data: tasks with symmetric readout requirements (Panichello & Buschman) show equal subspace sizes, while tasks with sequential readout (Xie et al.) show a decay in subspace size (1st > 2nd > 3rd).

D. Algorithmic Inference

The framework allows researchers to infer the underlying algorithm (how information flows) from neural data:

• Input Gating vs. Conveyor Belt: The theory distinguishes between algorithms where stimuli are gated directly into their final storage subspace versus a "conveyor belt" where they move through intermediate subspaces.
• Output Dynamics: By analyzing the relative sizes of subspaces, the authors infer that while PFC uses Input Gating (to handle variable sequence lengths), it likely uses a Conveyor Belt for output (to maximize speed), a prediction derived from the observed subspace size gradients.

4. Results and Validation

The authors validated their theory against existing neurophysiological datasets and new simulations:

• Unifying Discrepancies: They explained why Xie et al. (monkeys recalling 3 cues) found aligned subspaces, while Panichello & Buschman (monkeys recalling 2 colors) found orthogonal subspaces. The difference lies in the sampling method: Xie et al. sampled without replacement (creating correlations), while Panichello & Buschman sampled with replacement (independent).
• Single-Neuron Tuning: The theory predicts that orthogonal subspaces (independent memories) map to distinct, modular neurons, while aligned subspaces (correlated memories) map to "splitter" neurons tuned to specific contexts. This aligns with findings from Shima & Tanji (contextual) and Mushiake et al. (compositional).
• Dynamic Coding: The model reproduces the "rotational dynamics" observed in PFC during variable delay periods, showing that rotating memory through subspaces is more energy-efficient than maintaining a static high-energy representation if recall is not immediately required.

5. Significance

• Normative Framework for Cognition: This work extends the efficient coding hypothesis from sensory processing to high-level cognitive computation, providing a mathematical basis for understanding why the brain uses specific representational geometries.
• Resolution of Conflicting Literature: It resolves the apparent contradiction between "compositional" and "contextual" coding in PFC, showing they are optimal solutions to different statistical constraints of the task.
• Predictive Power: The theory generates testable predictions, such as how changing task statistics (e.g., sampling with vs. without replacement) should flip the alignment of neural subspaces, and how subspace sizes should correlate with the temporal distance of recall.
• Algorithmic Inference: It provides a tool to reverse-engineer the computational algorithms (e.g., gating vs. conveyor belts) used by the brain based solely on the geometry of neural population activity.

In summary, the paper establishes that prefrontal working memory representations are not arbitrary but are the energy-optimal solutions to specific computational constraints, where the geometry of the neural code (orthogonal vs. aligned, modular vs. mixed) is dictated by the statistical structure of the environment.