Opposing plasticity mechanisms in single neurons shape visual saliency assignment

This study reveals that single pyramidal neurons in the visual cortex autonomously assign visual saliency to unexpected inputs or omissions by employing divergent plasticity mechanisms that weaken adapted feedforward signals while strengthening generalized contextual feedback, thereby enabling saliency detection without requiring feedback-driven inhibition.

The Gist

Imagine your brain is a busy newsroom. Every second, it's flooded with thousands of reports (visual inputs) from the outside world. If the newsroom tried to read every single report with equal attention, it would be overwhelmed and miss the important stories. So, the brain has a smart editor: Visual Saliency. This editor decides what is "breaking news" (unexpected, important) and what is just "background noise" (predictable, boring).

For a long time, scientists thought this editor worked like a subtraction machine: Take the new input, subtract the old background, and what's left is the news. But there was a problem with this theory. The brain's wiring doesn't seem to have enough "subtractive" tools (inhibitory neurons) to do this math efficiently.

This paper discovers a brilliant, alternative solution. The brain doesn't subtract; it rebalances. It uses two opposing forces working inside the same neuron to highlight the unexpected.

Here is the story of how they found it, explained simply:

The Experiment: The "Occluded" Picture

The researchers (working with mice, but comparing results to monkeys and humans) showed the animals pictures of natural scenes.

1. The Full Picture: A clear image of a forest or a city.
2. The Occluded Picture: The same image, but with a gray square covering the top-left corner.

Crucially, the gray square was the exact same color as the screen when nothing was showing. So, when the gray square appeared, the neurons in that specific part of the brain received zero visual signal. It was a "blind spot."

The Discovery: The "Expert" Brain

They tested the mice in two states:

• The Novice: The mouse has never seen these specific pictures before.
• The Expert: The mouse has seen the "Full Picture" versions many times and learned them well.

Here is what happened when they looked at the brain cells (neurons) of the Expert mice:

1. The "Bored" Neurons (Familiar Inputs)

When the expert mouse saw the Full Picture it knew well, the neurons that usually fired up went quiet. They had become "bored" or adapted.

• Analogy: Imagine you hear the same song on the radio every morning. Eventually, you stop really listening to it; it fades into the background. The brain learned, "I know this, no need to shout about it."

2. The "Alert" Neurons (Contextual Inputs)

But when the Occluded Picture appeared (with the gray square), something magical happened. A different set of neurons suddenly lit up!

• Why? Even though the gray square blocked the direct view, the neurons could "see" the rest of the image surrounding the square. They knew what should be there based on the context.
• Analogy: Imagine you are reading a book, and someone covers a word with their hand. You don't need to see the word to know what it is; your brain fills it in based on the sentence. In this case, the brain didn't just "fill it in"—it actually got excited about the missing piece.

The "Opposing Plasticity" Mechanism

The paper reveals that inside a single neuron, two different learning rules are happening at the same time, pulling in opposite directions:

1. The "Turn Down" Rule: If a neuron gets the same direct signal over and over (the familiar full picture), it weakens its connection. It learns to ignore the predictable.
2. The "Turn Up" Rule: If a neuron receives signals from the surroundings (the context), it strengthens those connections. It learns to pay attention to the "big picture" context.

The Result:

• Familiar Scene: The "Turn Down" rule wins. The neuron stays quiet because the input is predictable.
• Missing Piece (Occlusion): The "Turn Down" rule is irrelevant (because there is no direct input), but the "Turn Up" rule is active. The neuron fires strongly because the context says, "Something is missing here!"
• New Scene (Novelty): If you show the expert mouse a brand new picture it has never seen, the "Turn Down" rule hasn't kicked in yet, but the "Turn Up" rule (the strengthened context) is ready. The result? A massive burst of activity! The brain screams, "This is new and important!"

The "Surround Suppression" Secret

Why didn't the neurons fire when the full picture was shown?
The researchers found that when a neuron sees a full image, it gets "shushed" by its neighbors. This is called surround suppression.

• Analogy: Imagine a crowded room. If one person starts talking, everyone else quiets down to listen. But if the room is empty (the occluded part), that one person can shout, and everyone hears them.
• In the brain, the "full picture" activates neighbors that suppress the "context" neurons. But when the picture is blocked (occluded), the neighbors are quiet, so the "context" neurons are free to shout about the missing information.

Why This Matters

This discovery changes how we understand the brain:

1. No Math Needed: The brain doesn't need complex subtraction circuits to find what's important. It just needs to learn to ignore the boring stuff and amplify the context.
2. Universal Design: They found this same pattern in mice, monkeys, and humans. It's a fundamental rule of how we see the world.
3. Mental Health Clues: The authors suggest that if this "balancing act" goes wrong, the brain might start assigning importance to things that aren't important (like hearing voices or seeing things that aren't there), which could explain conditions like psychosis.

The Big Picture Metaphor

Think of your brain as a spotlight operator in a theater.

• The Old Theory: The operator tries to calculate the difference between the stage and the background to find the actor.
• The New Theory: The operator has two switches.
  • Switch A: "If the actor does the same move 100 times, dim the light." (Adaptation)
  • Switch B: "If the actor steps out of the light but the surrounding stage tells us they are there, crank the light up!" (Contextual amplification)

By using these two switches, the brain automatically highlights the unexpected, the missing, and the new, allowing us to navigate a complex world without getting overwhelmed.

Technical Summary

1. Problem Statement

The visual system must prioritize unexpected or salient inputs over predictable backgrounds to navigate complex environments. A prevailing theoretical framework suggests this "saliency assignment" occurs via subtractive computations, where contextual feedback (predictive signals) subtracts from feedforward sensory input to highlight deviations (prediction errors).

However, a fundamental anatomical and functional challenge exists in the visual cortex:

• Contextual feedback is predominantly amplifying, not subtractive.
• Feedback-driven inhibitory neurons required for strict subtraction are relatively sparse.
• The Question: How can the cortex compare feedforward input with complex contextual signals to detect deviations if both inputs are primarily excitatory, without relying on a sparse inhibitory mechanism to perform subtraction?

2. Methodology

The authors employed a multi-modal approach combining behavioral training, high-resolution imaging, and cross-species comparison to dissociate feedforward and contextual signals.

• Subjects: Awake mice (TIT2L-GCaMP6f x G35-3-Cre) expressing calcium indicators in Layer 2/3 pyramidal cells (PyCs). Data was also compared with existing datasets from monkeys (electrophysiology) and humans (fMRI).
• Stimuli:
  • Natural Scenes: Six natural images presented in two conditions: Non-Occluded (NO) and Partially Occluded (O).
  • Occlusion Paradigm: A gray occluder covered the top-left quadrant of the screen. Crucially, the occluder's luminance matched the inter-trial interval (gray screen), ensuring no luminance change occurred in the Classical Receptive Field (cRF) of neurons targeting that region. Thus, any response to the occluded image in the cRF must arise from contextual (extraclassical) inputs.
  • Surround Suppression/Annulus: Gratings and annuli of varying sizes to map receptive field properties.
• Experimental Design:
  • Familiarization: Mice were trained to detect familiar images (4 out of 6) via a water-reward licking task. Two images remained novel (untrained).
  • Imaging Sessions: Two-photon calcium imaging was performed in three states:
    1. Naive: Before training.
    2. Expert: After familiarization (passive viewing).
    3. Task: During active engagement in the detection task.
  • Targeting: Wide-field (WF) population receptive field (pRF) mapping was used to target two-photon (2P) imaging specifically to the cortical region representing the occluded quadrant.
• Analysis:
  • Separation Index (SI): Quantified the degree to which neurons responded exclusively to NO or O stimuli.
  • Decoding: Linear Discriminant Analysis (LDA) to test if image identity could be decoded from population responses.
  • Reverse Correlation: Used Shannon entropy to map the contribution of cRF vs. extraclassical receptive field (ecRF) inputs.

3. Key Contributions

The paper proposes a novel cellular mechanism for saliency assignment that does not rely on feedback-driven inhibition. Instead, it demonstrates that divergent plasticity rules within single pyramidal neurons dynamically optimize the contrast between feedforward and contextual inputs.

Core Mechanism:

1. Feedforward Adaptation: Repeated presentation of familiar feedforward inputs leads to selective weakening (adaptation/surround suppression) of responses.
2. Contextual Potentiation: Repeated exposure strengthens contextual (ecRF) inputs in a generalized manner across scenes.
3. Opposing Plasticity: These two mechanisms operate in opposite directions within the same neuron population, effectively creating a "difference" signal without subtraction.

4. Key Results

A. Divergent Plasticity in Single Neurons

• Opposing Trends: Upon familiarization, responses to non-occluded (feedforward) stimuli decreased significantly, while responses to occluded (contextual) stimuli increased.
• Neuronal Separation: In naive mice, many neurons responded to both stimulus types. After familiarization, the population split into two distinct, non-overlapping groups:
  • Feedforward Responders: Strongly suppressed by familiarization.
  • Contextual Responders: Strongly potentiated by familiarization.
• Single-Cell Dynamics: Chronically matched neurons showed that individual cells could shift from being feedforward-driven to context-driven, or vice versa, but strong responses to both input types were never expressed simultaneously in the same cell.

B. Mechanism of Separation: Surround Suppression & ecRF Inputs

• Surround Suppression: Neurons that responded to occluded images exhibited stronger surround suppression than those responding to non-occluded images.
  • Hypothesis: In a full scene, local inhibition suppresses "context-only" neurons. When the feedforward input is removed (occlusion), this inhibition is lifted, allowing the strengthened contextual inputs to drive the neuron.
• Response Latency: Responses to occluded images were significantly delayed (253 ms) compared to non-occluded images (189 ms), consistent with feedback-driven activation.
• Stimulus Specificity: While contextual responses generalized across scenes, they retained stimulus-specific information (decodable image identity), suggesting the learning of structured statistical regularities rather than simple feature repetition.

C. Novelty Detection and Saliency

• Novel Images: Responses to novel (unfamiliar) non-occluded images increased after training.
  • Explanation: Novel images did not undergo feedforward adaptation. However, they recruited the strengthened, generalized contextual inputs learned from familiar scenes. The coincidence of non-adapted feedforward drive and potentiated contextual drive resulted in amplified responses.
• Saliency Assignment: This mechanism allows the cortex to amplify:
  1. Omissions: When feedforward input is missing (occlusion), the strengthened context drives activity.
  2. Deviations: When a novel input violates the learned context, the lack of adaptation combined with strong context drives high activity.

D. Cross-Species Conservation

• Decoding of image identity from occluded scenes was successful in mice, monkeys, and humans.
• The pattern of decoding (high accuracy for within-condition, lower but significant for cross-condition) was remarkably similar across species and recording modalities (2P, MUA, fMRI), suggesting a conserved cortical principle.

5. Significance

• Redefining Predictive Coding: The findings challenge the classical view that predictive coding relies on feedback-driven inhibition to "subtract" predictions from sensory input. Instead, saliency arises from multiplicative amplification of deviations via opposing plasticity rules (weakening the expected, strengthening the context).
• Cellular Basis of Saliency: It identifies a specific circuit mechanism in Layer 2/3 pyramidal neurons where dendritic integration of contextual inputs and somatic adaptation of feedforward inputs work in concert to highlight "surprises."
• Clinical Implications: The authors suggest that disruptions in this "context-contrasting" plasticity could underlie psychiatric conditions like psychosis, where the brain may assign saliency to inappropriate inputs (hallucinations/delusions) due to a failure to properly weigh context against sensory input.
• Efficiency: This mechanism allows the visual system to efficiently detect both the presence of unexpected objects and the absence of expected objects (occlusions) using a unified, autonomous cellular process.

In summary, the paper demonstrates that the brain does not need complex subtractive circuits to detect deviations. Instead, individual neurons autonomously learn the statistical structure of their environment, weakening predictable inputs while strengthening contextual ones, thereby naturally amplifying the unexpected.