Local Synaptic Rules Can Implement a SIGReg Gradient Without Backpropagation

Technical Summary: Local Synaptic Rules Implementing a SIGReg Gradient

Problem Statement
The central problem addressed is how biological neural networks can learn useful, high-dimensional representations without access to explicit labels or the global error signals required by backpropagation. While Self-Supervised Learning (SSL) objectives like SIGReg and VICReg have proven effective in artificial systems by maximizing similarity between input views while preventing dimensional collapse via variance and covariance regularization, these methods lack biological plausibility. They typically require global gradient calculations, weight transport, and non-local error signals that do not exist in known neurobiological mechanisms. This work seeks to bridge this gap by demonstrating that canonical, locally available synaptic rules can mathematically implement the exact gradient of a SIGReg-like objective.

Methodology
The authors propose a circuit model where learning is driven entirely by local synaptic factors: pre- and post-synaptic firing rates, local firing statistics, and the temporal contiguity of natural sensory streams. The model integrates two primary mechanisms:

1. Spike-Timing-Dependent Plasticity (STDP+): The potentiation arm of STDP is modeled as a three-factor Hebbian rule. It strengthens synapses when a presynaptic neuron fires shortly before a postsynaptic neuron. Mathematically, this corresponds to gradient descent on a forward prediction loss ($L_{pred}$), where the postsynaptic propensity at time $t+1$ acts as a fixed "teaching signal" (stop-gradient) for the weights at time $t$.
2. Flashlight Homeostatic Plasticity: To prevent dimensional collapse (a common failure mode in SSL), the model employs a "flashlight" population of neurons (analogous to cerebellar granule cells) that project the encoder's output $h$ via a fixed, random, mixed-sign matrix $A$.
   • Variance Regulation: Each flashlight neuron emits a retrograde signal proportional to the mismatch between its variance and a target (unity). This implements the diagonal (variance) component of the collapse-prevention term.
   • Covariance Regulation: Lateral inhibition from interneurons (basket/stellate cells) driven by correlated flashlight pairs implements the off-diagonal (covariance) component.
   • Together, these homeostatic mechanisms implement the gradient of a weak-SIGReg objective ($L_{Weak}$), which penalizes deviations of the projected covariance from the identity matrix.

The core theoretical contribution is the direct construction showing that the sum of the STDP+ update and the homeostatic update equals the negative gradient of the full SIGReg objective ($L_{SIGReg} = L_{pred} + \lambda L_{Weak}$). The authors explicitly note that the STDP depression arm (STDP−) acts as a sparsifying projection to enforce non-negative weights ($W_{ij} \geq 0$) but, on its own, destabilizes learning and is excluded from the primary successful conditions.

Key Contributions

• Mathematical Equivalence: The paper proves that STDP+ and flashlight homeostasis, when combined, implement gradient descent on the SIGReg objective without backpropagation, global error signals, weight transport, or label information.
• Biological Plausibility: The mechanism relies solely on locally available factors (pre/post-synaptic activity, local gain, and variance/covariance statistics) and avoids iterative inference loops (unlike sparse coding), operating in a strictly online, single-pass manner.
• Handling Non-Negativity: The work clarifies the role of STDP− not as a gradient step for the objective, but as a mechanism for projecting weights onto the non-negative orthant, though it notes that a stable implementation requires an additional inhibitory population not yet fully modeled.

Experimental Results
The authors validated the mechanism on two tasks:

1. Synthetic Clustering: On a 3-class dataset with high-dimensional noise, the network was exposed to inputs either in class-contiguous temporal order or random order.
   • Ordered Input: The network learned to separate classes, achieving a Cluster Separation Ratio (CSR) of 2.49, a roughly threefold improvement over the baseline.
   • Random Input: With random ordering, the CSR remained near baseline (0.83), demonstrating that the learned structure arises solely from temporal contiguity.
2. MNIST Linear Probe: A two-layer network trained on MNIST using only STDP and homeostatic plasticity (no labels during training) achieved 87.3% linear-probe accuracy.
   • Ablations: Removing homeostasis caused performance to collapse to near chance (18.5%). Randomizing the input order reduced accuracy to 86.5%, indicating that homeostasis alone provides a "whitening floor," while temporal ordering provides a marginal but consistent gain.
   • STDP− Impact: Including STDP− with weight clipping significantly degraded performance (dropping to ~59–69%), confirming the theoretical claim that STDP− destabilizes the system without a compensatory inhibitory population.

Significance and Claims
The paper claims that this work establishes a direct correspondence between independently observed biological synaptic rules and state-of-the-art self-supervised learning objectives. It argues that the brain may utilize these canonical mechanisms (STDP and homeostasis) to compute the exact gradients required for learning useful representations in an unsupervised manner.

The authors emphasize that this is not a deep mathematical derivation but a "direct construction" where each biological rule is differentiated to match a specific term in the SIGReg objective. The significance lies in expanding the toolkit of biologically plausible operations: showing that SIGReg is not just an abstract algorithm but can be realized through local, biologically grounded interactions. The work suggests that temporal contiguity in natural sensory streams serves as the positive-pair signal for SSL, eliminating the need for explicit data augmentation or negative sample construction.

The paper remains modest regarding depth scaling, acknowledging that applying these decorrelation objectives at every layer leads to a "whitening cascade" where subsequent layers receive structureless inputs, resulting in performance degradation in deeper networks. It suggests that future work should explore convolutional variants to handle spatial scales and the integration of specific inhibitory populations to stabilize the non-negative projection.