Technical Summary: Meta-Representational Predictive Coding (MPC)

Problem Statement

Self-supervised learning (SSL) has become a dominant paradigm in machine intelligence, yet current approaches rely on biologically implausible mechanisms, specifically the backpropagation of errors for credit assignment and feedforward inference. While predictive coding (PC) offers a biologically plausible alternative via local plasticity rules and message passing, existing PC formulations face a dichotomy:

1. Unsupervised PC: Learns a generative model of raw sensory input (e.g., predicting pixel values), which is computationally expensive and akin to "generative AI."
2. Supervised PC: Learns a mapping from inputs to target labels, requiring human annotation and suffering from the drawbacks of supervised learning.

There is a gap for a neuroscience-informed self-supervised learning (NeuroSSL) framework that avoids backpropagation, does not require reconstructing high-dimensional raw inputs, and does not rely on human labels or contrastive negative sampling.

Methodology: Meta-Representational Predictive Coding (MPC)

The authors propose Meta-Representational Predictive Coding (MPC), an encoder-only learning scheme grounded in the Free Energy Principle. Instead of predicting raw sensory data, MPC learns to predict latent representations across parallel neuronal streams.

Core Architectural Principles

1. Encoder-Only Architecture: MPC inverts the standard predictive processing premise. Rather than a top-down generative decoder predicting sensory input, the system focuses on bottom-up representation acquisition. It learns a generator of latent states (an encoder) without a decoder for raw pixels.
2. Multi-Stream Processing (Foveal/Parafoveal/Peripheral): Inspired by the functional anatomy of the visual system, the architecture comprises multiple parallel streams:
   • Foveal Streams: Process high-resolution, detailed patches (e.g., 8x8 pixels).
   • Parafoveal Streams: Process medium-resolution patches (e.g., 16x16 pixels).
   • Peripheral Streams: Process low-resolution, coarse-grained patches (e.g., 24x24 pixels).
   • These streams interact via cross-stream prediction, where each stream attempts to predict the activity dynamics of the others. This creates a "meta-representational" narrative where complementary views of input patterns must yield embeddings that resonate with one another.
3. Active Inference via Saccades: The model employs an epistemic glimpse planning mechanism. Instead of processing a full image at once, the model performs a sequence of saccades (eye movements) to sample "glimpses" of the input.
   • Input Filtering: Raw inputs are pre-processed by Lateral Geniculate Nucleus (LGN)-inspired filters (Difference-of-Gaussians) to extract contrast-invariant features.
   • Saccade Planner: A reflex-based planner selects the next fixation point by optimizing a focus map. This map balances bottom-up saliency (local structural energy) with top-down epistemic uncertainty (prediction errors from previous glimpses). The planner actively seeks regions of high uncertainty to reduce long-term free energy.
4. Learning Dynamics:
   • Free Energy Minimization: The system minimizes Variational Free Energy (VFE) defined over the latent states. The energy function includes terms for cross-representation prediction (inter-stream), residual energy (intra-stream), and synaptic priors.
   • Neuronal Dynamics: Neuronal activities follow gradient flows of free energy, driven by intra-stream and inter-stream prediction errors.
   • Synaptic Plasticity: Learning occurs via local Hebbian rules. Synaptic weights are updated based on the correlation between pre-synaptic activity and post-synaptic prediction errors. No global error backpropagation is used.
   • Latent Glimpse Path Integration: To form a global representation of a stimulus, the model iteratively integrates latent codes from sequential glimpses, inspired by spatial mapping in biological grid cells, avoiding the need to store full histories of gaze paths.

Key Contributions

1. Biologically Plausible SSL Framework: The paper formulates a self-supervised learning scheme that relies on local neural statistics and layer-wise parallel inference, eliminating the need for backpropagation.
2. Encoder-Only Predictive Coding: It demonstrates that a generative model of latent states can be learned without predicting raw sensory inputs. By predicting latent activity across visual streams, the model obviates the need for contrastive learning mechanisms (positive/negative pairs).
3. Active Perception Integration: The framework incorporates an enactive perspective where the coordinates of the model's "gaze" (saccades) inform the dynamics of neuronal units. This drives the learning of representations through sequences of decisions to sample informative portions of the sensorium.
4. Scalability: The scheme is agnostic to the dimensionality of sensory data, as it processes subsets of variables (glimpses) rather than the entire high-dimensional input simultaneously.

Experimental Results

The authors evaluated MPC on four datasets: MNIST, Kuzushiji-MNIST (K-MNIST), NYU NORB (natural 3D objects), and ETH-80.

• Downstream Classification: MPC representations were evaluated using a K-Nearest Neighbor (KNN) classifier. MPC achieved competitive performance, often outperforming fully supervised MLPs and other self-supervised baselines like I-JEPA and standard Generative PC (GPC). On MNIST, MPC-epi achieved ~98.1% accuracy, comparable to supervised CNNs.
• Reconstruction: While MPC is encoder-only, a separate decoder was retrofitted to the frozen latent codes. MPC facilitated effective learning of a decoder, yielding Structural Similarity Index (SSIM) scores superior to whole-image GPC models on natural images (NORB), producing sharper and more coherent reconstructions.
• Zero-Shot Generalization: In cross-dataset experiments (e.g., training on K-MNIST and testing on MNIST), MPC demonstrated robust zero-shot generalization, suggesting it learns universal geometric primitives (strokes, edges) rather than dataset-specific artifacts.
• Sample Efficiency: MPC showed superior sample efficiency compared to supervised backpropagation models, maintaining consistent performance even with very small training sets (e.g., 100 samples), likely due to its ability to treat each data point as a "mini sensorium" via saccadic sampling.
• Receptive Fields: Visualizations of learned receptive fields showed foveal units resembling classical sparse coding filters (strokes) and peripheral units encoding larger object chunks, consistent with biological visual hierarchies.

Significance and Claims

The paper claims that MPC represents a significant step toward NeuroAI and biomimetic intelligence by:

• Inverting Predictive Processing: Shifting the goal from reconstructing raw sensory data to acquiring distributed latent representations through mutual prediction among specialized streams.
• Solving the Credit Assignment Problem: Providing a mechanism for learning distributed representations using only local plasticity rules and message passing, sidestepping the biological implausibility of backpropagation.
• Bridging Active Vision and Learning: Demonstrating how active sensing (saccades) can drive the acquisition of robust representations without requiring explicit supervision or contrastive negative sampling.
• Scalability and Efficiency: Offering a pathway to learn complex representations that are invariant to lighting and viewpoint, capable of generalizing to novel physical structures, without the computational burden of modeling high-dimensional pixel spaces.

The authors conclude that while MPC is a loose inspiration of biological circuitry, it provides a computationally viable instantiation of neuroscience-informed self-supervised learning, suggesting that abstract encodings can emerge purely from predictions and prediction errors across complementary sensory streams.