Technical Summary: Platonic Representations in the Human Brain

Problem Statement

The Strong Platonic Representation Hypothesis posits that independently trained artificial neural networks converge toward geometrically similar representations, allowing embeddings to be translated across models via a universal latent space without paired data or shared inputs. While evidence for this exists in artificial vision and language systems, it remains unknown whether this principle extends to biological neural systems.

Existing neuroscience methods for aligning brain data across subjects (e.g., hyperalignment) typically rely on shared stimuli to establish cross-subject correspondences or depend on external reference spaces (e.g., model-derived features). The central question addressed by this paper is whether a shared neural geometry can be recovered from independently learned brain representations alone, using only the intrinsic geometry of neural responses and without paired cross-subject samples or intermediate model representations.

Methodology

The authors propose a three-stage framework to test this hypothesis using the Natural Scenes Dataset (NSD), where 8 subjects viewed complex natural images. Crucially, the method uses subject-specific, non-shared images for learning and a held-out set of shared images exclusively for evaluation.

1. Learning Geometry-Preserving Embeddings (Subject-Specific)

For each subject $s$, a self-supervised encoder $f_s$ maps fMRI responses $X^{(s)}$ to a low-dimensional embedding space $Z^{(s)}$.

• Self-Supervision via Repetition: The method exploits repeated stimulus presentations. Instead of averaging repetitions (which assumes independence), the encoder treats them as noisy views of a stable latent signal.
• Pipeline:
  1. Voxel Reliability Weighting: Voxels are reweighted based on their correlation across repetitions to suppress noise.
  2. Linear Projection: Reliability-weighted responses are projected via PCA, followed by Multi-View Canonical Correlation Analysis (MCCA) to extract components shared across views.
  3. Nonlinear Refinement: A residual Multi-Layer Perceptron (MLP) refines the linear embedding. The model is trained using a contrastive InfoNCE loss and a cosine pull term to ensure that embeddings of the same image (across different repetitions) are close, while embeddings of different images are separated.
• Goal: To learn an embedding space $Z^{(s)}$ that reflects the intrinsic organization of the subject's stimulus responses without external guidance.

2. Pairwise Brain-to-Brain Translation

Given independently learned embeddings $\{Z^{(s)}\}$, the authors seek orthogonal transformations $R_{s \to t}$ to translate subject $s$'s space into subject $t$'s space without paired data.

• Pseudo-Correspondence: Since no shared images are available during training, the method constructs pseudo-matched pairs by clustering each subject's embedding space (K-means) and matching centroids based on pairwise similarity structures.
• Iterative Refinement: An iterative approach similar to Iterative Closest Point (ICP) refines the orthogonal Procrustes solution.
• Symmetrization: The resulting rotation is symmetrized to ensure $R_{t \to s} = R_{s \to t}^\top$, a requirement for global consistency.

3. Shared Latent Space Construction (Global Synchronization)

To move beyond pairwise translations and establish a single coherent coordinate system, the authors solve an orthogonal synchronization problem over the group $O(d)$.

• A block matrix of pairwise rotations is constructed.
• A spectral method recovers a single orthogonal transformation $R_s$ for each subject such that $R_{s \to t} \approx R_s R_t^\top$.
• This maps all subject embeddings into a shared latent space $Z$, enforcing cycle consistency across the subject graph.

Key Results

Within-Subject Performance

The self-supervised encoder successfully extracts stable stimulus representations from noisy fMRI data.

• Retrieval: The model achieves a Mean Rank of 5.28 (chance = 258) and R@1 of 0.82 in retrieving matching images across repetitions within the same subject.
• Comparison: This outperforms direct neural baselines (GLMsingle, PCA), multiview self-supervised methods (SplitAE, DeepCCA), and model-guided baselines (CLIP, DINOv2). The repetition-based self-supervision recovers stronger instance-level information than externally guided encoders.

Cross-Subject Translation (Unsupervised)

The method successfully translates embeddings between subjects using only unpaired data.

• Performance: On held-out shared images, the unsupervised orthogonal translation achieves a Mean Rank of 2.56 and R@1 of 0.78 across subject pairs.
• Baselines: This significantly outperforms "no-translation" controls (Mean Rank ~189) and optimal transport matching. It also outperforms model-guided baselines (e.g., CLIP, ViT) which rely on external feature spaces.
• Geometry: The high Representational Similarity Analysis (RSA) scores indicate that the subject-specific spaces are approximately isometric.

Shared Latent Space

Synchronizing pairwise rotations into a single shared space further improves performance.

• Improvement: Global synchronization reduces the average Mean Rank from 2.47 (pairwise) to 1.97 and increases R@1 from 0.79 to 0.83.
• Implication: This confirms that independently learned subject spaces are mutually compatible with a common coordinate system.

Model-Brain Alignment

When mapping the recovered neural embeddings to artificial neural network (ANN) layers via supervised linear maps:

• Ridge regression improves retrieval (Mean Rank ~5.86) compared to semi-orthogonal maps, indicating ANNs capture relevant stimulus information.
• However, these supervised mappings remain inferior to the unsupervised brain-to-brain translation (Mean Rank 1.97). This suggests that while ANNs share information with the brain, they do not share the same near-isometric geometric transformations observed across human subjects.

Significance and Claims

The paper claims to provide the first evidence that the Strong Platonic Representation Hypothesis holds for biological neural systems. Specifically:

1. Shared Geometry: Subject-specific fMRI representations in the human visual cortex are approximately isometric across individuals.
2. Universal Latent Space: These representations can be translated into a shared coordinate system using purely geometric transformations (orthogonal rotations) learned from intrinsic data, without paired cross-subject samples or external model anchors.
3. Practical Implications: This shared geometry suggests that data, encoders, and decoders learned from one subject could potentially be transferred to another, reducing the need for extensive subject-specific calibration in applications like fMRI-to-image decoding or synthetic neural data generation.

The authors remain modest regarding limitations, noting that the analysis relies on high-quality fMRI data with repetitions and that the unpaired translation procedure is sensitive to initialization. They do not claim that current ANNs perfectly replicate human geometry, but rather that the brain possesses a recoverable universal structure that models have yet to fully align with via isometric transformations.