🧬 biology

Cross-Species RSA Reveals Conserved Early Visual Alignment but Divergent Higher-Area Rankings Across Human fMRI and Macaque Electrophysiology

This study demonstrates that while early visual cortical alignment is robust and conserved across human fMRI and macaque electrophysiology for various learning rules, higher-area alignment diverges significantly between species due to differences in model capacity and stimulus domains rather than the learning rules themselves.

Original authors: Nils Leutenegger

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nils Leutenegger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to figure out how different types of "learning" shape the brain's visual system. Scientists have built computer models (artificial brains) that learn in different ways, similar to how a human, a dog, or a robot might learn to recognize a cat.

This paper asks a big question: Do the rules that make these computer brains look like a human's brain also make them look like a monkey's brain?

Here is the breakdown of the study using simple analogies:

1. The Experiment: Testing Five "Learning Styles"

The researchers took five different computer models. Each model was built with the same "hardware" (architecture) but taught using a different "learning rule":

Backpropagation (BP): The standard, math-heavy way computers usually learn (like a strict teacher correcting every mistake).
Feedback Alignment (FA): A slightly looser version of the above.
Predictive Coding (PC): The model tries to guess what comes next and learns from its mistakes (like predicting the weather).
STDP: A rule based on timing, mimicking how real neurons fire when they happen close together in time (like a biological spark).
Random Weights: A model that was never taught anything; it just has random connections (like a baby with no experience).

They tested these models against two groups:

Humans: Using fMRI scans (which are like taking a blurry, low-resolution photo of the whole brain).
Macaque Monkeys: Using electrodes (which are like high-definition microphones listening to individual neurons).

2. The First Discovery: The "Early Vision" Match

When the models looked at simple, early visual information (like edges and basic shapes, similar to the brain's V1 area), the results were very clear.

The Monkey Advantage: The monkey data was much clearer (higher signal-to-noise ratio) than the human data. It's like comparing a grainy black-and-white photo to a 4K video. The models matched the monkey brain much better than the human brain simply because the monkey data was "sharper."
The Winners: In both humans and monkeys, the models that used biological timing rules (STDP) and predictive guessing (PC) did the best job at matching the early visual cortex.
The Loser: The standard computer learning method (Backpropagation) was actually the worst at matching the early visual cortex in both species.
The Surprise: The "Random" model (the one with no training) was surprisingly good at matching the human V1, but it wasn't the top performer for the monkeys. The monkeys preferred the "biological" learning rules.

The Takeaway: The way early vision is organized seems to be a fundamental trait shared by humans and monkeys. If you want a computer model to look like a real brain's early vision, you shouldn't use standard computer training; you should use rules that mimic how real neurons talk to each other.

3. The Second Discovery: The "High-Level Vision" Confusion

When the researchers looked at the higher-level parts of the brain (the IT area), which handles complex object recognition (like knowing a picture is a "dog" and not just "fur and ears"), the results got messy.

No Clear Pattern: There was no agreement between humans and monkeys on which learning rule worked best. It was like asking two people to rank their favorite ice cream flavors, and they gave completely different lists with no overlap.
Why? The paper suggests this isn't necessarily because humans and monkeys think differently. It's because the computer models were too small and the training data was too limited.
- The Analogy: Imagine trying to judge a chef's ability to cook a complex banquet (High-Level Vision) by only giving them a tiny kitchen and a few ingredients. No matter what cooking style they use, they all fail to make a great banquet.
- The Proof: When the researchers tested a much larger, pre-trained computer model (ResNet-50) that had seen millions of images, it suddenly became very good at matching the monkey's high-level brain. This proves that the "failure" of the smaller models wasn't about the learning rule; it was just that the models were too weak to handle the complex task.

4. The "Stimulus" Problem

The study also noted a tricky problem: The humans and monkeys were shown different pictures.

Humans saw a wide variety of everyday objects.
Monkeys saw some textures and some specific objects.
The Result: Because the "test questions" were different, it's hard to say for sure if the ranking differences were due to the species or just the pictures they were shown. The paper suggests that if we used the exact same pictures for both, the results might change.

Summary

Early Vision (Simple shapes): The rules that mimic real biology (STDP and Predictive Coding) work best for both humans and monkeys. This is a solid, shared truth.
High-Level Vision (Complex objects): We couldn't tell if the rules mattered because the computer models were too small and weak. When we used a bigger, stronger model, it worked much better, suggesting the problem was the model's size, not the learning rule.
The Big Picture: The brain's early visual system is robust and similar across species, but our ability to test the "higher" parts of the brain is currently limited by how small our computer models are and what pictures we show them.

Technical Summary: Cross-Species RSA Reveals Conserved Early Visual Alignment but Divergent Higher-Area Rankings

Problem Statement
Computational neuroscience has established that deep neural networks (DNNs) trained with backpropagation (BP) reproduce the representational hierarchy of the primate ventral visual stream. However, most model–brain comparisons are confined to a single species (typically human fMRI), leaving it unclear whether alignment patterns generalize across species and measurement modalities. A companion study by the author [Leutenegger, 2026] found that untrained random-weights and local learning rules (STDP, Predictive Coding) outperformed BP in aligning with human V1, while all rules converged at IT. This study addresses the cross-species question: does this pattern hold when evaluated against macaque electrophysiology, which offers single-neuron resolution and higher signal-to-noise ratios (SNR) than fMRI?

Methodology
The study employs Representational Similarity Analysis (RSA) to compare five learning rule conditions against neural data from both humans and macaques.

Models: The authors use identical model weights from the companion study [Leutenegger, 2026]: a custom 3-layer CNN trained on CIFAR-10 for 40 epochs. The five conditions are: Backpropagation (BP), Feedback Alignment (FA), Predictive Coding (PC), Spike-Timing-Dependent Plasticity (STDP), and an untrained Random Weights baseline.
Macaque Datasets:
- V4/IT: MajajHong2015 (multi-electrode array, 88 V4 and 168 IT neurons, 3,200 HVM object presentations).
- V1/V2: FreemanZiemba2013 (single-unit recordings, 102 V1 and 103 V2 neurons, 135 texture stimuli).
Human Dataset: THINGS-fMRI (720 stimuli, 3 subjects), with RSA values for V1, V2, LOC, and IT taken directly from the companion study.
Capacity Control: A pretrained ResNet-50 (ImageNet) was evaluated to assess the impact of model capacity and training data richness.
Analysis Pipeline:
- RSA: Feature extraction, Representation Dissimilarity Matrix (RDM) construction using correlation distance, and Spearman $\rho$ scoring between model and neural RDMs.
- Cross-Species Comparison: Kendall's $\tau$ was computed between human and macaque $\rho$ -vectors for each learning rule. Due to $n=5$ rules, statistical power is limited to detecting perfect correlations ( $\tau = \pm 1.0$ ).
- Stimulus Control: Model RSA was computed on both THINGS and macaque stimuli to determine if ranking differences stem from species or stimulus domain.

Key Results

Higher Absolute Alignment in Macaque: Macaque alignment values ( $\rho = 0.15–0.30$ at V1/V2) substantially exceed human fMRI values ( $\rho = 0.01–0.08$ ), consistent with the higher SNR of electrophysiology. However, all values remain well below noise ceilings (0.52–0.79), indicating significant room for improvement.
Conserved Early Visual Alignment: At macaque V1/V2, STDP ( $\rho \approx 0.30$ ) and PC ( $\rho \approx 0.28$ ) achieved the highest alignment, outperforming BP, FA, and Random weights. This mirrors the qualitative pattern found in human V1, where local learning rules outperform BP, confirming that early visual alignment is robust across species and modalities.
Divergent Higher-Area Rankings: At macaque IT, learning rule rankings showed no detectable correlation with human rankings ( $\tau = 0.00, p = 1.00$ ). However, the authors note this null result is expected given the low statistical power of $n=5$ (only $\tau = \pm 1.0$ is significant).
Capacity and Data Limitations: The ResNet-50 control achieved $\rho = 0.25$ at macaque IT, substantially outperforming all custom CNN conditions ( $\rho = 0.07–0.14$ ). This suggests that the "convergence" of learning rules at IT observed in the companion study may reflect architectural and data limitations (small CNN, CIFAR-10) rather than a fundamental property of learning rules.
Stimulus Domain Confounds: The sharp drop in alignment from V2 to V4 is confounded by a switch in datasets (texture stimuli vs. object images). Stimulus control analysis revealed that IT rankings showed a weak inversion ( $\tau = -0.40$ ) across stimulus sets, suggesting that cross-species ranking differences at IT may be partially driven by stimulus domain (THINGS vs. HVM objects) rather than species differences.

Significance and Claims
The paper claims that early visual alignment is qualitatively conserved across human fMRI and macaque electrophysiology. Specifically, local learning rules (STDP, PC) consistently outperform backpropagation in early visual areas (V1/V2) across both species, suggesting that the architectural inductive bias favoring these rules is not an artifact of fMRI methodology.

Conversely, the study finds no detectable cross-species correspondence in higher-area (IT) rankings within its specific experimental setup. The authors modestly attribute this to a combination of factors:

Statistical Power: The small number of learning rules ( $n=5$ ) limits the ability to detect correlations.
Model Capacity: The custom CNN likely lacks the capacity to capture the object-level representations encoded in macaque IT, a limitation highlighted by the superior performance of ResNet-50.
Stimulus Mismatch: Differences in stimulus domains between datasets confound the interpretation of IT rankings.

The authors conclude that while early visual alignment is robust, higher-area alignment is modulated by model capacity, training data richness, and stimulus domain. These results motivate future cross-species studies utilizing shared stimulus sets, larger architectures, and more learning rule conditions to enable adequately powered ranking comparisons.