🤖 machine learning

Beyond Activation Alignment: The Geometry of Neural Sensitivity

This article introduces the Spectral Riemannian Alignment Score (S-RAS), a novel framework based on locally decodable information and Fisher geometry that complements traditional activation alignment methods by measuring neuronal sensitivity to small perturbations, thereby enabling more robust comparisons of biological and artificial neural representations across diverse tasks and datasets.

Original authors: Amirhossein Yavari, Farnaz Zamani Esfahlani

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amirhossein Yavari, Farnaz Zamani Esfahlani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand two different chefs, both preparing a complex dish, say, lasagna.

The Old Way: Comparing the Finished Plates
Traditionally, scientists compare how two neural networks (or a human brain and a computer) "think" by examining their final output or the activity pattern they generate when viewing an image. This is comparable to looking at the two finished lasagna plates. If the plates look identical—the same shape, the same cheese distribution, the same sauce swirls—scientists say: "Great! These two chefs used the same recipe."

Methods like RSA, CCA, and CKA are the tools used to measure how similar these "plates" look.

The Problem: The "Hidden" Recipe
The authors of this paper argue that just because the final plates look the same, it does not mean the chefs used the same ingredients or the same technique to get there.

Chef A might have chopped the vegetables very finely and stirred the sauce gently.
Chef B might have used a food processor and a high-speed blender.

If you only look at the finished plate, you miss the fact that Chef A's dish is sensitive to tiny changes in the vegetable chopping technique, while Chef B's is not. If you place a tiny speck of dust on the plate, Chef A's dish might fall apart, while Chef B's remains unchanged. The old methods cannot detect this difference because they only look at the "big picture" of the activity, not at how the system responds to tiny perturbations.

The New Idea: The "Sensitivity Map"
This paper introduces a new method to compare these systems. Instead of just looking at the finished plate, they want to see a sensitivity map.

Imagine you have a magical magnifying glass that shows you how much the lasagna wobbles when you poke it at a specific spot with a toothpick.

If you poke the cheese, does the whole thing wobble?
If you poke the sauce, does it stay calm?

The authors suggest that we should compare these "wobble maps" (which they call local sensitivity). They ask: "If we make a tiny, specific change to the input (such as changing the lighting on a photo by a fraction of a percent), how does the system's internal representation change?"

How They Do It (The Analogy)

The Subspace (The "Poke Zone"): You cannot poke the lasagna everywhere at once. Therefore, the researchers select a specific "family" of pokes. Perhaps they are only interested in poking the edges of the lasagna or only the center. In the paper, this is referred to as the "stimulus coordinate subspace."
The Map (The "Fisher Metric"): They calculate a mathematical map that shows exactly how sensitive the system is to these specific pokes. This map tells them which directions are amplified (the system reacts strongly) and which are suppressed (the system ignores them).
The Score (S-RAS): They take these maps and compare them using a special ruler called S-RAS (Spectral Riemannian Alignment Score). This score tells them how similar the sensitivity of two systems is, not just how similar their final outputs are.

What They Found (The Results)
The authors tested this on both computer brains (Artificial Neural Networks) and real mouse brains.

Finding Layers: When they looked at two different computer networks trained separately, they could use their new "sensitivity map" to figure out which layer in Network A corresponds to which layer in Network B. It worked almost as well as the old methods, proving that their new idea is valid.
The "Robustness" Test: They trained some networks to be "robust" (resistant to deception) and others normally. The old methods could not always detect the difference in how these networks processed information. But the new sensitivity maps showed a clear difference: the robust networks reacted to tiny changes in a completely different way than the normal ones.
Mouse Brains: They examined recordings from the visual cortex of a mouse (the part of the brain that sees). They found that the sensitivity of the mouse brain to certain light patterns (such as the angle of a stripe) could be mapped and compared, just like with computer networks. They discovered that the brain's "sensitivity map" changes depending on which specific light pattern is being viewed.

The Core Message
This paper says: "Do not just look at the final answer. Look at how the system reacts to tiny pokes."

By mapping how sensitive a system is to small changes in specific directions, we gain a much deeper understanding of how it works, not just what it produces. It is the difference between judging a car by how fast it goes (the old way) and judging it by how it handles a small bump in the road (the new way). Sometimes two cars drive at the same speed, but one handles bumps like a tank, while the other bounces like a balloon. This new method helps us see that difference.

Technical Summary: Beyond Activation Alignment: The Geometry of Neural Sensitivity

Problem Statement

Activation alignment metrics such as Representational Similarity Analysis (RSA), Canonical Correlation Analysis (CCA), and Centered Kernel Alignment (CKA) are standard tools for comparing biological and artificial neural representations. These methods primarily quantify the geometric similarity of stimulus-induced activation patterns. However, a critical limitation exists: agreement at the level of global readouts does not determine how a system utilizes local stimulus evidence. Representations may match in activation space yet differ significantly in their sensitivity to small perturbations. Furthermore, activation-based similarity measures are not invariant to simple post-hoc transformations (e.g., fixed subset translations followed by global mean centering) that can alter similarity scores without changing the underlying encoding process. While decoder-based perspectives and local geometry approaches have been proposed to address this issue, they often fail to precisely specify how information should be aggregated across stimuli or which dataset summaries suffice to characterize discrimination tasks.

Methodology

The authors propose a framework based on locally decodable information, focusing on a representation's ability to discriminate small perturbations within a specified stimulus coordinate subspace under noise.

1. Local Perturbation Tasks and Fisher Geometry

The framework treats a neural representation as a differentiable mapping $f: \mathbb{R}^d \to \mathbb{R}^m$ . Local sensitivity is characterized by the pullback metric $M_f(x) = J_f(x)^\top J_f(x)$ , where $J_f$ is the Jacobian matrix.

Task Definition: A local discrimination task consists of distinguishing between a stimulus $s$ and a perturbed stimulus $s + \eta$ given noisy population activity. The squared discriminability $d'^2$ is determined by the Fisher metric $I_{f,\psi}(s) = J_{hf}(s)^\top \Sigma^{-1} J_{hf}(s)$ , where $\Sigma$ is the noise covariance.
Dataset Summary: To move from pointwise sensitivity to a representation comparison, the authors define a Projected Second-Moment Local Task Family. By fixing a family of coordinates (parameterized by $\psi$ ) and a perturbation subspace (defined by an orthonormal matrix $P$ ), they average the local Fisher metric over the stimulus distribution.
The Operator: The minimal, complete dataset summary for this family is the Projected Expected Fisher Operator:
$F_{f,\psi,P,\Sigma} := \mathbb{E}_{s \sim \nu} [P^\top J_{hf}(s)^\top \Sigma^{-1} J_{hf}(s) P]$
In the case of isotropic noise, this reduces to the Projected Expected Pullback Metric $G_{f,\psi,P}$ . This operator fully characterizes the expected discriminability for any second-moment perturbation family defined by $P$ .

2. Decomposition: Gain and Shape

The authors decompose the summary operator into:

Gain ( $\gamma$ ): The total expected sensitivity (trace of the operator).
Shape ( $\hat{G}$ ): The directional allocation of sensitivity (trace-normalized operator).
This enables the comparison of representations based on total sensitivity versus the specific directions in stimulus space they amplify or suppress.

3. Comparison Metric: S-RAS

To compare these positive definite (SPD) summaries, the authors use the Affine-Invariant Riemannian Metric (AIRM) on the manifold of SPD matrices. They define the Spectral Riemannian Alignment Score (S-RAS):
$S\text{-}RAS_P(f, g) = \exp\left[ -\frac{1}{\sqrt{k}} d_{AIRM}(\tilde{G}_{f,P}, \tilde{G}_{g,P}) \right]$
where $\tilde{G}$ is a trace-scaled SPD lifting used for regularization. Theoretical analyses show that S-RAS provides a uniform multiplicative bound over the family of lifted task values and limits the deviation in expected discriminability between two representations.

Main Contributions

Formalization of Local Discrimination: The paper defines second-moment local perturbation discrimination tasks and identifies the projected expected Fisher/Pullback operator as the minimal complete dataset summary for these tasks.
S-RAS Metric: A geometry-aware comparison method (S-RAS) is introduced using the log-spectral distance on SPD matrices, providing a dimension-normalized similarity score with theoretical guarantees for task value bounds.
Empirical Validation: The framework is validated across artificial and biological systems, demonstrating utility in layer correspondence, transferable probes, robust training analysis, and biological data.

Experimental Results

1. Layer Matching (Artificial Networks)

Using independently trained Tiny10-CNNs, the authors tested whether S-RAS could recover expected layer correspondence.

Results: S-RAS achieved meaningful correspondence (85.3% accuracy at $K=768$ with random perturbations; 93.8% with whitened PCA) and approached the performance of strong activation baselines like linear CKA (94.4%).
Comparison: S-RAS outperformed pointwise local geometry controls (pointwise AIRM and MSA-like baselines), suggesting that aggregating expected task-family operators is superior to averaging pointwise metric deviations.

2. Transferable Class-Conditional Probes

The authors tested whether sensitivity contrasts could be externalized into finite perturbations that generalize to held-out images and models.

Results: Probes derived from class-conditional sensitivity contrasts (comparing ResNet-18 and Small ViT) produced significantly greater separation of held-out images (1.10) compared to controls with random contrasts (0.43), pooled sensitivity (0.31), and label permutation (0.03). This suggests the method captures shared local evidence structures that transfer across models.

3. Dissociations in Robust Training

Using ResNet-18 banks trained with ERM, PGD, TRADES, and MART, the authors investigated whether the framework could distinguish training regimes.

Results: Contrast-derived probes successfully separated standard models from robust models (PGD, TRADES, MART) on held-out images. The method also revealed nuanced differences between robust training objectives (e.g., separating PGD from TRADES) and showed that probes are sensitive to specific changes in local evidence utilization, not just regime differences.

4. Biological Application (Allen Brain Observatory)

The framework was applied to mouse visual cortex data using static gratings with coordinates $(\theta, \rho, \phi)$ .

Results: In cross-area identity matching, the noise-aware Full-Fisher summary achieved a Top-1 accuracy of 0.458, significantly outperforming CKA (0.288) and naive geometry. In the more challenging VISp-depth benchmark, Full Fisher (0.466) remained competitive with CKA (0.483).
Insight: The results indicate that the structure of biological sensitivity depends on both total noise-aware sensitivity and directional allocation. The choice of stimulus coordinate family (e.g., $\rho + \phi$ versus the full set) has a substantial impact on matching performance, underscoring the "family-relative" nature of the metric.

Significance and Claims

The paper claims that activation-based comparisons are most informative for stimulus-induced organization or the structure of downstream readouts but are insufficient to determine whether representations rely on the same local stimulus evidence. The proposed framework offers a complementary perspective by asking whether two representations make the same small stimulus directions discriminable on average.

Key claims include:

Family-Relative Similarity: Agreement is defined relative to a specific perturbation family; representations may agree under one family but not another.
Robustness to Transformations: Since the method relies on Jacobian matrices rather than activation point clouds, it is insensitive to post-hoc shifts that are constant with respect to the stimulus (e.g., global mean centering), thereby addressing a known limitation of CKA.
Empirical Utility: The framework successfully recovers layer correspondences, generates transferable diagnostic probes, reveals dissociations in robust training, and uncovers family-dependent sensitivity in biological data.

The authors remain modest, noting that the framework is local, dataset-level, and second-moment based. It does not capture image-conditioned optima, higher-order effects, or strongly nonlinear behavior under finite perturbations. They conclude that broader benchmarks are required to determine when dataset-level sensitivity geometry provides distinctive scientific insights compared to existing methods.