Neural Functional Alignment Space: Brain-Referenced Representation of Artificial Neural Networks

This paper introduces the Neural Functional Alignment Space (NFAS), a brain-referenced framework that characterizes diverse artificial neural networks by modeling their layer-wise dynamics via Dynamic Mode Decomposition and projecting them into a biologically anchored coordinate system to reveal structured modality-specific clustering and cross-modal convergence.

The Gist

The Big Idea: Giving AI a "Brain Map"

Imagine you have three different types of chefs:

1. A French Chef (Vision AI) who only cooks with vegetables.
2. A Sushi Chef (Audio AI) who only works with fish.
3. A Pastry Chef (Language AI) who only bakes cakes.

If you ask them, "How good are you at cooking?" it's hard to compare them directly. One uses knives, another uses chopsticks, and the third uses ovens. They work in different ways, so comparing their "layers" (how they chop, mix, or bake) doesn't tell you if they are actually thinking like a human.

This paper proposes a new way to compare them. Instead of looking at their tools (the architecture), the authors created a "Human Brain Map" (called NFAS). They want to see: When these chefs cook, does their mental process look like what happens inside a human brain?

---

Step 1: Watching the Movie, Not the Frames

The Problem:
Usually, scientists compare AI by looking at one specific layer at a time. It's like judging a movie by looking at a single frozen frame. You miss the story.

The Solution (Dynamics):
The authors realized that an AI doesn't just "think" in one step; it thinks in a flow. Information travels from the first layer to the last, changing as it goes.

• The Analogy: Imagine a river flowing from a mountain to the ocean. You don't just look at the water at the top or the bottom; you watch how the river moves and changes shape as it travels.
• The Math Trick: They used a mathematical tool called Dynamic Mode Decomposition (DMD). Think of this as a "Stability Filter." It filters out the noise and finds the one "steady rhythm" or "core pattern" that the AI uses to process information, regardless of how many layers it has.

Step 2: The "Brain-Referenced" GPS

The Goal:
Now that they have the "rhythm" of how the AI thinks, they need to see if it matches a human.

The Method:
They took the AI's "rhythm" and projected it onto a Human Brain Map.

• The Analogy: Imagine the human brain is a giant city with 200 different neighborhoods (called ROIs or Regions of Interest). Some neighborhoods handle vision, some handle sound, and some handle language.
• The Test: They asked: "When the AI sees a picture of a cat, does its internal 'rhythm' light up the same neighborhoods in the human brain map that actually light up when a human sees a cat?"

If the AI's "cat thought" lights up the human "visual neighborhood," that's a good match. If it lights up the "language neighborhood," that's a weird mismatch.

Step 3: The "Consistency Score" (SNCI)

The Problem:
You have 45 different AI models. Some are huge, some are small. If you just average their scores, one weird, super-unique model might mess up the whole group's reputation.

The Solution:
They invented a score called SNCI (Signal-to-Noise Consistency Index).

• The Analogy: Imagine a choir. If everyone sings the same note, that's a strong "Signal." If everyone is singing different random notes, that's "Noise."
• How it works: The SNCI checks: Do all the Vision AIs agree on how to light up the visual part of the brain? If they all do it similarly, the score is high (High Consistency). If they all do it differently, the score is low. This helps separate the "real" brain-like patterns from the random quirks of specific AI models.

---

What Did They Find?

After testing 45 different AI models (Vision, Audio, and Language) against human brain data, they found some fascinating things:

1. The AI Neighborhoods are Real:

   • Vision AIs (like those that recognize faces) lit up the back of the brain (the visual cortex), just like humans do.
   • Audio AIs (like those that understand speech) lit up the side of the brain (temporal lobe), just like humans.
   • Language AIs lit up the front of the brain (prefrontal cortex), which handles complex thinking and planning.
   • Takeaway: Even though these AIs are built differently, they naturally organize themselves to match human brain geography.

2. The "Emotional" Connection:

   • Surprisingly, all types of AI (even those that don't "feel" emotions) showed a strong connection to the Limbic System (the brain's center for emotion and memory).
   • Takeaway: This suggests that to process information effectively, any intelligent system (biological or artificial) needs to connect with memory and context, just like we do.

3. A New Way to Measure Intelligence:

   • Before this, we compared AI to AI (Is Model A better than Model B?).
   • Now, we can compare AI to the Human Brain. We have a "Universal Coordinate System" where we can see exactly how close an AI is to thinking like a human, regardless of whether it's a robot, a chatbot, or a self-driving car.

The Bottom Line

This paper gives us a universal ruler made of human biology. Instead of asking "Is this AI smart?", we can now ask, "Does this AI think like a human?" And the answer is: Yes, but in very specific, organized ways that mirror our own brains.

Technical Summary

1. Problem Statement

The field of Brain-AI alignment currently faces three critical limitations:

• Fragmented Comparability: Existing methods typically compare Artificial Neural Networks (ANNs) to the brain at the level of individual layers or task-specific activations. These comparisons are often modality-specific (vision vs. language vs. audio) and lack a unified framework to compare heterogeneous models (e.g., Transformers vs. CNNs) on equal functional grounds.
• Static Representation: Current approaches often treat network layers as independent units, ignoring the intrinsic dynamical evolution of information as it propagates through network depth.
• Lack of a Unified Coordinate System: There is no principled, biologically anchored coordinate system that allows for the systematic positioning and geometric comparison of diverse AI models relative to distributed neural responses.

2. Methodology

The authors propose the Neural Functional Alignment Space (NFAS), a framework that models ANNs as dynamical systems and projects them into a brain-referenced coordinate system. The methodology consists of four main stages:

A. Modeling Network Dynamics (Depth-wise Evolution)

Instead of treating layers as static snapshots, the framework models the sequence of layer-wise embeddings $\{x_\ell\}_{\ell=1}^L$ as a dynamical trajectory.

• Linear Approximation: Leveraging the fact that modern architectures (ResNet, Transformers) use residual connections, the evolution of features across layers is approximated as a discretized ordinary differential equation: $X_2 \approx A X_1$, where $X_1$ and $X_2$ are depth-shifted snapshot matrices.
• Dynamic Mode Decomposition (DMD): The authors apply DMD to extract the linear operator $A$. This yields eigenvalues $\{\lambda_i\}$ and dynamic modes $\{\phi_i\}$.
• Stable Mode Extraction: The mode associated with an eigenvalue closest to unity ($|\lambda| \approx 1$) is selected as the Stable Mode. This represents the most consistent, non-decaying, and non-amplifying pattern of information transformation across the network depth.
• Stable Representation ($z$): The depth-averaged representation is projected onto this stable mode to generate a single, robust vector $z$ that characterizes the stimulus-specific computation of the entire network.

B. Constructing the Neural Functional Alignment Space (NFAS)

The stable representations are aligned with human brain activity to create a coordinate system.

• Hemodynamic Convolution: The temporal sequence of stable representations $z(t)$ is convolved with a canonical Hemodynamic Response Function (HRF) to simulate the BOLD signal ($\tilde{z}(t)$).
• Encoding Models: For each Region of Interest (ROI) in the brain (based on the Schaefer-200 atlas), a linear encoding model is fitted to predict fMRI responses from the AI's $\tilde{z}(t)$.
• Alignment Vector: The squared Pearson correlation between predicted and actual neural responses for each ROI forms an Alignment Vector ($c_m$). This vector serves as the coordinate of the AI model within the $R$-dimensional NFAS, where $R$ is the number of ROIs.

C. Signal-to-Noise Consistency Index (SNCI)

To analyze modality-level trends without being biased by outlier architectures, the authors introduce the SNCI.

• Definition: For a given modality, SNCI quantifies the mean alignment ($\mu_r$) relative to the cross-model variability ($\sigma_r$) across different architectures.
• Formula: $SNCI_r = \text{sigmoid}(\frac{\mu_r}{\sigma_r + \epsilon})$.
• Purpose: High SNCI indicates that a specific brain region is consistently aligned with a modality across diverse architectures, distinguishing robust functional patterns from architecture-specific noise.

D. Experimental Setup

• Datasets: 45 pretrained models (Vision, Audio, Language) evaluated against three large-scale fMRI benchmarks: Narratives, Algonauts 2021, and The Little Prince.
• Brain Data: Preprocessed fMRI data parcellated into 200 ROIs using the Schaefer atlas.

3. Key Results

A. Structured Geometry of the NFAS

• Modality Clustering: When projected into low-dimensional space (via PCA), models cluster distinctly by modality (Vision, Audio, Language).
• Statistical Significance: PERMANOVA confirmed a significant effect of modality on NFAS structure ($p < 0.001$). Inter-modality distances were substantially larger than intra-modality distances.
• Conclusion: The NFAS captures a geometric organization where modality is the primary axis of separation, while architecture-specific variations exist within these clusters.

B. Cortical Organization of Alignment

• Visual Models: Show strong alignment in posterior occipital cortex (early visual areas V6, V7).
• Audio Models: Exhibit dominant alignment in lateral temporal regions associated with higher-level auditory processing.
• Language Models: Peak alignment occurs in prefrontal control regions, consistent with executive and semantic integration systems.
• Observation: These patterns mirror the known functional specialization of the human cortex.

C. Functional Network-Level Convergence

• Shared Systems: Across all modalities, the Limbic network (emotion/memory) showed elevated alignment, suggesting a shared convergence with affective and memory-related systems.
• Specialization:
  • Vision models aligned with the Visual network.
  • Audio models aligned with attention-related systems.
  • Language models showed broad alignment across association networks (Default Mode and Control systems).
• Interaction: A two-way ANOVA revealed significant interactions between modality and functional network, proving that alignment differences are network-specific rather than uniform magnitude shifts.

4. Key Contributions

1. Neural Functional Alignment Space (NFAS): A novel, brain-referenced coordinate system that allows for the geometric comparison of heterogeneous AI models based on their functional similarity to distributed brain regions.
2. Dynamical Modeling Approach: Moving beyond static layer-wise comparison, the paper introduces a method to model network depth as a dynamical evolution, extracting a "stable mode" via DMD to represent intrinsic network properties.
3. Signal-to-Noise Consistency Index (SNCI): A new metric that quantifies cross-architecture consistency, filtering out architecture-specific outliers to reveal robust modality-level alignment patterns.
4. Empirical Validation: Demonstrated on 45 models across three modalities, providing evidence that AI representations converge toward specific, biologically plausible cortical systems.

5. Significance and Implications

• Unified Framework: NFAS provides a principled basis for evaluating AI not just by task performance, but by how its internal computational dynamics relate to biological intelligence.
• Beyond Architecture: The findings suggest that intelligence may reflect an underlying computational organization that transcends specific physical implementations (e.g., CNNs vs. Transformers), as different architectures converge on similar brain-referenced trajectories.
• Interpretability: By mapping AI models to specific cortical systems (e.g., Limbic, Visual, Control), the framework offers a new lens for interpreting what specific AI models "understand" or prioritize, bridging the gap between machine learning and cognitive neuroscience.
• Future Directions: This approach enables systematic positioning of new models, potentially guiding the design of more brain-aligned AI architectures and offering deeper insights into the computational principles of biological intelligence.