Topographic Constraints Shape Brain-Like Component Structure in Auditory Models

Technical Summary: Topographic Constraints Shape Brain-like Component Structure in Auditory Models

Problem Statement
Topographic organization is a fundamental feature of the brain, where neurons with similar response properties cluster spatially. While the "wiring-length minimization" hypothesis explains this spatial arrangement, it remains unclear whether topography fundamentally shapes the internal structure of neural representations (i.e., how information is partitioned within the population). Previous work in artificial neural networks (ANNs) has successfully induced topographic maps in vision and language models, improving spatial smoothness and neural predictivity. However, a critical gap remains: does topography alter the component structure of internal representations to better match the brain? This question has been difficult to evaluate in vision and language due to a lack of comprehensive inventories of neural components. The auditory system offers a unique test case, as human auditory cortex responses (measured via fMRI and ECoG) can be decomposed into a small set of interpretable components corresponding to sound categories like speech, music, and song.

Methodology
The authors introduce TopoAudio, a new class of topographic auditory models designed to test whether spatial constraints reshape internal representations.

• Architecture: The models are based on CochResNet50, a ResNet50 backbone adapted to process time-frequency cochleagrams. This architecture was selected because it previously demonstrated high accuracy in predicting human auditory cortex responses.
• Topographic Constraint (TopoLoss): To induce topography, the authors adapted the TopoLoss framework. They define a 2D "cortical sheet" by mapping convolutional kernels onto a spatial grid. A blurring operation is applied to this sheet to remove high-frequency variations, and the TopoLoss is calculated as the negative mean cosine similarity between the original and blurred cortical maps. This encourages spatially nearby units to develop similar response tunings.
• Training Objective: Models are trained on the Word-Speaker-Noise dataset using a multi-task objective (word recognition, speaker identification, background noise classification). The total loss combines the task loss ($\mathcal{L}_{training}$) and the topographic loss ($\mathcal{L}_{topo}$) scaled by a coefficient $\tau$.
• Evaluation:
  • Task Performance: Evaluated on standard benchmarks: ESC-50 (environmental sounds), NSynth (musical instruments), and Speech Commands.
  • Brain Predictivity: Voxel-wise linear encoding models were fitted to predict human fMRI responses from two datasets (NH2015 and B2021).
  • Component Analysis: The authors performed Independent Component Analysis (ICA) on model activations (specifically Layer 3, which showed peak brain predictivity) to decompose representations into latent components.
  • Alignment: Model components were aligned with components derived from human ECoG recordings (fixed at $N=15$) using a greedy matching procedure based on correlation. Representational Similarity Analysis (RSA) was also used to compare the geometry of model and brain representations over time.

Key Results

1. Preserved Performance: TopoAudio models achieved performance comparable to non-topographic baseline models on all engineering benchmarks (ESC-50, NSynth, Speech Commands) and maintained high accuracy in predicting human fMRI responses across the auditory cortex. The introduction of topographic constraints did not degrade task performance or brain predictivity.
2. Compact Component Structure: Imposing topographic constraints systematically reduced the number of inferred components required to explain a fixed amount of variance (e.g., 80%). Topographic models required fewer components than the baseline, indicating more compact and structured internal representations.
3. Improved Brain Alignment: Crucially, the components inferred from topographic models showed stronger alignment with the component structure derived from human ECoG recordings compared to the baseline.
   • Correlation: Topographic models consistently exhibited higher correlations with ECoG component responses across the stimulus window.
   • Interpretability: Specific components linked to speech, music, and song were recovered more faithfully by topographic models. For instance, the topographic model reproduced the selectivity for speech and music categories more cleanly than the baseline, which showed weaker and more mixed responses.
   • Spatial Contiguity: The weight maps of the topographic models showed spatially contiguous unit organization, whereas baseline units were randomly organized.
4. Optimal Strength: The improvement in brain alignment was most pronounced at intermediate levels of topographic strength ($\tau = 5–25$), suggesting a "sweet spot" where constraints improve alignment without over-constraining the representation.

Significance and Claims
The paper claims to provide initial evidence that topographic constraints offer a general mechanism for producing biologically aligned internal representations in artificial neural networks.

• Beyond Spatial Maps: The work moves beyond evaluating topography solely by the emergence of spatial maps or improvements in neural predictivity. It demonstrates that topography fundamentally alters the decomposable structure of population codes, making them more similar to the brain's "parts list."
• Mechanistic Link: The results suggest a mechanistic link between spatial constraints (wiring-length minimization) and the formation of compact, interpretable population codes. By encouraging nearby units to be similar, the network is pushed toward locally redundant, smoothly varying representations that align with efficient coding principles.
• New Benchmark: The authors propose that component-level alignment serves as a complementary and more sensitive benchmark for testing topographic models than standard task performance or voxel-wise predictivity alone.
• Domain Generality: While focused on audition, the authors suggest that these findings, combined with prior work in vision and language, point to a domain-general principle: topography may be a fundamental organizing principle of the brain that shapes how information is represented across sensory systems.

The paper concludes modestly, noting that while the results are promising, they are limited to a single architecture (CochResNet50) and a specific topographic strategy (TopoLoss via weight blurring). Future work is needed to test the robustness of these effects across different architectures and stimulus sets.