🧬 biology

Dissociating spatial frequency reliance from adversarial robustness advantages in neurally guided deep convolutional neural networks

This study demonstrates that while neurally aligned deep convolutional neural networks exhibit both increased reliance on low spatial frequencies and improved adversarial robustness, the shift toward low-frequency processing is merely an emergent property of learning human-like representations rather than the primary mechanism driving robustness, as directly biasing models toward these frequencies fails to replicate the robustness gains.

Original authors: Zhenan Shao, Tianyu Ren, Chengxiao Wang, Leyla Isik, Diane M. Beck

Published 2026-05-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhenan Shao, Tianyu Ren, Chengxiao Wang, Leyla Isik, Diane M. Beck

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are teaching a robot to recognize animals in photos. You want it to be as tough and reliable as a human, so it doesn't get tricked by tiny, invisible changes to the picture (like a few pixels shifting) that would make a human laugh but cause the robot to think a "dog" is actually an "ostrich."

Scientists recently discovered that if they teach the robot to "think" more like a human brain, it becomes much harder to trick. But they didn't know why.

One popular theory was that the robot became tougher because it started ignoring fine details (like fur texture) and focused only on the big, blurry shapes (like the overall outline of the animal). Another theory suggested it focused on a specific "sweet spot" of detail that humans use to recognize things.

This paper is like a detective story where the researchers test these theories by forcing the robot to focus only on those specific things. Here is what they found, explained simply:

The Setup: The "Blurry vs. Sharp" Debate

Think of an image like a song.

Low Frequencies (LSF): The deep bass notes. These give you the general vibe and the shape of the song.
High Frequencies (HSF): The high-pitched cymbals and tiny notes. These give you the fine details and texture.
The "Human Channel": A specific range of notes in the middle that humans seem to rely on most to recognize faces and objects.

Previous studies showed that when robots were trained to mimic the human brain, they naturally started listening more to the "bass" (Low Frequencies) and that specific "Human Channel," and less to the "cymbals" (High Frequencies). This made them harder to trick.

The Experiment: Forcing the Robot to Listen

The researchers asked: Is the robot tough because it's listening to the bass, or because it's listening to the Human Channel?

To find out, they didn't just let the robot learn naturally. They put on "training wheels" (data manipulation) to force the robot to focus only on specific parts of the song during training:

The "Human Channel" Group: Forced to listen only to that specific middle range.
The "Bass Only" Group: Forced to listen only to the deep, blurry shapes.
The "Combo" Group: Forced to listen to both.

The Twist: The Results Were Surprising

The researchers expected that forcing the robot to listen to these "human" frequencies would make it tough. Instead, they found a big disconnect:

The "Human Channel" Trap: When they forced the robot to focus only on the human sweet spot, it didn't get tougher. In fact, it got weaker. It became easier to trick. It was like trying to learn to drive a car by only looking at the speedometer; you miss the road and crash.
The "Bass" (Low Frequency) Effect: When they forced the robot to focus only on the big, blurry shapes, it did get slightly tougher. But the improvement was tiny.
The "Combo" Failure: Even when they forced the robot to listen to both the bass and the human channel, it still didn't get as tough as the robots that were allowed to learn naturally by mimicking the brain.

The Big Reveal: Correlation is Not Causation

Here is the main takeaway using a simple analogy:

Imagine you see a professional chef (the human brain) making a perfect cake. You notice they always use a specific type of flour (Low Frequencies) and a specific whisk (Human Channel). You think, "Ah! The secret to the perfect cake is that flour and that whisk!"

So, you go home and force yourself to use only that flour and only that whisk. But your cake turns out terrible.

Why? Because the flour and whisk weren't the cause of the good cake. They were just side effects of the chef's overall skill and technique. The chef was actually using a complex, invisible "geometry" of how ingredients fit together to make the cake sturdy.

The paper concludes:
The reason the "neurally guided" robots are tough isn't because they are listening to the bass or the human channel. Those are just things they happen to do because they are learning to think like a human. The real secret is that they are learning a better, more human-like way of organizing information (a better "geometry") that makes them naturally robust.

Forcing a robot to focus on specific frequencies is like trying to fix a car by painting the tires red; it might look like the right thing to do, but it doesn't actually make the engine run better. The real magic is in the engine's design, not the color of the paint.

Technical Summary: Dissociating Spatial Frequency Reliance from Adversarial Robustness Advantages in Neurally Guided Deep Convolutional Neural Networks

Problem Statement

Deep Convolutional Neural Networks (DCNNs) have achieved human-level performance in visual recognition but remain strikingly vulnerable to adversarial attacks—near-imperceptible perturbations that cause catastrophic misclassifications. Recent research indicates that aligning DCNN representations with human visual cortex activity (neural guidance) significantly improves adversarial robustness. However, the mechanism driving this improvement is unclear.

Two competing hypotheses have emerged regarding the spectral features responsible for this robustness:

Low Spatial Frequency (LSF) Bias: Neural alignment may improve robustness by shifting reliance from brittle high-frequency textures to coarse, global LSF structures, which are more stable under transformation.
Human Channel Bias: Recent work suggests human object recognition relies on a narrow, mid-frequency "human channel" (approx. 28–56 cycles per image). Since prior LSF-biasing methods (e.g., Gaussian blur) inadvertently preserve this channel, robustness gains might actually stem from the preservation of this specific mid-frequency band rather than a general LSF shift.

This paper investigates whether the adversarial robustness observed in neurally guided (NG) models is causally driven by an increased reliance on the LSF, the human channel, or both.

Methodology

1. Characterizing Neurally Guided (NG) Models

The authors first analyzed a set of DCNNs trained with "Neural Guidance" (NG), where the penultimate layer of the network is aligned to fMRI responses from seven regions along the human ventral visual stream (V1, V2, V4, VO, PHC, LO, TO).

Metric: Adversarial robustness was quantified as the Area Under the Curve (AUC) of top-1 accuracy under $\ell_\infty$ -bounded Projected Gradient Descent (PGD) attacks across various perturbation magnitudes ( $\epsilon$ ).
Spectral Analysis: To determine spatial frequency (SF) reliance, the authors generated successful adversarial perturbations for each model. They computed the Fourier power spectrum of these perturbations, radially averaged them, and normalized the energy distribution. Higher power in a specific band indicates greater model reliance on that frequency range.

2. Explicitly Biasing DCNNs via Phase Scrambling

To test causality, the authors trained new DCNNs (ResNet-18 on ImageNet-50) with explicit SF biases using phase-scrambling data augmentation. This method preserves the amplitude spectrum (energy) of an image while scrambling the phase of specific frequency bands, effectively removing structural information from those bands while retaining others.

Conditions Tested:
- Human Channel: Preserved only the 28–56 cpi band; scrambled the rest.
- Extreme-LSF: Preserved frequencies < 3.5 cpi; scrambled the rest.
- Fixed- $\sigma$ -LSF: Used a Gaussian blur mask ( $\sigma=1.5$ ) to preserve low frequencies (overlapping with the human channel).
- Mixed- $\sigma$ -LSF: Used a mixture of Gaussian kernels to simulate a broader LSF preference (overlapping with the human channel).
- Combined-Channel: A training distribution mixing Human Channel and Extreme-LSF preservation.
- Necessity Checks: Inverse masks were used to selectively disrupt (scramble) the Human Channel or Extreme-LSF to test if their absence was uniquely detrimental.

3. Representational Similarity Analysis (RSA)

To determine if SF bias alone could replicate the "human-like" geometry of NG models, the authors performed RSA. They compared the Representational Dissimilarity Matrices (RDMs) of the SF-biased models against fMRI data from human V1, V4, and TO regions.

Key Results

1. Correlation in Neurally Guided Models

In NG models, robustness gains correlated strongly with increased reliance on both the Human Channel (Spearman's $\rho = 0.98$ ) and the LSF ( $\rho = 0.87$ ). Conversely, reliance on high spatial frequencies (HSF) was anticorrelated with robustness. This confirmed that NG models shift their spectral reliance toward human-relevant bands.

2. Causal Dissociation via Explicit Bias

When the authors explicitly imposed these biases during training, the results diverged sharply from the NG model behavior:

Human Channel Bias: Explicitly biasing models toward the human channel degraded adversarial robustness. Stronger bias led to worse performance.
LSF Bias: Biasing models toward the LSF (Extreme-LSF, Fixed- $\sigma$ , Mixed- $\sigma$ ) produced only modest improvements in robustness. Crucially, these gains were disproportionately small compared to the massive shifts in SF reliance induced by the manipulation.
Combined Bias: Biasing models toward both the Human Channel and LSF simultaneously still resulted in worse robustness than the baseline, indicating that the detrimental effect of the Human Channel bias was not offset by LSF gains.
Necessity: Disrupting the Human Channel reduced robustness, but disrupting the Extreme-LSF caused a larger drop, suggesting the Human Channel is not uniquely necessary for robustness.

3. Representational Geometry

SF-biased models failed to replicate the representational geometry of the human brain.

While NG models showed significant increases in similarity to human neural representations (V1, V4, TO), SF-biased models showed little to no increase in similarity compared to the baseline.
Even the "Combined-Channel" condition, which mimicked the SF reliance profile of NG models, failed to improve representational similarity.

Key Contributions

Dissociation of Correlation and Causation: The paper demonstrates that while increased reliance on the Human Channel and LSF correlates with robustness in neurally guided models, explicitly forcing these biases is insufficient to generate robustness. In fact, forcing a Human Channel bias is actively harmful.
Refutation of the "Human Channel" Mechanism: The study refutes the hypothesis that the robustness of NG models is driven by the preservation of the mid-frequency human channel.
Limited Role of LSF: While LSF bias offers minor robustness gains, it cannot account for the magnitude of improvements seen in NG models, nor does it induce human-like representational geometry.
Emergent Property Hypothesis: The authors propose that altered SF reliance is an emergent property of learning more human-like representations, rather than the primary causal mechanism of robustness.

Significance and Claims

The paper concludes that the adversarial robustness conferred by neural guidance is not primarily driven by a simple shift in spatial frequency reliance (neither LSF nor the Human Channel). Instead, the authors argue that robustness likely arises from broader geometric reconfigurations of the representational space (e.g., the "untangling" of object manifolds) that naturally lead to the observed spectral biases as a secondary consequence.

The authors maintain a modest stance, noting that while SF bias is a partial property of brain-like representations, it is not the primary computational mechanism. They suggest future research should focus on characterizing the specific geometric transformations in representational space that confer robustness, rather than focusing solely on spectral features. They also acknowledge limitations regarding their use of population-level data augmentation versus the diverse tuning profiles of individual neurons in biological systems.