🧠 neuroscience

Spatial correspondences of Audiovisual Stimuli on Double Flash Illusion Perception and its Cognitive Modeling

This study demonstrates that susceptibility to the Sound-Induced Flash Illusion increases with visual eccentricity due to higher integration weights in the periphery rather than sensory uncertainty, while spatial congruence between auditory and visual stimuli has no significant effect on this multisensory integration.

Original authors: Zheng, Y., Chen, L.

Published 2026-02-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zheng, Y., Chen, L.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The Big Picture: The Brain's "Sound-Visual" Mixer

Imagine your brain is a DJ mixing two different music tracks: one track is sight (visuals) and the other is sound (auditory). Usually, these tracks play in perfect sync, creating a smooth song. But sometimes, the tracks get out of sync, and the DJ (your brain) gets confused, creating a weird remix that doesn't match reality.

This paper investigates a specific "glitch" in that remix called the Sound-Induced Flash Illusion (SIFI).

The Illusion: If you see one quick flash of light but hear two quick beeps, your brain often tricks you into thinking you saw two flashes. It's as if the sound "pulled" the visual image apart.

The researchers wanted to know two things:

Does it matter where the light is? (Is the illusion stronger if the light is in the corner of your eye vs. right in front of you?)
Does it matter where the sound comes from? (Does the illusion happen if the sound comes from the same side as the light, or the opposite side?)

Experiment 1 & 2: The "Corner of the Eye" Effect

The Setup:
The researchers showed participants a single flash of light at different distances from the center of their vision. Some were right in the middle (like looking at a friend's face), and some were far out in the "periphery" (like seeing a bird in the corner of your eye). At the same time, they played beeps.

The Analogy: The "Fuzzy Edge" vs. The "Sharp Center"
Think of your vision like a high-definition camera.

The Center (Fovea): This is the lens. It's super sharp. You can read tiny text here.
The Periphery (Edges): This is the blurry background. You can see movement, but details are fuzzy.

The Finding:
The researchers discovered that the illusion is much stronger in the blurry edges than in the sharp center.

When the light was in the center, your brain said, "I see one flash clearly. The beeps are just noise."
When the light was in the periphery, your brain said, "Hmm, that flash is a bit fuzzy. I'm not 100% sure it's just one. Since I heard two beeps, I'll bet there were two flashes."

The "Why": The Bayesian Detective
The paper uses a "Bayesian Model" to explain this. Think of your brain as a detective trying to solve a crime.

The Clue: A blurry visual clue (the peripheral flash).
The Witness: A loud, clear audio witness (the beeps).

In the center of your vision, the visual clue is so clear (high confidence) that the detective ignores the witness. But in the periphery, the visual clue is shaky (low confidence). The detective thinks, "The visual evidence is weak, so I'll trust the audio witness more." The brain essentially gives the sound more weight when the sight is far away.

Key Takeaway: The further out in your vision the light is, the more likely your brain is to let the sound trick you. It's not because your eyes are "worse" at seeing; it's because your brain decides to trust the sound more when the sight is far away.

Experiment 3: The "Same Side vs. Opposite Side" Test

The Setup:
Now that they knew location mattered, they asked: "Does the sound have to come from the same place as the light to trick us?"

Scenario A: Light on the left, Sound on the left (Congruent).
Scenario B: Light on the left, Sound on the right (Incongruent).

The Analogy: The "Party Guest"
Imagine you are at a party.

Scenario A: You see a friend on your left waving (Light), and you hear them say "Hello" from your left (Sound). This makes perfect sense.
Scenario B: You see a friend on your left waving, but you hear "Hello" coming from the right side of the room.

Usually, if the sound and sight don't match, your brain gets confused and might say, "Wait, that's not the same person!"

The Finding:
Surprisingly, it didn't matter.
Whether the sound came from the same side or the opposite side, the illusion happened just as strongly. The brain didn't care that the sound was "coming from the wrong place." It just merged the sound and the light together anyway.

Key Takeaway: For this specific illusion, the brain is very "lazy" or "automatic." It doesn't bother checking if the sound and light are in the exact same spot before merging them. It just assumes they belong together if they happen at the same time.

The Grand Conclusion: How Your Brain Mixes Reality

This paper teaches us three main things about how your brain builds reality:

Location Matters (The Eccentricity Effect): Your brain treats things in the corner of your eye differently than things in the center. It's more willing to let sound "rewrite" what it sees in the periphery.
It's About Trust, Not Just Noise: The brain isn't just "blurry" in the periphery; it actively decides to trust sound more than sight when the sight is far away. It's a strategic decision, not a mistake.
Spatial Alignment is Optional: For this illusion, the brain doesn't need the sound and light to be in the exact same spot to merge them. It's a very flexible (and sometimes gullible) mixer.

In a Nutshell:
Your brain is like a DJ who is great at mixing music when the lights are bright and clear (center of vision). But when the lights get dim and fuzzy (periphery), the DJ starts letting the sound effects take over the show, even if the sound is coming from a different speaker than the light. And surprisingly, the DJ doesn't even check if the speakers are in the right place before hitting the mix button!

1. Problem Statement

The Sound-Induced Flash Illusion (SIFI) is a well-documented phenomenon where the perceived number of visual flashes is altered by conflicting auditory beeps (e.g., one flash paired with two beeps is perceived as two flashes). While the temporal dynamics of SIFI are well-understood, the role of spatial characteristics remains debated.

The Conflict: Previous literature offers inconsistent findings regarding whether visual stimulus eccentricity (distance from the center of vision) modulates SIFI susceptibility. Some studies suggest increased susceptibility in the periphery, while others find no effect.
The Gap: It is unclear whether this spatial modulation is driven by:
1. Increased Visual Uncertainty: Peripheral vision is noisier, leading the brain to rely more heavily on auditory input (Bayesian weighting based on reliability).
2. Intrinsic Spatial Weighting: The brain inherently assigns higher integration weights to auditory inputs for peripheral stimuli due to specific neural connectivity, independent of visual noise.
Secondary Question: Does the spatial congruence between auditory and visual stimuli (e.g., sound coming from the same side as the flash) influence the integration process?

2. Methodology

The study employed a multi-experiment approach combining behavioral psychophysics with hierarchical Bayesian computational modeling.

Experimental Design

Experiment 1 (Initial Spatial Mapping):
- Participants: 9 valid participants.
- Stimuli: Visual flashes (white disks) presented at eccentricities of 0°, 7°, and 21° (both hemifields). Auditory beeps were presented binaurally.
- Conditions: Varied Stimulus Onset Asynchrony (SOA) from -120ms to +120ms.
- Control: Used spatial pre-cues to ensure equal attentional allocation across locations, ruling out attentional bias as a confound.
Experiment 2 (Extended Range & Modeling):
- Participants: 11 valid participants.
- Stimuli: Expanded eccentricity range to 0°, 15°, 30°, 45°, and 60° using a curved "audio-screen" setup.
- Goal: To map the full spatial profile and distinguish between the "Visual Uncertainty" and "Spatial Weighting" hypotheses using Bayesian modeling.
- Modeling: Implemented five hierarchical Bayesian models using PyMC5 to compare:
  1. Baseline (constant parameters).
  2. Individual differences.
  3. Weight Model: Integration weight ( $w$ ) varies with eccentricity.
  4. Uncertainty Model: Visual sensory noise ( $\sigma_v$ ) varies with eccentricity.
  5. Integrated Model: Both weight and noise vary.
Experiment 3 (Spatial Congruence):
- Participants: 10 participants.
- Stimuli: Visual flashes at 7° eccentricity. Auditory beeps were presented ipsilaterally (same side), contralaterally (opposite side), or binaurally.
- Goal: To test if spatial mismatch between modalities reduces integration.

Data Analysis

Behavioral: Repeated-measures ANOVA and Bayes Factor analysis (JASP) to handle unbalanced trial designs and quantify evidence for null vs. alternative hypotheses.
Computational: Hierarchical Bayesian inference to estimate posterior distributions of model parameters. Model comparison was performed using the Widely Applicable Information Criterion (WAIC).

3. Key Results

Behavioral Findings

Eccentricity Effect (Exp 1 & 2):
- Unimodal visual performance (flash-only) remained consistent across all eccentricities when attention was controlled.
- SIFI Susceptibility: Increased significantly as visual stimuli moved from the fovea (0°) to the periphery (up to 60°).
- Spatial Profile: The susceptibility followed a Gaussian distribution, peaking in the far periphery.
- Interaction: The eccentricity effect was specific to conflicting audiovisual conditions (e.g., 1 Flash, 2 Beeps). It was absent in unimodal or congruent conditions.
- Temporal Context: The spatial effect was robust across SOAs of 40ms and 70ms, showing no significant interaction between space and time in this range.
Spatial Congruence (Exp 3):
- The spatial relationship between the sound and the flash (ipsilateral vs. contralateral vs. binaural) did not significantly affect the magnitude of the illusion.
- Integration occurred regardless of whether the auditory and visual stimuli were spatially aligned, suggesting a lack of strict spatial binding requirements for SIFI.

Computational Modeling Findings

Model Comparison: The Weight Model (where the prior probability of fusing AV signals increases with eccentricity) provided the best fit (lowest WAIC), outperforming both the Uncertainty Model and the Integrated Model.
Mechanism: The data indicates that the increased illusion in the periphery is not due to increased visual noise (uncertainty). Instead, the brain intrinsically assigns a higher integration weight to auditory information when processing peripheral visual events.
Parameter Estimation: The estimated integration weight ( $w$ ) was lowest at 0° (approx. 0.26) and highest at 60° (approx. 0.89), confirming a monotonic increase in auditory dominance with eccentricity.

4. Key Contributions

Resolution of Spatial Debate: The study definitively demonstrates that SIFI susceptibility is not uniform but follows a Gaussian gradient, increasing significantly in the far periphery (>15°).
Mechanistic Insight: By using Bayesian modeling, the authors ruled out the "Visual Uncertainty" hypothesis. They established that the effect is driven by a location-specific shift in integration weights, suggesting that the brain is "hard-wired" to trust auditory cues more for peripheral visual events.
Independence from Spatial Congruence: The finding that spatial mismatch (contralateral presentation) does not diminish the illusion challenges strict spatial binding theories for SIFI, suggesting integration may occur at a stage where spatial precision is less critical or is processed via higher-order mechanisms.
Methodological Rigor: The use of spatial pre-cues effectively controlled for attentional biases, and the application of hierarchical Bayesian modeling provided a robust computational explanation for behavioral data.

5. Significance

Theoretical Impact: The findings support the view that multisensory integration is an early-stage process modulated by the spatial architecture of primary sensory cortices (e.g., specific cross-modal connectivity in the periphery), rather than solely a high-level cognitive inference based on reliability.
Neuroscientific Implications: The results align with neuroanatomical evidence of direct connections between auditory and peripheral visual cortices (e.g., V1/V2), suggesting these pathways facilitate the automatic weighting of auditory input in the periphery.
Practical Application: These findings provide a critical framework for designing multisensory interfaces and stimuli. For instance, in Virtual Reality (VR) or Augmented Reality (AR), auditory cues may be more effective at capturing attention or altering perception in the user's peripheral vision than in the central field.
Future Directions: The study suggests that future research should focus on neuroimaging (fMRI/EEG) to localize these weighting shifts in early sensory areas and investigate how top-down expectations might modulate these spatial weights.