Serial dependence generalizes across the senses

Through a series of experiments combining behavioral and EEG data, this study demonstrates that serial dependence in numerosity perception generalizes across vision and audition via a functional, mid-level multisensory network, thereby challenging existing low-level and high-level accounts of the phenomenon.

The Gist

Imagine your brain is like a very busy, slightly forgetful artist trying to paint a picture of the world in real-time. Sometimes, this artist gets a little "sticky." If they just painted a blue sky, they might accidentally make the next sky look a little bluer than it actually is. If they just heard a fast drumbeat, the next drumbeat might sound a tiny bit faster.

This "stickiness" is called Serial Dependence. It's a quirk of our brains where what we just experienced pulls our current perception toward it, making things look or sound more similar to the past than they really are.

For a long time, scientists argued about why this happens. They had two main theories:

1. The "Low-Level" Theory: This suggests the stickiness happens inside each sense separately. Like, your eyes have their own sticky glue, and your ears have their own, and they never talk to each other.
2. The "High-Level" Theory: This suggests the stickiness happens after you've seen or heard something, when your brain is making a decision. It's like your brain's "decision committee" getting confused by the last meeting and carrying that confusion into the next one.

The Big Question: Can the "stickiness" jump from one sense to another? If I just saw a bunch of flashing lights, will that make a sound I hear next seem like it has more "flashes" (or a different number of beats)?

This paper says: Yes, it can! But there's a catch: Attention is the gatekeeper.

Here is the story of their three experiments, explained simply:

Experiment 1: The "Sticky" Senses

The researchers set up a game. Participants had to guess how many things they saw or heard.

• The Setup: First, a "distractor" appeared (either a flash of light or a beep). Then, a "target" appeared (another flash or beep).
• The Trick: The distractor was irrelevant to the game, but the researchers wanted to see if it still "stuck" to the target.
• The Result:
  • When participants were only paying attention to one sense (just looking at lights), the "stickiness" worked perfectly. If they saw a lot of lights before, they thought the next light sequence had more lights. Even if the distractor was a sound, it still made the lights look "stickier."
  • The Surprise: When participants had to pay attention to both senses at the same time (looking at lights and listening to sounds), the "stickiness" changed. The senses stopped sticking to themselves (light didn't stick to light) and started sticking to the other sense (sound made the lights stickier, and lights made the sounds stickier).

The Analogy: Imagine you are juggling red balls (vision) and blue balls (audition).

• If you only care about the red balls, the red balls stick to each other, and the blue balls accidentally stick to the red ones too.
• But if you are juggling both at once, the red balls stop sticking to other red balls and start sticking to the blue ones instead! The brain is mixing them together because it's trying to handle the whole picture.

Experiment 2: The "Traffic Cop" (Attention)

The researchers wanted to see if they could control this stickiness with a "traffic cop" (attention).

• The Setup: They showed a mix of lights and sounds together, but told the participant: "Only look at the lights!" or "Only listen to the sounds!"
• The Result: The "traffic cop" worked. If you were told to focus only on the lights, only the lights influenced your next guess. The sounds were ignored.
• The Twist: However, if the lights and sounds were "confused" (one was fast, one was slow), the brain got picky. It would only stick to the thing it was told to watch.

The Analogy: Think of your brain as a radio.

• If you are listening to one station (Vision), the static from another station (Audition) might bleed through and change how you hear the music.
• But if you turn the dial to focus only on that station, the static disappears. The "stickiness" only happens with the channel you are tuned into.

Experiment 3: The "Brain Camera" (EEG)

Finally, they hooked people up to EEG machines (which take pictures of brain waves) to see when this stickiness happens.

• The Question: Does this happen when the brain is seeing the object (perception), or only after the person thinks about the answer (decision)?
• The Result: The brain waves changed while the person was still looking at the stimulus. The "stickiness" happened almost immediately, before the person could even make a decision.
• The Conclusion: This proves the "High-Level" theory (decision-making) is wrong. The stickiness isn't a mistake in the decision committee; it's a feature of the sensory processing itself.

The Analogy: It's like putting on a pair of sunglasses that tint the world slightly based on what you saw five seconds ago. You don't decide to tint the world; the tint happens automatically as the light hits your eyes.

The Takeaway

This paper teaches us that our brain is a multisensory mixer, not a set of isolated rooms.

1. It's Connected: Your eyes and ears talk to each other. What you see can change what you hear, and vice versa.
2. It's Flexible: Your brain decides how much to mix them based on what you are paying attention to. If you are busy with one thing, it keeps the senses separate. If you are busy with everything, it blends them together to create a stable, continuous experience.
3. It's Early: This happens right at the beginning of processing, helping your brain build a smooth, stable movie of reality, even when the raw data is noisy and chaotic.

In short, your brain isn't just a camera recording the world; it's a director that edits the footage in real-time, using the last scene to smooth out the next one, and it uses all its senses to do the editing.

Technical Summary

1. Problem Statement

Serial dependence is a perceptual bias where current perceptions are systematically attracted toward recent past stimuli, promoting stability in a noisy environment. A central debate in the field concerns the computational level at which this mechanism operates:

• Low-level sensory account: Suggests serial dependence is encapsulated within specific sensory modalities (e.g., visual or auditory) to smooth noise in early sensory processing.
• High-level decisional account: Proposes the effect occurs post-perceptually, based on abstract decisional representations independent of sensory modality.
• The Gap: Previous studies on cross-modal serial dependence (e.g., auditory influencing visual perception) have yielded mixed or null results, often attributed to methodological issues (e.g., dissimilar stimulus formats). It remains unclear whether serial dependence is modality-specific, modality-independent (decisional), or operates at an intermediate multisensory perceptual stage. Furthermore, the role of attention in gating these cross-modal effects is not fully understood.

2. Methodology

The authors conducted three complementary experiments combining psychophysics and electroencephalography (EEG) to investigate serial dependence in numerosity perception (the estimation of quantity) across vision and audition.

• Participants: 91 healthy adults total (32 in Exp 1, 27 in Exp 2, 32 in Exp 3).
• Stimuli: Streams of brief visual flashes or auditory tones.
  • Inducer: Task-irrelevant stream (7 or 20 events).
  • Reference: Constant stream (12 events).
  • Probe: Variable stream (6–24 events).
• Experimental Design:
  • Experiment 1 (Behavioral): Compared a Uni-sensory task (participants judged only visual stimuli; inducer could be visual or auditory) vs. a Multi-sensory task (participants judged alternating visual and auditory stimuli). This manipulated whether attention was modality-specific or cross-modal.
  • Experiment 2 (Behavioral): Used a trial-by-trial attentional cueing paradigm. Participants alternated between multi-sensory (simultaneous audio-visual) and uni-sensory stimuli. A cue indicated which component (visual or auditory) to attend. This tested if attention to a specific component of a past multi-sensory stimulus drives the serial dependence effect.
  • Experiment 3 (Behavioral + EEG): Replicated Experiment 1's paradigm but added high-density EEG (128 channels) to identify the neural signature and temporal dynamics of the effect.
• Analysis:
  • Behavioral: Point of Subjective Equality (PSE) and Weber's Fraction (WF) were derived from psychometric curves. A "Serial Dependence Index" was calculated as the normalized shift in PSE based on inducer numerosity.
  • Neural (EEG): Event-Related Potentials (ERPs) were time-locked to the reference stimulus. Analyses focused on posterior occipital/parietal channels. Researchers used sliding-window paired t-tests with cluster-based permutation corrections to identify latency windows where ERP amplitude differed based on the preceding inducer. They also correlated neural contrast amplitudes with behavioral bias strength.

3. Key Contributions

1. Demonstration of Cross-Modal Serial Dependence: The study provides robust evidence that serial dependence in numerosity perception transfers across sensory modalities (auditory inducers biasing visual perception and vice versa), provided the stimuli share a similar spatio-temporal structure (sequential streams).
2. Attention as a Gating Mechanism: The research reveals that attention modulates the direction and strength of these effects. Specifically, cross-modal attention (in multi-sensory tasks) suppresses within-modal effects while enhancing cross-modal integration, whereas modality-specific attention allows both within- and cross-modal effects to coexist.
3. Perceptual vs. Decisional Timing: By using EEG, the authors pinpointed the emergence of serial dependence effects to early perceptual processing stages (within the first half of the reference stimulus presentation), well before a decision could be formed. This challenges high-level decisional accounts.
4. Neural-Behavioral Coupling: The study established a direct correlation between the magnitude of ERP modulation (neural signature) and the strength of the behavioral bias, confirming the neural relevance of the effect.

4. Key Results

• Experiment 1 (Behavioral):

  • Uni-sensory Task: Both visual and auditory inducers produced significant attractive serial dependence on visual perception (approx. 9% effect size).
  • Multi-sensory Task: A striking reversal occurred. Cross-modal pairs (e.g., auditory inducer $\to$ visual reference) showed robust serial dependence (approx. 11–13%). Conversely, within-modal pairs (e.g., visual inducer $\to$ visual reference) showed negligible or non-significant effects.
  • Conclusion: Cross-modal attention suppresses modality-specific stability mechanisms in favor of integrating information across senses.

• Experiment 2 (Behavioral - Attention Cueing):

  • When participants attended to a specific component of a multi-sensory stimulus, only that attended component induced a serial dependence effect on the subsequent stimulus.
  • Congruent Components: When audio and visual components were congruent (both high or both low), effects favored within-modal transfer.
  • Incongruent Components: When components differed, the attended component dominated, inducing effects across modalities.

• Experiment 3 (EEG):

  • Temporal Dynamics: Significant differences in ERP amplitude (based on inducer numerosity) emerged during the presentation of the reference stimulus, often starting as early as 140–200 ms post-onset.
  • Timing Significance: These effects occurred before the reference stimulus offset and before any decision could be made, strongly supporting a perceptual rather than post-perceptual origin.
  • Correlation: The amplitude of the neural contrast (difference in ERP between high/low inducer conditions) significantly predicted the magnitude of the behavioral bias in specific latency windows (e.g., 560–570 ms).
  • Polarity Reversal: The neural signature of cross-modal effects often showed opposite polarity compared to within-modal effects, suggesting distinct underlying neural mechanisms that are modulated by attention.

5. Significance

• Theoretical Resolution: The findings refute the idea that serial dependence is strictly encapsulated in low-level sensory networks. Instead, they support a mid-level multisensory network hypothesis. The brain integrates past information from different senses to form a stable, unified perceptual representation of events, rather than maintaining separate, modality-specific histories.
• Role of Attention: The study clarifies that attention acts as a dynamic gate. In complex, multi-sensory environments, the brain prioritizes the integration of complementary signals (cross-modal) to optimize stability, suppressing redundant within-modal noise.
• Neural Mechanism: By localizing the effect to early perceptual processing stages (prior to decision), the paper challenges high-level decisional accounts and suggests that serial dependence is an intrinsic part of how sensory information is accumulated and represented in real-time.
• Methodological Impact: The study demonstrates that previous failures to find cross-modal effects may have been due to stimulus dissimilarity. Using matched spatio-temporal formats (sequential streams) is crucial for observing these interactions.

In summary, this paper establishes that serial dependence is a flexible, attention-gated mechanism operating at an intermediate perceptual level within a functional multisensory network, crucial for maintaining a coherent and stable experience of the world across different sensory modalities.