Multivariate Transfer Entropy Quantifies Information Transfer Between Dyadic Partners During Real-Time Perceptual Decision-Making

This study demonstrates that humans dynamically and adaptively integrate social information in real-time perceptual decision-making by selectively relying on partners' choices and confidence based on stimulus reliability and partner quality, utilizing multivariate transfer entropy to quantify these continuous information flows.

The Gist

The Big Idea: Two People, One Brain?

Imagine you are playing a video game where you have to steer a boat through a foggy river. You can't see the water very well, but you have a friend sitting next to you in a separate boat. You can see their boat moving, and they can see yours.

This study asked a simple question: When you are trying to figure out where to go, how much do you actually look at your friend's boat to help you decide?

Most previous studies asked people to make decisions in short, choppy "trials" (like a quiz). But in real life, decisions happen smoothly and continuously, like driving a car. This study used a continuous game to see how humans blend their own senses with what their friends are doing in real-time.

The Game: The "Foggy River" Challenge

The researchers set up a game called the Continuous Perceptual Report (CPR) task.

• The Setup: Two people sat in separate rooms. On their screens, a cloud of white dots moved around (like a school of fish).
• The Goal: They had to use a joystick to point a cursor in the direction the dots were moving.
• The Twist: They also had to show how confident they were. They did this by tilting the joystick. If they were super sure, the cursor got wide (like a confident stance). If they were unsure, the cursor got narrow (like a hesitant shrug).
• The Social Part: They could see their partner's cursor moving and changing shape in real-time. They were told to maximize their own score, but they weren't forced to talk or coordinate.

The Secret Weapon: "Information Thermometers"

To understand what was happening in their brains, the researchers didn't just look at who won. They used a mathematical tool called Multivariate Transfer Entropy.

Think of this tool as a super-sensitive thermometer for information flow.

• It measures how much "heat" (information) flows from one person to another.
• Crucially, it can tell the difference between: "I moved because I saw the dots move" vs. "I moved because I saw you move."

The Three Big Discoveries

1. We Copy What We See, Not What We Feel (The "Mirror Effect")

The study found that people mostly copied their partners in the same category.

• If your partner's cursor moved left, you were more likely to move your cursor left.
• If your partner's cursor got wide (showing confidence), your cursor got wide too.
• The Analogy: Imagine two dancers. If one dancer spins, the other spins. If one dancer jumps, the other jumps. But if one dancer spins, the other doesn't suddenly start jumping. We tend to mirror the type of action we see.

2. We Trust the "Expert" When the Fog Gets Thick

The researchers made the "fog" (the dots) harder to see by making them move more randomly (low "coherence").

• When the dots were clear: People trusted their own eyes.
• When the dots were blurry: People started looking at their partner more.
• The Catch: They only looked at their partner if the partner was actually good at the game. If they were playing with a computer that was perfect at the game, they leaned on the computer heavily when the dots got blurry. If the partner was bad, they ignored them.
• The Analogy: Imagine you are driving in a heavy storm. If your GPS (the partner) is working perfectly, you trust it completely. If your GPS is glitching, you ignore it and try to drive yourself.

3. Speed vs. Confidence: The "Fast Reaction" vs. The "Slow Thought"

The researchers measured exactly how fast people reacted to their partners.

• Reaction to Direction: When the partner's cursor moved, the subject reacted very quickly (about 400ms faster than reacting to the dots themselves).
• Reaction to Confidence: When the partner's cursor got wide (showing confidence), the subject took longer to react.
• The Analogy: Think of it like a car. Seeing the partner's car turn left is a visual cue you can react to instantly (braking or steering). But realizing "Oh, they look really confident about that turn" requires a moment of thought. It takes longer to process the feeling of confidence than the fact of movement.

Why Does This Matter?

This paper is a big deal because it moves away from "lab rat" experiments where people sit still and click buttons. Instead, it looks at how our brains work when we are constantly moving and interacting, just like in real life.

It proves that:

1. We are constantly "tuning" our brains to listen to others, even without being told to.
2. We are smart about who we listen to (we listen to the experts).
3. We process "what they are doing" faster than "how sure they are."

In a nutshell: Humans are like a flock of birds. We constantly watch the bird next to us. If the sky is clear, we fly on our own. If the sky gets stormy, we flock closer to the leader. And we react to their turns instantly, but it takes us a second to understand how confident they feel about the turn.

Technical Summary

1. Problem Statement

Social behaviors in naturalistic settings unfold continuously in time, yet the majority of existing research on social influence relies on discrete, trial-based paradigms. These discrete methods fail to capture the dynamic, real-time modulation of behavior between interacting individuals. Furthermore, while information theory has been successfully applied to neuroscience, its application to real-time, continuous behavioral integration in social contexts remains underexplored.

The authors aim to address four specific questions regarding human social interaction during ongoing perceptual decision-making:

1. Do subjects integrate continuously changing stimulus and partner information into their behavioral outputs?
2. What are the specific sources of the integrated information?
3. How does stimulus reliability modulate the interaction between subjects?
4. What are the temporal latencies of integrating partner behavior?

2. Methodology

Experimental Design: Continuous Perceptual Report (CPR) Task

• Participants: 38 human participants (13 with corrected vision) performed tasks in three conditions: Solo, Human-Human Dyad (two people interacting), and Human-Computer Dyad (one person interacting with a computer-controlled partner, unbeknownst to the human).
• Task: Participants tracked a dynamic Random Dot Pattern (RDP) motion stimulus using a joystick.
  • Joystick Direction: Reported the perceived motion direction.
  • Joystick Tilt: Reported perceptual confidence (narrower cursor = higher confidence; wider cursor = lower confidence).
  • Feedback: Participants received immediate visual and auditory feedback based on accuracy and confidence. Crucially, in dyadic conditions, both partners could see each other's cursor (direction and tilt) in real-time.
• Stimulus: The RDP motion direction and coherence (signal-to-noise ratio) changed pseudo-randomly. Seven coherence levels were tested (0% to 98%).

Analytical Framework: Multivariate Transfer Entropy (TE)

The core methodological contribution is the application of Multivariate Transfer Entropy to quantify information flow.

• Definition: TE measures the amount of information a source variable ($S$) provides about the future state of a target variable ($T$), conditioned on the past of the target and other relevant source variables ($S_{-j}$).
• Variables Analyzed:
  • Sources: Stimulus Direction ($sd$), Stimulus Coherence ($sc$), Partner's Direction ($pd$), Partner's Tilt ($pt$).
  • Targets: Subject's Joystick Direction ($jd$), Subject's Joystick Tilt ($jt$).
• Advantage: The multivariate approach allows the detection of information sources while accounting for redundancy. For example, it can distinguish whether a subject's direction is influenced by the partner's direction after accounting for the stimulus direction, preventing false positives due to shared stimulus input.
• Implementation: Used the IDTxl toolbox with a Gaussian estimator and non-uniform embedding to select optimal time lags.

Statistical Modeling

• Bayesian Linear Regression: Used to model the relationship between Transfer Entropy values and task performance (reward score).
• Models:
  • SI Model: Predicts score based only on stimulus information transfer.
  • SSI Model: Predicts score based on both stimulus and social information transfer.
  • HSSI Model: Hierarchical SSI model distinguishing between Human-Human and Human-Computer dyads.
• Model Comparison: Evaluated using Leave-One-Out (LOO) cross-validation to determine predictive accuracy.

3. Key Results

A. Structure of Information Flow

• Dimensional Matching: Social information transfer occurred predominantly within matching behavioral dimensions.
  • Partner's Direction influenced Subject's Direction (90% of subjects).
  • Partner's Tilt (Confidence) influenced Subject's Tilt (38% of subjects).
  • Cross-dimensional influences (e.g., partner's direction affecting subject's confidence) were negligible.
• Interaction Types: Dyads were classified into Mutual Influence (81.2% for direction), Leader-Follower (8.3% for direction), or No Interaction. Mutual influence was the dominant pattern.

B. Information Transfer Improves Performance

• Correlation with Score: Bayesian models showed a positive correlation between the amount of information transferred and the subject's reward score.
  • Higher TE from Stimulus $\to$ Direction and Partner $\to$ Direction significantly predicted higher scores.
  • The SSI model (Stimulus + Social) outperformed the SI model (Stimulus only), confirming that social information contributes to task performance.
• Performance-Dependent Acquisition: Subjects acquired significantly more information from partners who performed better in solo trials. The transfer entropy from a partner's direction increased linearly with the difference in solo performance ($\Delta$ solo score).

C. Modulation by Stimulus Reliability

• Adaptive Weighting: Subjects flexibly adjusted their reliance on social vs. stimulus information based on stimulus noise.
  • High Coherence: Information flow from Stimulus $\to$ Direction increased.
  • Low Coherence (Noisy): Information flow from Partner $\to$ Direction increased, but only in Human-Computer dyads.
• Implication: Subjects only increased reliance on social cues when the social partner was highly reliable (the computer). In Human-Human dyads, this adaptive shift was not observed, suggesting humans are selective about which social sources they trust when sensory evidence is poor.

D. Temporal Latencies

• Direction vs. Confidence: Subjects responded to changes in their partner's direction significantly faster (approx. 415 ms faster) than to changes in their partner's confidence (tilt).
• Partner vs. Stimulus: Subjects reacted to changes in their partner's direction faster than to changes in the stimulus direction itself. This suggests a "priming" effect where the partner's reaction to the stimulus serves as a faster, secondary cue for the subject.

4. Key Contributions

1. Methodological Innovation: Successfully applied Multivariate Transfer Entropy to continuous, real-time social interaction data, establishing a framework to disentangle stimulus-driven from socially-driven behavioral changes without discrete trials.
2. Behavioral Specificity: Demonstrated that social influence is highly specific to behavioral dimensions (direction influences direction; confidence influences confidence) rather than being a general "social noise."
3. Adaptive Social Cognition: Provided evidence that humans dynamically re-weight social information based on the reliability of both the sensory stimulus and the social partner.
4. Temporal Dynamics: Revealed distinct processing latencies for integrating social choice (fast) versus social confidence (slow), offering insights into the cognitive mechanisms of social decision-making.

5. Significance

This study bridges the gap between discrete trial-based social psychology and continuous naturalistic behavior. By proving that multivariate information-theoretic analyses can effectively quantify "interacting minds" in real-time, the paper offers a powerful tool for studying complex social cognition. The findings suggest that human social integration is not a passive process but an active, adaptive strategy where individuals selectively trust social partners based on their demonstrated reliability and the clarity of the environment. This approach paves the way for analyzing more complex, naturalistic social networks where traditional modeling is difficult.