From Perception to Cognition: How Latency Affects Interaction Fluency and Social Presence in VR Conferencing

This paper investigates how end-to-end latency impacts interaction fluency and social presence in VR conferencing compared to traditional video conferencing through subjective experiments, aiming to clarify the underlying perceptual and cognitive mechanisms to guide system optimization.

The Gist

Here is an explanation of the paper, translated into simple language with some creative analogies.

🌐 The Big Picture: The "Glitch" in the Virtual Room

Imagine you are in a video call with a friend. Usually, you talk, they listen, and they reply instantly. It feels natural, like you are in the same room.

Now, imagine there is a delay (latency). You say "Hello," but your friend doesn't hear it for two seconds. They reply, and you don't hear them for another two seconds. Suddenly, the conversation feels awkward, like a bad walkie-talkie connection. You might talk over each other, or you might feel like your friend isn't listening.

This paper is a scientific study about Virtual Reality (VR) meetings. The researchers wanted to know: How much delay can we handle before the meeting feels broken? And, does a VR meeting (where you see digital avatars) feel different from a normal video call (where you see real people on a screen)?

They looked at two main things:

1. Interaction Fluency (The "Flow"): How smooth does the conversation feel? (Perception)
2. Social Presence (The "Connection"): Do you feel like you truly understand your friend's feelings and intentions? (Cognition)

---

🧪 The Experiment: The "Speed Dating" of Delays

The researchers set up a lab where pairs of friends had conversations under different conditions:

• Scenario A: A normal video call on a laptop (like Zoom).
• Scenario B: A VR call where they wore headsets and saw each other as 3D digital avatars.

They introduced delays ranging from almost zero (100ms) to a very long, painful delay (3 seconds). They also gave the pairs three types of tasks to do:

1. Counting: "One, two, three..." (Fast, no thinking required).
2. Math: "What is 2+2?" (Medium thinking).
3. Free Chat: "How was your weekend?" (Slow, lots of thinking).

---

🔍 Key Findings: The Surprising Results

1. The "Flow" Breaks Faster in Simple Tasks

Think of Interaction Fluency like a dance.

• The Counting Task: This is a fast-paced dance. If the music (the connection) has a delay, the dancers trip immediately. The study found that in simple, fast tasks, even a tiny delay made the conversation feel "clunky" and broken.
• The Free Chat: This is a slow, relaxed stroll. If there is a delay, you can just pause, think, and keep walking. People were much more forgiving of delays during casual chat.

The Twist: When the delay got bad, VR users were more patient than video call users.

• Why? The paper suggests that because VR users are looking at a digital avatar (a cartoon-like person), their brains are already working harder to imagine the person is "real." Because they are already doing extra mental work, they don't notice the delay as much as they do when looking at a real human face on a screen. It's like wearing noise-canceling headphones; you are so focused on the virtual world that you ignore the glitches.

2. The "Connection" Crumbles in VR

Think of Social Presence like the emotional bond or the "vibe" between two people.

• At Low Delays: VR wins! When the connection is fast, VR feels amazing. You feel like you are actually sitting next to your friend. The "vibe" is stronger than a 2D screen.
• At High Delays: VR loses big time. Once the delay gets too long (over 1 second), the "vibe" in VR crashes harder than in a normal video call.
• Why? In a video call, if there is a delay, you can still see the real person's face and read their expression, even if it's late. In VR, if the avatar's movement is delayed, it looks like a glitchy robot. You stop believing the person is "there," and the emotional connection breaks.

3. The "Attribution" Game

The study found something interesting about who people blame for the delay.

• In Fast Tasks (Counting): If the delay happens, people blame the system ("This computer is slow!").
• In Slow Tasks (Chatting): If the delay happens, people blame their friend ("Oh, they are just thinking hard about their answer").
• Analogy: If you are playing a fast video game and it lags, you blame the internet. If you are having a deep conversation and there is a pause, you assume your friend is just thoughtful.

---

💡 The Takeaway: What Does This Mean for the Future?

The researchers concluded that VR conferencing is a double-edged sword:

1. It feels smoother (more fluent) than video calls when things go wrong, because our brains are distracted by the cool 3D graphics.
2. But it feels less "real" (less social) when things go wrong, because the digital avatars can't hide the glitches as well as real human faces can.

The Golden Rule:
To make VR meetings feel great, the delay needs to be under 1 second.

• If the delay is under 1 second, VR is magical and feels better than video calls.
• If the delay goes over 1 second, the "magic" disappears, and the connection feels worse than a standard video call.

In short: VR is a fantastic tool for the future of work and socializing, but it needs a very fast internet connection to keep the "human" feeling alive. If the internet is slow, a simple video call might actually feel more human than a glitchy VR world.

Technical Summary

Here is a detailed technical summary of the paper "From Perception to Cognition: How Latency Affects Interaction Fluency and Social Presence in VR Conferencing."

1. Problem Statement

Virtual Reality (VR) conferencing offers immersive, geographically dispersed interactions via avatars, potentially surpassing traditional Video Conferencing (VC). However, both systems suffer from End-to-End (E2E) latency, which degrades user experience. While latency's impact on traditional VC is well-studied, its specific effects on VR conferencing remain under-explored. Crucially, existing research fails to distinguish between:

• Perceptual Level: How latency affects Interaction Fluency (the perceived pace and responsiveness of the conversation).
• Cognitive Level: How latency affects Social Presence (the cognitive ability to understand others' intentions, emotions, and behaviors).
• Comparative Gap: Whether the unique features of VR (wide Field of View, 6DoF, avatars) alter the latency tolerance and cognitive processing compared to traditional 2D VC.

2. Methodology

The authors conducted two subjective experiments involving 22 participants (university students familiar with VR and VC) to analyze the impact of E2E latency (ranging from 100ms to 3000ms) on both VR and VC systems.

• Experimental Setup:
  • VR Platform: Meta Quest 2/Pro using Unity and Meta Avatars SDK (upper-body avatars, facial tracking, 6DoF).
  • VC Platform: Custom PC-based application mimicking Zoom/Tencent Meeting (2D video, head-and-shoulders view).
  • Latency Control: Precise E2E latency injection (80ms base + variable delay) via WebRTC rendering time manipulation.
• Tasks:
  • Experiment 1 (Interaction Fluency): Participants engaged in three task types: Counting (high frequency, low cognitive load), Arithmetic (medium frequency), and Free Conversation (low frequency, high cognitive load). Fluency was rated using the Absolute Category Rating (ACR) method (1–5 scale).
  • Experiment 2 (Social Presence): Participants engaged in Free Conversation only. Social presence was measured using the Networked Minds Social Presence Inventory (NMSPI), covering six subscales: Co-presence (CP), Attentional Allocation (AA), Message Understanding (MU), Affective Understanding (AU), Emotional Interdependence (EI), and Behavioral Interdependence (BI). Ratings used a 7-point Likert scale.
• Analysis: Statistical analysis included Pearson correlation, T-tests, Wilcoxon signed-rank tests, and linear regression to determine the relationship between fluency and social presence decay rates.

3. Key Contributions

• Dual-Dimensional Framework: The study explicitly separates latency impact into perceptual (fluency) and cognitive (social presence) dimensions, revealing that they degrade differently under latency.
• VR vs. VC Latency Tolerance: It establishes that users exhibit higher tolerance for latency in VR regarding interaction fluency compared to VC, likely due to the cognitive load required to process digital avatars, which distracts from the perception of delay.
• Task-Dependent Sensitivity: The study identifies that low-cognitive tasks (counting) are most sensitive to latency, while high-cognitive tasks (free conversation) allow users to attribute delays to the partner rather than the system, masking latency effects.
• Cognitive Thresholds: It defines specific latency thresholds where VR's cognitive advantages (social presence) collapse, falling below traditional VC performance.

4. Key Results

A. Impact on Interaction Fluency (Perception)

• General Trend: Fluency decreases as latency increases for both VR and VC.
• Task Sensitivity:
  • Counting Task: Most sensitive to latency. Fluency drops sharply (e.g., from ~5 to ~1.3 in VC at 1500ms).
  • Free Conversation: Least sensitive. Users tolerate higher delays (e.g., ~3.5 at 3000ms in VC).
• VR vs. VC Comparison:
  • At <200ms, VR and VC fluency are statistically similar.
  • At >200ms, VR demonstrates higher fluency scores than VC. Users are more tolerant of latency in VR, possibly because interacting with avatars requires more cognitive resources, reducing perceptual sensitivity to delay.

B. Impact on Social Presence (Cognition)

• Low Latency (<1000ms): VR generally outperforms or matches VC in most social presence subscales (CP, AA, MU, AU).
• High Latency (>1500ms): VR performance degrades faster than VC.
  • Message Understanding (MU) & Attentional Allocation (AA): VR scores drop significantly below VC when latency exceeds 2000–2500ms.
  • Emotional/Behavioral Interdependence (EI/BI): VR consistently scores lower than VC across most latency levels. This is attributed to current avatars lacking the fidelity to accurately reflect real human emotions and micro-behaviors.
• Critical Threshold: To maintain social presence scores above 5 (and superior to VC), latency in VR must be kept below 1000ms.

C. Relationship Between Fluency and Social Presence

• VR: There is a strong positive correlation (SRCC > 0.9) between interaction fluency and most social presence dimensions (CP, AA, MU, BI). As fluency drops, cognitive understanding drops rapidly. Affective Understanding (AU) is the most tightly coupled with fluency.
• VC: The correlation is weaker for some dimensions. Notably, Attentional Allocation (AA) in VC is unaffected by fluency decay (slope ≈ 0), suggesting users in VC remain focused on the conversation partner even with high latency, whereas VR users get distracted by the environment.

5. Significance and Implications

• System Optimization: The findings suggest that for VR conferencing, latency control is critical for cognitive aspects (social presence) even if users perceive the interaction as "fluid" (high fluency). Systems must prioritize keeping latency under 1 second to prevent the collapse of social presence.
• Avatar Design: The lower scores in Emotional and Behavioral Interdependence highlight the need for more expressive, high-fidelity avatars that can accurately convey micro-expressions and body language to bridge the gap with real human interaction.
• Task Adaptation: VR conferencing systems could dynamically adjust interaction protocols based on task type. High-frequency, low-cognitive tasks (like rapid Q&A) require stricter latency constraints in VR than free-form discussions.
• Theoretical Insight: The study challenges the assumption that VR is universally superior; while it offers better perceptual tolerance for delay, it imposes a higher cognitive burden that makes social presence more fragile under network stress.