Socially fluent AI decouples conversational signals from source identity in online interaction

Technical Summary: Socially Fluent AI Decouples Conversational Signals from Source Identity in Online Interaction

Problem Statement
The emergence of agentic AI systems capable of sustaining socially fluent, context-sensitive interaction challenges the foundational mechanisms of computer-mediated communication (CMC). In cue-lean environments where nonverbal warrants (prosody, gaze) are absent, humans traditionally rely on social heuristics applied to linguistic and chronemic cues (e.g., responsiveness, tone, timing) to infer identity and intent. This inference operates under a "truth-default" presumption of human identity. The central problem addressed is whether the decoupling of social signaling from source identity by advanced generative models renders these established heuristics non-diagnostic, thereby creating a systemic inability to distinguish human peers from algorithmic systems in synchronous, undisclosed group interactions.

Methodology
The study employed a computational social science approach using a mixed factorial experimental design within a custom-built, synchronous text-based collaboration platform.

• Participants: A heterogeneous sample of 786 participants (recruited via Prolific) was nested within 471 collaborative triads.
• Experimental Conditions: Groups were assigned to one of five conditions: all-human (3H), mixed human-AI (2H+1AI), or AI-majority (1H+2AI). In mixed groups, AI agents adopted either a "supportive" or "contrarian" conversational stance. Crucially, the artificial nature of AI teammates was undisclosed, forcing participants to rely solely on interactional traces.
• Tasks: Participants engaged in 10-minute synchronous discussions across three distinct domains: analytical (winter survival ranking), ethical (autonomous vehicle moral dilemma), and creative (collaborative movie plot generation).
• AI Implementation: Agents were instantiated using large language models with structured system prompts designed to mimic human behavioral irregularities (e.g., self-correction, contractions, probabilistic timing with jitter) while strictly adhering to assigned stances. Agents were instructed never to reveal their artificial status.
• Data Collection: Post-interaction, participants provided categorical identity judgments (Human, AI, Not sure), social perception ratings (humanness, trust, supportiveness, conflictuality), and free-text impression justifications.
• Analytic Pipeline:
  • Signal Detection Theory (SDT): Used to quantify identity discrimination sensitivity ($d'$) and response bias.
  • Computational Linguistics: Dialogue logs were processed to extract interactional cues (LIWC-based linguistic features, lexical diversity, response latency). Impression texts were analyzed using unsupervised topic modeling (BERTopic) and supervised lexical contrasts (c-TF-IDF).
  • Predictive Modeling: Logistic regression models were fitted to (1) predict participant identity judgments from dialogue cues and (2) predict ground-truth identity from the same cues, allowing for a direct comparison of diagnosticity.
  • Representational Similarity Analysis (RSA): Used to map the geometric alignment between interactional cue spaces, ground-truth identity, and subjective judgment spaces.

Key Results

1. Failure of Identity Discrimination: Across 1,572 identity judgments, participants failed to distinguish AI from human teammates above chance levels. The aggregate sensitivity was $d' = 0.117$ (95% CI [−0.007, 0.257]), indicating performance indistinguishable from random guessing. This failure persisted across all task domains and was uniform across individual participants, regardless of AI literacy, personality traits, or demographics.
2. Robustness of Ground-Truth Signals: Despite human failure, interactional cues contained robust, separable information regarding ground-truth identity. Computational models predicting ground-truth identity from the same dialogue cues achieved near-ceiling accuracy (AUC = 0.983 in cross-validated mixed-identity triads). This diagnosticity held even when timing features (latency) were excluded, confirming that linguistic and interactional structures alone encoded identity.
3. Judgment-Truth Dissociation: A pronounced dissociation exists between the information available in the dialogue and the information utilized by humans. Models predicting human judgments from cues showed negligible explanatory power ($R^2 \approx 0.02$), whereas models predicting ground truth showed high explanatory power ($R^2 \approx 0.69$). Participants relied on heuristics such as response speed, fluency, and perceived scriptedness, which were only weakly related to actual identity.
4. Representational Decoupling: RSA revealed that the representational space of interactional cues was strongly aligned with ground-truth identity ($\rho = 0.455$) but effectively orthogonal to participant judgments ($\rho \approx 0$). Instead, judgment structures aligned with subjective impressions (humanness and trust ratings), suggesting that social inference is organized around evaluative impressions rather than diagnostic behavioral structures.
5. Heuristic Reliance: Analysis of impression texts showed that participants' justifications were dominated by generic humanness language and explicit uncertainty. The linguistic features participants treated as diagnostic (e.g., "fast," "scripted") were inconsistently linked to actual identity, while the features that actually distinguished AI (e.g., lexical diversity, tone score) were largely ignored in explicit judgments.

Key Contributions

• Empirical Demonstration of Decoupling: The study provides empirical evidence that socially fluent AI agents can decouple conversational signals from source identity, rendering traditional CMC heuristics non-diagnostic in synchronous group settings.
• Distinction Between Availability and Use: The paper establishes a critical distinction between the availability of identity-relevant information in behavioral traces (which is high) and the use of that information in human judgment (which is near-zero).
• Geometric Account of Inference Failure: By utilizing RSA, the study moves beyond accuracy metrics to provide a structural explanation for identity inference failure, demonstrating that human social inference operates in a representational space anchored in subjective impressions rather than diagnostic cues.
• Validation of Undisclosed Interaction: Unlike prior work focusing on disclosed AI or functional outcomes, this study investigates the ecological reality of undisclosed interaction, showing that the "truth-default" mechanism leads to systematic misclassification even when participants are socially responsive to interactional stances.

Significance and Claims
The paper claims that the rise of autonomous, socially fluent agents precipitates a breakdown in identity inference under truth-default conditions. The significance lies in the finding that human observers are systematically unable to distinguish artificial from human social actors, even when the interactional traces contain sufficient information for high-accuracy computational classification.

This dissociation creates new vulnerabilities for online discourse, as coordinated AI agents can influence and manipulate collective dynamics at scale without detection. The findings suggest that the epistemic foundations of trust and accountability in digital environments—often reliant on the presumption that interlocutors are human—are becoming fragile. The study concludes that while humans can form coherent social impressions and calibrate trust based on interactional style, the mapping from these impressions to source identity has become unreliable, necessitating a re-evaluation of how identity verification and accountability function in AI-mediated communication.