Technical Summary: Accommodation Goes Both Ways: Studying Linguistic Convergence Between Humans and Language Models

Problem Statement

As Large Language Models (LLMs) become increasingly integrated into daily life, a critical open question remains: how does their presence shape human linguistic behavior? While LLMs can produce text indistinguishable from human language, it is unclear how human interlocutors adapt their speech when conversing with them compared to human-human interactions. Specifically, the paper investigates the phenomenon of linguistic accommodation—the tendency of speakers to adapt their language (phonology, syntax, lexical choices) to match their interlocutor. The study seeks to answer three primary research questions:

1. Do LLMs converge to the style of their users over multi-turn conversations?
2. Do human users converge to the style of their LLM interlocutor?
3. Does human accommodation of LLMs differ from their accommodation of human interlocutors?

Prior work has examined model-to-model convergence or synthetic turn-replacement tasks, but there is a lack of empirical data on the per-turn coordination of both humans and LLMs in realistic, multi-turn chatbot settings across multiple languages.

Methodology

The authors employ an asymmetric convergence metric to disentangle the accommodation behaviors of humans and LLMs. The study utilizes WildChat, a corpus of approximately one million real-world conversations between human users and ChatGPT (GPT-3.5-Turbo and GPT-4), filtered to conversations with 2–10 turn pairs.

Data

• Primary Dataset: WildChat, sampled across eight languages (English, French, Spanish, Portuguese, Italian, Russian, Turkish, and Chinese), totaling up to 5,000 conversations per language.
• Baselines: Two human-human dialogue datasets were used for comparison:
  • DailyDialog: Everyday conversations.
  • Ubuntu Dialogue Corpus: Technical support conversations (representing a task-oriented baseline similar to common LLM use cases).

Metrics

Convergence is measured using the metric defined by Danescu-Niculescu-Mizil and Lee (2011), which calculates the probability of a speaker using a feature given their partner used it, minus the speaker's baseline rate. Two feature types were analyzed:

1. LIWC Function Words: Categories from the Linguistic Inquiry and Word Count lexicon (e.g., pronouns, articles, prepositions, negations) which serve as unconscious markers of accommodation.
2. NOUN Lemmas: The 100 most frequent noun lemmas per corpus (filtered for WordNet synonyms where possible) to measure lexical entrainment on open-class words.

The analysis was conducted separately for the LLM and the human user in each conversation to capture asymmetric adaptation.

Key Results

1. Asymmetric Convergence

The study reveals a significant asymmetry in accommodation:

• LLMs Overconverge: LLMs significantly overconverge toward their users on both function words and open-class (noun) features.
  • Function Words: Mean LIWC convergence for LLMs is roughly twice that of users (0.068 vs. 0.035 in English). The relative increase ranges from 22% (prepositions) to 164% (negation).
  • Nouns: LLM noun convergence is even more pronounced, with ChatGPT achieving a mean score of 0.442 compared to 0.141 for users. This indicates LLMs are highly sensitive to user lexical choices beyond function words.
• Human Stability: Human users accommodate LLMs at rates broadly consistent with human-human baselines.
  • Mean LIWC convergence for humans in WildChat (0.035) falls within the range observed in DailyDialog (0.036) and Ubuntu (0.022–0.024).
  • Human noun convergence in WildChat (0.141) is comparable to DailyDialog (0.132) and Ubuntu (0.104).
  • Exception: Humans showed higher convergence on negation when talking to LLMs (0.081), likely driven by the LLM's extreme convergence on that specific feature.

2. Cross-Lingual Robustness

The asymmetry holds consistently across all eight languages examined. Despite typological differences and variations in training data coverage:

• LLM convergence scores consistently exceed user rates by 95% to 213% for LIWC features.
• LLM noun convergence exceeds user rates by 214% to 330% across all languages.
• English, often the best-resourced language, falls in the middle of this range, suggesting the asymmetry is not merely a function of model quality on a specific language.

3. Temporal Dynamics

A per-turn analysis reveals that convergence rates do not follow a consistent linear trend over the course of a conversation.

• Most fluctuations appear as noise.
• Significant linear trends (positive or negative) are sporadic and language-specific (e.g., Russian speakers and models increasing convergence, while Portuguese speakers diverge), rather than a systematic tendency for either party to accommodate more or less over time.
• This contrasts with some prior findings on progressive convergence, suggesting that local turn-by-turn coordination remains relatively stable.

Significance and Claims

The paper claims that accommodation in human-LLM dialogue is fundamentally asymmetric. While LLMs dramatically overfit to their users' linguistic styles, humans linguistically accommodate LLMs no differently than they would another person.

The authors hypothesize that this stability in human behavior is due to the fluency of modern LLMs. Citing the consensus that linguistic accommodation is a largely unconscious process (Giles et al., 1991; Pickering and Garrod, 2004), the paper suggests that once an AI produces sufficiently coherent language, the unconscious mechanisms of accommodation trigger as they do in human-human dialogue, regardless of the interlocutor's nature. This finding contrasts with earlier work (e.g., Bhatt and Rios, 2021) where humans accommodated bots differently, a discrepancy the authors attribute to the incoherent responses of earlier bot systems versus the coherence of modern LLMs.

The study concludes that as LLMs become more integrated into everyday communication, understanding these subconscious human responses is critical for anticipating the long-term effects of LLMs on natural language, noting that the "overaccommodating" behavior of the model does not necessarily elicit a reciprocal over-accommodation from the human.