When a new word works: neologism acceptance and collective identity in human and LLM-agent groups

Technical Summary: Neologism Acceptance and Collective Identity in Human and LLM-Agent Groups

Problem and Motivation
Collective identity, the sense of belonging to a group, is a fundamental driver of human social behavior, influencing cognition and self-evaluation. While qualitative research and correlational studies suggest that distinctive group language (e.g., neologisms) strengthens collective identity, establishing a causal link has been methodologically challenging. Traditional experimental designs require participants to generate new words, which introduces stress and variability, or necessitate large-scale recruitment for repeated trials to mitigate bias, rendering such studies impractical. This study addresses the causal question: Does the introduction of an emergent in-group neologism actively strengthen collective identity, or is the relationship merely correlational?

Methodology
The authors conducted a controlled intervention experiment using a hybrid environment of one human participant and four Large Language Model (LLM) agents.

• Participants: 100 Japanese adults were recruited via CrowdWorks and assigned to two conditions: an intervention group ($N=50$) and a control group ($N=50$). Assignment was quasi-experimental, conducted in sequential waves rather than individual randomization.
• Setting: Participants engaged in approximately 40-minute discussions on Discord regarding the topic "What is happiness, and how can happiness be achieved?"
• Agent Implementation: The four LLM agents (powered by Gemini 2.0 Flash) were assigned distinct personas (age, gender, personality, values) to simulate natural social dynamics. Agents were instructed to use concrete examples and proper nouns to enhance naturalism.
• Intervention: In the intervention group, approximately 30 minutes into the discussion, a randomly selected agent proposed a newly coined Japanese word (katakana neologism) for happiness, derived from the preceding discussion context. Agents subsequently used this neologism with a 50% probability in non-reply utterances. No neologism was introduced in the control group.
• Measures:
  • Collective Identity (CID): Measured via a four-item scale (self-identification, belonging, importance, emotional response to criticism) administered at the midpoint and post-discussion. The outcome variable was the change ($\Delta$CID) between these two time points.
  • Neologism Evaluation: In the intervention group, participants rated the proposed word on "willingness to use" and "perceived interestingness."
  • Interaction Variables: Message count (behavioral engagement) and Team impression (sentiment polarity of open-ended comments, analyzed via LLM).

Key Results
The study employed regression models to estimate intervention effects and interaction associations:

1. Average Intervention Effect: The introduction of a neologism did not produce a statistically significant average increase in collective identity across all participants ($\beta = -0.183, p = .135$). This indicates that the mere presence of a new word does not automatically bind a group together.
2. Conditional Effects in Intervention Group: Within the intervention group, a positive association was found between neologism evaluation and $\Delta$CID ($\beta = +0.227, p = .038$). Participants who rated the new word more favorably showed larger increases in collective identity.
3. Interaction of Engagement: In the intervention group, a significant interaction was observed between message count and team impression ($\beta = +0.372, p = .010$). Participants who were both behaviorally active (high message count) and emotionally positive about the team experienced the greatest growth in collective identity. This interaction was not significant in the control group.
4. Characteristics of Successful Neologisms: Exploratory analysis of word ratings suggested that highly rated neologisms were concrete, evoked clear imagery (e.g., Kokoro Tone - "tone of the heart"), and described current subjective states. Poorly rated words tended to be abstract or refer to long-term processes (e.g., Historia - "life historia").
5. Team Impression: Favorable team impressions were positively associated with increased collective identity in both groups, suggesting that the quality of interaction is a foundational factor regardless of linguistic intervention.

Significance and Claims
The paper claims to provide the first controlled, intervention-based test of the causal relationship between distinctive language and collective identity using LLM agents. The primary contribution is the refinement of the theoretical understanding of group language: the relationship between group-specific language and collective identity is conditional, not automatic.

The authors argue that distinctive language strengthens collective identity only when:

1. The new expression is favorably evaluated by members (acceptance).
2. The expression is embedded in an active, positively experienced interactional context.

Methodologically, the study demonstrates the utility of mixed human-LLM agent groups as a controllable environment for testing social psychological hypotheses that are difficult to examine in naturalistic settings. It highlights that agent design (persona, utterance style) is a critical experimental factor, not merely an implementation detail, as variations in agent behavior significantly influenced participant perceptions of the team.

Limitations Acknowledged by the Authors
The authors explicitly note several constraints:

• Effect Size: The average intervention effect was not statistically clear, suggesting that a single, short exposure may be insufficient for uniform identity shifts; repeated usage over time may be required.
• Group Size: The study used small groups (1 human + 4 agents); dynamics may differ in larger groups where conformity pressures are stronger.
• Simplification: Only one neologism was introduced per group, whereas real communities involve multiple competing terms.
• Generalizability: The study focused on Japanese speakers on Discord; cultural and platform-specific biases may limit external validity.
• Disclosure: Participants were informed that partners were AI, which may have influenced their interpretation of the interaction and the proposed word.