💬 NLP

Talk2AI: A Longitudinal Dataset of Human--AI Persuasive Conversations

Talk2AI is a large-scale longitudinal dataset comprising 3,080 conversations between 770 Italian adults and various large language models across four weekly sessions, designed to facilitate research on how AI-mediated dialogue influences opinion change, conviction stability, and human-AI interaction on topics like climate change, math anxiety, and health misinformation.

Original authors: Alexis Carrillo, Enrique Taietta, Ali Aghazadeh Ardebili, Giuseppe Alessandro Veltri, Massimo Stella

Published 2026-04-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Alexis Carrillo, Enrique Taietta, Ali Aghazadeh Ardebili, Giuseppe Alessandro Veltri, Massimo Stella

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand how a new, super-smart robot friend might slowly change your mind about things like climate change, math, or health rumors. Do you just talk to it once and see what happens? Or do you hang out with it for a month, day after day, to see if it really gets under your skin?

This paper introduces Talk2AI, a massive digital diary that does exactly that. It's not just a chat log; it's a longitudinal experiment (which is a fancy way of saying "a study that watches things change over time") designed to see how AI can persuade humans.

Here is the story of the study, broken down into simple parts:

1. The Cast of Characters

Think of this as a reality TV show, but instead of drama, the drama is about ideas.

The Humans: 770 real Italian adults were recruited. They weren't just random people; the researchers made sure they looked like a "mini-Italy" in terms of age, jobs, and education.
The Robots: Each human was paired with one of four famous AI "brains" (like GPT-4o, Claude, DeepSeek, or Mistral).
The Topics: They didn't chat about the weather. They debated three heavy-hitting topics: Climate Change, Math Anxiety (fear of math), and Health Misinformation.

2. The Game Plan: A Four-Week Marathon

Most studies are like a "speed date." You talk to the AI once, answer a few questions, and leave. That's like judging a movie after seeing only the trailer.

Talk2AI was a marathon.

The Schedule: Participants met their AI partner four times, once a week for a month.
The Conversation: Each session was a 10-turn debate (10 things you say, 10 things the AI says).
The Rules: The AI was programmed to be a tough opponent. It was told to find flaws in your arguments and keep the conversation focused. It couldn't just say "You're right!" to be nice. It had to challenge you.

3. The "Mind-Reading" Tools

Before and after every chat, the humans filled out surveys. Think of these as psychological X-rays.

Before the Chat: They measured things like "How much do you like thinking hard?" (Need for Cognition) and "How much do you trust robots?"
After the Chat: They asked: "Did your opinion change?" "Did you feel like you were talking to a human?" and even a tricky question: "If you had €100, how much would you donate to a charity about this topic?" (This measures if they actually acted on their new beliefs).

4. The "Gold Mine" of Data

The result is a dataset of 3,080 conversations (over 30,000 lines of text!).

The Files: Imagine a giant library where every book is a conversation.
- One file holds the bios of the people.
- One file holds their personality test scores.
- One file holds the actual chat logs (in Italian, but also translated into English, Spanish, German, etc.).
- One file holds the results (did they change their mind?).

5. Why This Matters (The "So What?")

The researchers wanted to fix a problem in how we study AI.

The Old Way: "I talked to a bot for 5 minutes, and it made me angry." (Too short, too shallow).
The Talk2AI Way: "I talked to a bot for a month, and it slowly convinced me that my fear of math was unnecessary."

This dataset allows scientists to see how persuasion works over time. It helps answer questions like:

Does a specific AI model (like GPT-4o) change your mind faster than another?
Do people who are naturally skeptical of technology get persuaded easier?
Can an AI make you feel like you're talking to a real human, and does that make you trust it more?

The Bottom Line

Think of Talk2AI as a time-lapse video of human belief. Instead of a single snapshot, it gives researchers a moving picture of how our minds shift when we argue with a machine. It's a massive toolkit for anyone trying to understand the future of human-AI relationships, ensuring we know not just if AI can persuade us, but how it does it, and who it persuades best.

1. Problem Statement

The paper addresses critical methodological gaps in current research regarding algorithmic persuasion and Human-AI Interaction (HAI):

Static vs. Dynamic: Existing studies often treat attitude change as a single end-point following one-shot exposures, failing to capture persuasion as a multidimensional, temporal process.
Artificial Constraints: Many experiments force participants to debate pre-specified claims that may not reflect their intrinsic baseline positions, limiting ecological validity.
Lack of Granularity: There is a scarcity of high-resolution data linking rich psychometric profiles, sociodemographics, and longitudinal conversational turns to specific persuasion outcomes.
Scalability of Influence: While evidence suggests LLMs are persuasive, the mechanisms by which repeated interactions alter belief certainty and social appraisal over time remain poorly understood.

2. Methodology

The study employed a within-subject longitudinal design conducted in Spring 2025 with the following specifications:

Participants:
- Initial Pool: 2,644 registered users from a certified Italian online panel (Bilendi Italia), stratified by age, gender, and region (18–69 years).
- Final Cohort: 770 fully profiled participants who completed all four waves and met strict quality control criteria.
Experimental Design:
- Duration: Four weekly sessions (4-week loop).
- Models: Participants were randomly assigned to one of four LLMs: GPT-4o, Claude 3.7 Sonnet, DeepSeek-chat V3, or Mistral Large.
- Topics: Each participant discussed one of three socially relevant topics consistently across all waves: Climate Change, Math Anxiety, or Health Misinformation.
- Interaction Structure: Each session consisted of exactly 10 conversational turns (20 messages total).
Data Collection Pipeline:
1. Registration: Collection of sociodemographics (age, gender, education, income, financial distress) and AI literacy.
2. Psychometric Profiling: Administered before each chat using 5-point Likert scales for:
  - TILLMI: Trust in Large Language Models Inventory (8 items).
  - NFC: Need for Cognition scale (6 items).
  - Big Five: Short-form personality inventory (10 items).
3. Conversational Interaction:
  - User Constraints: Minimum 50 words for the first message; interface provided "flashcards" to encourage challenging the AI (demanding details, counter-reasoning, questioning certainty).
  - AI Constraints: System prompts instructed models to identify fallacies, keep responses to ~100 words, and prevent topic switching.
4. Feedback & Metrics: Post-session collection of:
  - Q0: Conviction stability (1–100).
  - Q1: Self-reported opinion change (1–100).
  - Q2: Perceived humanness (1–100).
  - Q3: Behavioral endowment (Dictator Game proxy: allocation of €100 between self and a topic-relevant charity).
  - Q4: Qualitative open-ended feedback (min. 50 words).
Data Cleaning: A three-stage filtering pipeline removed incomplete sessions, "straight-lining" (zero-variance) responses, and participants failing cumulative completion thresholds, reducing the raw data to the final 770-user cohort.

3. Key Contributions

The paper introduces the Talk2AI dataset, a large-scale, longitudinal resource comprising:

Scale: 3,080 conversations totaling 30,800 conversational turns.
Rich Metadata: Linked sociodemographic data, psychometric profiles, and session-specific persuasion metrics for 770 users.
Multilingual Accessibility: While the raw data is in Italian, the dataset includes machine-translated versions in English, Spanish, Dutch, German, Portuguese, and French.
Structured Files:
- talk2ai_registries.json: Demographics and experimental assignments.
- talk2ai_psyscales.json: Longitudinal psychometric and feedback data.
- talk2ai_conversations.json: Raw and translated dialogue logs.
- data_table.csv: Aggregated tabular data with standardized factor scores for immediate statistical modeling.

4. Technical Validation & Results

The authors performed rigorous psychometric validation to ensure the reliability of the longitudinal data:

Factor Analysis:
- TILLMI: Validated as a unidimensional 7-item model (after removing one item due to cross-loading) with excellent fit (CFI = .998, RMSEA = .043) and high internal consistency ( $\alpha$ = .902).
- NFC: Validated as a two-factor model (NFC_seek and NFC_diligence) explaining 53.9% of variance, outperforming a unidimensional model.
Longitudinal Measurement Invariance (LMI):
- Both TILLMI and NFC scales demonstrated scalar invariance across all four waves ( $\Delta$ CFI $\le$ .011).
- Significance: This confirms that the psychological constructs were measured consistently over time, allowing for valid comparisons of latent trait changes across the study duration.
Data Integrity: The cleaning process successfully filtered out low-quality data, resulting in a robust dataset where every participant contributed complete psychometric and conversational data across all four waves.

5. Significance and Applications

The Talk2AI dataset offers a foundational resource for advancing research in cognitive science, NLP, and human-computer interaction:

Mechanisms of Persuasion: Enables the study of how specific semantic content, syntactic structures, and model architectures influence belief stability and opinion shifts over time.
Individual Differences: Facilitates the modeling of how personality traits (e.g., Need for Cognition) and baseline trust moderate susceptibility to AI persuasion.
Advanced Analytics:
- NLP: Supports fallacy detection, stance classification, and adversarial argumentation modeling.
- Predictive Modeling: Allows for supervised learning to forecast persuasion outcomes based on demographic and psychometric baselines.
- Sequence Modeling: Enables the use of RNNs or Markov chains to predict argumentative trajectories and conviction stability.
- Unsupervised Learning: Supports clustering of user typologies based on linguistic compliance and persuasion resistance.

In summary, Talk2AI moves beyond static, one-off interaction studies to provide a high-resolution, longitudinal view of how AI-mediated dialogue shapes human beliefs, offering a rigorous empirical foundation for understanding the future of human-AI coexistence.