💬 NLP

Games That Teach, Chats That Convince: Comparing Interactive and Static Formats for Persuasive Learning

This study compares static essays, chatbots, and text-based games for persuasive sustainability learning, revealing that while chatbots most effectively boost perceived importance, text-based games yield superior long-term knowledge retention despite lower subjective learning ratings, highlighting a critical dissociation between user engagement and actual learning outcomes.

Original authors: Seyed Hossein Alavi, Zining Wang, Shruthi Chockkalingam, Raymond T. Ng, Vered Shwartz

Published 2026-02-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seyed Hossein Alavi, Zining Wang, Shruthi Chockkalingam, Raymond T. Ng, Vered Shwartz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to convince your friends to start recycling or taking the bus instead of driving. You have a specific set of facts and arguments you want them to hear (e.g., "Recycling one ton of paper saves 17 trees").

The big question this paper asks is: How should you deliver these facts to get the best result?

Should you:

Hand them a pamphlet (Static Essay)?
Have a friendly chat with a robot (Chatbot)?
Put them inside a text-based adventure game where they play a character making choices (Text-Based Game)?

The researchers at the University of British Columbia set up a "taste test" to find out which method works best for learning and changing minds. Here is what they discovered, explained simply.

The Three Contenders

The Pamphlet (Essay): This is the old-school method. You just read the information. It's clear, but you are a passive observer, like watching paint dry.
The Robot Friend (Chatbot): You talk to an AI. You can ask questions, and it answers. It feels like a conversation.
The Adventure Game (Text Game): You are the hero of a story. You make choices, and the story changes based on what you do. The facts are hidden inside the plot, like Easter eggs in a treasure hunt.

Crucially, all three groups heard the exact same facts. The only difference was how they heard them.

The Big Surprises

The study found some fascinating, and sometimes contradictory, results. Think of it like a magic trick where what you feel isn't always what actually happens.

1. The "Feel-Good" Winner: The Chatbot

If you asked participants, "How much did you enjoy this?" or "How important does this topic feel to you now?", the Chatbot won hands down.

The Analogy: It's like talking to a charismatic tour guide. You feel heard, engaged, and the topic suddenly feels very important to you.
The Result: People felt they learned a lot, trusted the information more, and felt more motivated to care about the issue.

2. The "Stealth Learner": The Game

Here is the twist. The people who played the Text-Based Game said, "I didn't learn much," and rated their experience lower than the chatbot.

The Analogy: It's like studying for a test by playing a video game. You might feel like you're just having fun and not studying, but your brain is actually working overtime to connect the dots.
The Result: When tested 24 hours later on what they remembered, the Game players actually remembered the most facts, even though they thought they learned the least. The essay readers remembered the least.

3. The "Fake Engagement" Trap

The researchers looked at the "stats" of the interactions (how many words people typed, how long they talked).

The Analogy: Imagine a gym. Just because someone is sweating and lifting heavy weights (high engagement stats) doesn't mean they are building muscle (actual learning).
The Result: In the game, people typed very little but the system did a lot of talking. In the chat, people typed a lot. The researchers found that how much you talk or how long you stay in the system has almost nothing to do with how much you actually remember. You can have a "loud" conversation and remember nothing, or a "quiet" game session and remember everything.

The "Realism" Problem

The game players had one major complaint: It felt fake.
Because they were in a story with a hero and a villain, some people felt the facts were just "part of the fiction." They thought, "This is a game, so these numbers might not be real." This lowered their trust.
The chatbot, however, felt like a real conversation, so people trusted the facts more, even if they didn't remember them as well later.

The Takeaway for Designers

If you are building an app or a tool to teach people something important (like climate change), here is the lesson:

Want to make people feel passionate and important? Use a Chatbot. It's great for building trust and making the topic feel urgent.
Want people to actually remember the facts later? Use a Game. Even if they feel like they are just playing, their brains are encoding the information better.
Don't trust the "hype": Just because a user is typing a lot or staying on the screen for 20 minutes doesn't mean they are learning. Engagement metrics (like word count) are not the same as learning metrics.

In short: The Chatbot is the best salesman (it convinces you to care), but the Game is the best teacher (it helps you remember). The Essay is just... okay. And don't be fooled by how "busy" an interaction looks; true learning often happens quietly in the background.

1. Problem Statement

While interactive systems (chatbots, games) and static formats (essays) are increasingly used for sustainability education and persuasion, there is limited empirical understanding of how delivery format shapes learning and persuasive outcomes when the informational content is held constant.

Current evaluations often rely on subjective self-reports (e.g., perceived learning, engagement), which prior research suggests may not align with objective knowledge retention. This creates a design gap: it is unclear whether the interactivity of games and chatbots genuinely improves long-term retention or merely enhances the feeling of learning without improving actual outcomes. The study aims to disentangle the effects of content from interaction structure to identify trade-offs between interactivity, realism, and learning efficacy.

2. Methodology

The authors conducted a controlled, between-subjects user study ( $N=43$ ) comparing three information delivery modes:

Static Essay: Pre-generated persuasive essays containing fixed arguments and facts.
Conversational Chatbot: An LLM-driven agent engaging in free-form conversation, delivering the same facts dynamically based on user input.
Text-Based Game: A narrative-driven, interactive story where the player assumes a role, makes choices, and encounters the same facts woven into the plot.

Key Experimental Controls:

Content Consistency: All three conditions used an identical set of five persuasive arguments and supporting factual statements for two sustainability topics: Recycling and Public Transit.
System Architecture (PersuLab): A custom system ensured that all predefined facts were presented in every condition before the session could end. It utilized LLM-based fact-checking modules to track coverage in real-time for interactive conditions.
Data Collection:
- Pre-study: Baseline attitudes, behavioral intentions, and epistemic confidence.
- Post-study: Subjective ratings (Likert scales) on ease of following, engagement, enjoyment, trust, self-reported learning, and perceived attitude change.
- Delayed Assessment: A 24-hour objective knowledge quiz (multiple-choice) to measure long-term retention, including control questions to prevent guessing.
- Interaction Logs: Detailed logs of turns, word counts, and session duration for interactive conditions.

Analysis:
The study employed non-parametric tests (Kruskal-Wallis, Mann-Whitney U) for ordinal data, ordinal logistic regression for perceived change, and exploratory Spearman correlations to analyze the relationship between interaction metrics and outcomes.

3. Key Contributions

Dissociation of Subjective and Objective Learning: The study provides empirical evidence that perceived learning does not reliably predict objective knowledge retention. Specifically, participants in the game condition reported lower learning than those reading essays but achieved higher scores on the delayed knowledge quiz.
Format-Specific Persuasion Effects: It demonstrates that while interactive formats generally boost subjective experience, they have divergent effects on specific persuasive constructs. Chatbots were most effective at increasing the perceived importance of the topic, whereas games showed a unique advantage in factual retention.
Critique of Engagement Proxies: The research challenges the assumption that common engagement metrics (verbosity, turn count, session duration) are valid proxies for learning. The study found these metrics correlated more strongly with subjective experience than with actual knowledge retention.
The Realism-Trust Trade-off: The paper identifies a tension in narrative games: while they support memory, the fictional framing can reduce perceived realism and trust, potentially dampening persuasive impact compared to more direct formats like chatbots.

4. Key Results

A. Subjective Experience

Chatbot Superiority: The chatbot condition consistently received the highest ratings across subjective measures, including ease of following, engagement, enjoyment, trust, and self-reported learning.
Game Limitations: The text-based game received the lowest subjective ratings. Participants found it harder to follow and reported learning less, likely due to the cognitive load of the narrative structure and a perceived lack of realism ("it felt like a game, not real advice").
Essay Performance: Essays fell between the two interactive modes, often scoring high on clarity but lower on engagement.

B. Persuasive Outcomes (Attitude Change)

Perceived Importance: The chatbot condition led to a significantly greater increase in the perceived importance of the sustainability topic compared to both essays and games.
Behavioral Intention: No significant differences were found in changes to behavioral intention across the three modes. The authors attribute this to high baseline engagement among participants (many already recycled or used transit), leaving little room for measurable change.

C. Objective Knowledge Retention (The Critical Finding)

Game vs. Essay: Despite reporting the least learning, participants in the text-based game condition achieved the highest scores on the 24-hour delayed knowledge quiz (Mean = 2.60) compared to the essay group (Mean = 2.07).
Chatbot Performance: The chatbot group scored highest overall (Mean = 2.93), but the gap between chat and game was not statistically significant, while both outperformed the essay group.
Conclusion: Interactive formats (games and chatbots) appear to support better long-term factual encoding than passive reading, even when users feel they learned less.

D. Interaction Log Analysis

No Correlation with Retention: Interaction metrics (e.g., number of turns, total words, session duration) showed no significant correlation with objective knowledge scores.
Correlation with Subjectivity: These metrics did correlate with subjective impressions. For example, higher user contribution (more words) was associated with higher self-reported learning, while longer system verbosity was associated with lower perceived learning.

5. Significance and Design Implications

This work offers critical guidance for designers of Persuasive Technology, Serious Games, and Conversational Agents:

Outcome-Driven Design: Designers should not assume "more interactivity" equals "better learning."
- If the goal is subjective engagement and perceived importance, Chatbots are the superior choice.
- If the goal is long-term factual retention, Narrative Games (or interactive formats) may be superior, even if users feel less "educated" during the experience.
Avoiding Proxy Traps: Relying on engagement proxies (time spent, words typed) to evaluate educational effectiveness is flawed. These metrics reflect user experience, not retention.
Balancing Realism and Narrative: For serious games, designers must mitigate the "fiction penalty." To maintain trust and persuasive power, games should incorporate grounding strategies (e.g., real-world settings, credible sources) to prevent the narrative from undermining the message's credibility.
Evaluation Standards: Future evaluations of persuasive systems must jointly consider subjective reports, objective retention, and behavioral traces, rather than relying on self-reported learning as a standalone metric.

In summary, the paper reveals a fundamental tension: Chatbots convince users they are learning and care more, while Games help them remember facts better. Effective persuasive systems must choose their format based on whether the primary goal is immediate attitude salience or long-term knowledge retention.