💻 computer science

AI as We Describe It: How Large Language Models and Their Applications in Health are Represented Across Channels of Public Discourse

This study analyzes how Large Language Models in healthcare are represented across five public discourse channels over two years, revealing generally positive and episodic narratives that increasingly overlook risks and technical mechanisms while varying significantly in tone and anthropomorphism across platforms.

Original authors: Jiawei Zhou, Lei Zhang, Mei Li, Benjamin D Horne, Munmun De Choudhury

Published 2026-03-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiawei Zhou, Lei Zhang, Mei Li, Benjamin D Horne, Munmun De Choudhury

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the public conversation about Artificial Intelligence (AI) in healthcare as a massive, noisy town square. In this square, there are five different "stages" where people are talking: News Stations (professional journalists), University Press Releases (scientists), YouTube (educators and creators), TikTok (trendy, short videos), and Reddit (forums for deep dives and personal stories).

This paper, written by researchers from Georgia Tech and the University of Tennessee, went into this town square to listen to what everyone was saying about AI doctors and AI health assistants over the last two years. They wanted to know: Are we telling the whole story, or just the highlight reel?

Here is what they found, explained simply:

1. The Mood is Mostly "Hype" (The Party Vibe)

If you walked into this town square, the overall vibe would feel like a block party. Almost everywhere, people are excited. They are talking about how cool AI is, how it can save time, and how it might help with mental health.

The Trend: The party is getting louder and more positive over time.
The Difference: The "University Scientists" and "News Reporters" are talking in a serious, analytical way (like a lecture hall). But the "TikTok" and "Reddit" crowds are more emotional, using words that sound like they are talking to a friend or a confidant.

2. The Story is "Episodic" (The "One-Off" Problem)

Most of the stories being told are like isolated movie scenes rather than a full documentary.

What they say: "Look! AI helped this one person write a diet plan!" or "AI gave a wrong answer about a headache!"
What they miss: They rarely talk about the big picture. They don't discuss how AI might change the whole healthcare system, how it affects doctors' jobs, or the long-term societal rules we need. It's like watching a trailer for a movie but never seeing the plot.

3. The "Magic Trick" is Hidden (The Black Box)

This is the most critical finding. The researchers noticed that people are treating AI like a magic 8-ball or a genie in a bottle.

The Problem: People think AI "knows" things because it's smart. But AI is actually just a predictive text machine (like a super-advanced autocomplete). It guesses the next word based on patterns, it doesn't "understand" medicine.
The Gap: Almost no one in the public square is explaining this. They aren't saying, "Hey, this tool is guessing, not thinking." Because of this, people might trust a wrong answer from an AI just as much as a fact from a textbook.
The Risk: When people treat a guessing machine like a human doctor, bad things can happen. The paper mentions scary cases where people have been led astray by AI, even to the point of self-harm, because they thought the AI was a real therapist.

4. The "Risk" Talk is Shallow

When people do talk about dangers, they usually say, "The AI made a mistake" or "It gave fake info."

The Metaphor: It's like saying a car is dangerous because "it got a flat tire." They aren't talking about the fact that the car has no brakes (the fundamental design flaw of AI).
Who talks about risk? The "News" and "Scientists" talk a bit more about risks. But on TikTok and Reddit, the conversation is almost entirely about how great AI is for mental health and companionship. They rarely mention the dangers.

5. The "Human" Illusion (Anthropomorphism)

On TikTok and Reddit, people are starting to treat AI like a living friend.

They say things like, "My AI therapist cares about me" or "My chatbot understands my pain."
The researchers found that while professional outlets keep AI as a "tool," the social media crowd is turning it into a "person." This is dangerous because you can't sue a chatbot for malpractice, and a chatbot can't feel empathy.

The Big Takeaway: A Diagnostic Tool

The authors suggest that listening to this "town square" is like a doctor's check-up for society.

The Diagnosis: We are currently very excited about AI in health, but we are illiterate about how it actually works. We are missing the "user manual" that explains its limitations.
The Prescription: We need better communication. We can't just say "AI is here." We need to explain how it works (it's a guesser, not a thinker) and where it fails.
The Goal: We need to help people become "critical drivers" of this technology, not just "passengers" who blindly trust the AI to steer the car.

In short: The public is falling in love with AI health tools, but they are wearing rose-colored glasses. They see the magic but miss the mechanics, and they aren't being warned enough about the traps hidden in the road.

1. Problem Statement

The rapid adoption of Large Language Models (LLMs) in high-stakes domains like healthcare has outpaced public literacy and regulatory oversight. Unlike traditional information retrieval tools, LLMs are generative (probabilistic "next-token predictors"), which creates a risk of users developing flawed mental models (e.g., anthropomorphizing the AI or assuming authoritative grounding).

The Gap: While prior research has examined public attitudes toward AI in specific contexts, there is a lack of holistic understanding regarding how LLMs are represented across diverse discourse channels (both professional and public).
The Risk: Misaligned narratives can lead to real-world harms, including reliance on hallucinated medical advice, erosion of critical thinking, and psychological distress (e.g., "AI psychosis" or suicidal ideation linked to AI interactions).
Objective: To analyze how public discourse introduces LLMs in health, specifically examining the balance between potential benefits, risk communication, and the explanation of the technology's generative nature.

2. Methodology

The study employed a large-scale, multi-platform quantitative analysis of public discourse over a two-year period (December 2022 – December 2024), covering the post-ChatGPT release era.

Data Collection & Filtering

Channels: Five distinct discourse channels were analyzed: News (mainstream media), Research Press (scientific press releases), YouTube, TikTok, and Reddit.
Corpus: Initial collection yielded 62,783 items using keyword combinations (LLM terms + Health terms).
Filtering: An LLM-assisted filtering process (using GPT-4o-mini) refined the dataset to 21,773 items that specifically discussed the adoption of LLMs for health-related uses, removing false positives (e.g., using AI to write essays).
Validation: Human annotation on a 100-item sample confirmed high reliability (F1-macro scores ranging from 0.74 to 0.90; Cohen's $\kappa$ up to 0.79).

Analytical Dimensions

The study analyzed three core dimensions based on Agenda-Setting Theory:

Lexical Style:
- Emotional Tone: Measured via LIWC (Linguistic Inquiry and Word Count) on a 0–100 scale.
- Writing Style: Analyzed using LIWC metrics for Analytic (logical/formal), Clout (confidence/authority), and Authentic (spontaneous/self-disclosure) styles.
Informational Content:
- Framing: Classified as Episodic (specific cases/stories) vs. Thematic (broad societal context) and mapped to 14 policy dimensions (e.g., economy, ethics, safety).
- Risk & Mechanism Disclosure: Categorized using a health-specific risk taxonomy (Individual, Human-centered care, Information ecosystem, Technology accountability) and checked for explanations of the generative nature of LLMs.
- Health Subdomains: Categorized into six broad groups (e.g., Clinical Decision Support, Consumer Health Support).
Symbolic Representation:
- Anthropomorphism: Calculated using a masked language model approach to determine the probability of an LLM entity being framed as human vs. non-human in a sentence.

Statistical Analysis

Kruskal–Wallis H-tests were used to identify significant differences in distributions across the five channels.
Dunn's post hoc tests with Bonferroni correction were applied for pairwise comparisons.

3. Key Contributions

First Cross-Channel Discourse Analysis: The first study to simultaneously analyze "top-down" (professional) and "bottom-up" (public) discourse on LLMs in health across five distinct media types.
Taxonomy of Health Applications: Developed and validated a categorization of six broad health subdomains and 20 specific subcategories where LLMs are applied, providing a framework for future deployment evaluation.
Diagnostic Tool for Literacy Gaps: Established public discourse as a diagnostic mechanism to identify gaps in AI literacy and governance, specifically highlighting the disconnect between public enthusiasm and risk awareness.

4. Key Results

A. Lexical Style & Tone

Overall Positivity: Discourse is generally positive and has become increasingly positive over time.
Platform Divergence:
- TikTok & YouTube: Showed the most optimistic tone, high "Clout" (confidence), and lower "Analytic" scores. Creators often present themselves as persuasive influencers.
- News: Maintained a neutral tone, significantly more negative than social media, with high "Analytic" scores reflecting journalistic norms.
- Reddit: Showed the widest variation in emotional tone, reflecting its community-driven, polarized nature.

B. Informational Content

Episodic Framing Dominance: Narratives are overwhelmingly episodic (focusing on isolated cases, specific tools, or individual stories) rather than thematic (systemic implications). News was the only channel with a significant proportion of thematic framing (31.5%).
Risk Communication Deficit:
- Risks are rarely discussed thoroughly. When mentioned, they are almost exclusively framed as information quality issues (e.g., hallucinations, inaccuracies, bias).
- Generative Nature: Explanations of the probabilistic, generative nature of LLMs (distinguishing them from search engines) were nearly absent (<3% in most channels, ~31% in YouTube).
- Channel Disparity: Layperson platforms (TikTok, Reddit) were significantly less likely to mention risks or mechanisms compared to News and Research Press.
Health Subdomains:
- TikTok & Reddit: Heavily focused on Consumer Health Support, specifically Mental Health (therapeutic support, emotional companionship).
- News & Research Press: Focused on Clinical Decision Support (diagnosis, treatment planning) and Research/Data Analysis.
- YouTube: Served as a middle ground with broader coverage, including professional training and data analysis.

C. Symbolic Representation (Anthropomorphism)

General Trend: Overall anthropomorphism was low to neutral across all channels.
Outliers: TikTok and Reddit exhibited the widest range of anthropomorphism scores. While most content was neutral, these platforms contained highly anthropomorphized examples (e.g., "my ChatGPT therapist knows what's up," "he fell in love with his AI bot").
Professional Outlets: Research Press showed the lowest anthropomorphism, adhering to technical descriptions.

5. Significance & Implications

For Communication & Design

Beyond Surface Disclosure: Simply telling users "this is AI" is insufficient. The study highlights a critical need to educate users on the mechanistic differences between LLMs (probabilistic generation) and traditional tools (retrieval).
Addressing the "Mental Health" Gap: There is a high public demand for AI in mental health (especially on TikTok/Reddit) but a lack of professional oversight and risk communication in these spaces. This creates a safety vacuum where users may rely on unverified AI for therapeutic support.

For Governance & Policy

Targeted Regulation: Regulatory efforts should prioritize domains with high public demand but low risk awareness (e.g., AI companionship for mental health).
Literacy Interventions: Policies should move beyond professional training to include "in-situ" literacy scaffolds that help the general public critically evaluate AI outputs in real-time.

For Future Research

The study suggests that public discourse acts as a "negotiation space" where AI perceptions are actively formed. Future work should investigate how visual elements and narrative depth (beyond text) influence these perceptions and how to design interventions that correct flawed mental models without stifling innovation.

Conclusion

The paper concludes that while public discourse on LLMs in health is currently optimistic and growing, it suffers from a systemic lack of risk communication and mechanistic explanation. The disparity between professional outlets (which focus on clinical/systemic issues) and social media (which focus on personal/mental health needs with high anthropomorphism) creates a dangerous gap in public understanding. Bridging this gap requires coordinated efforts in communication strategy, design (to support mechanistic understanding), and governance.