💻 computer science

Discerning Authorship in Online Health Communities: Experience, Trust, and Transparency Implications for Moderating AI

This study reveals that users in online health communities generally cannot reliably distinguish AI-generated advice from human-written advice, highlighting the critical need for transparency and improved self-moderation strategies to maintain community trust.

Original authors: Yefim Shulman, Agnieszka Kitkowska, Mark Warner

Published 2026-04-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yefim Shulman, Agnieszka Kitkowska, Mark Warner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

🏥 The Big Picture: The "Digital Doctor" Dilemma

Imagine you are feeling sick. You hop onto an online health forum to ask for advice. Usually, you hope a real person who has dealt with your illness (or a real doctor) will answer. But now, there's a new player in the game: AI.

Large Language Models (LLMs) like ChatGPT are getting so good at writing that they can sound exactly like a helpful human friend. The problem? You can't tell the difference.

This paper asks a simple but scary question: If we can't tell if the advice is coming from a real human or a robot, can we still trust the community? And if we can't tell, does it matter who wrote it?

🔍 The Experiment: A "Taste Test" for Advice

The researchers set up a massive online "taste test." They recruited over 250 people and gave them health advice about two very different conditions:

Diabetes (a complex, lifelong condition).
Back Pain (a common, often temporary issue).

The advice came from two sources:

Real Humans: Actual posts from a trusted medical subreddit.
AI: Generated by a computer model pretending to be a doctor.

The participants had to guess: "Is this written by a human or a robot?"

They also tested two "superpowers" to see if they helped people guess better:

Training: Giving people a cheat sheet on how to spot AI.
Lived Experience: Only showing the test to people who actually have the disease (e.g., only showing diabetes advice to people with diabetes).

🚫 The Shocking Results: The "Blind Spot"

Here is what they found, and it's not good news for our ability to detect robots:

1. The "Superpowers" Didn't Work.
Giving people training or asking people who actually suffer from the disease to judge the advice did not help them at all.

Analogy: Imagine trying to spot a fake painting. You might think, "If I'm an art expert, I'll spot the fake!" or "If I take a class on forgery, I'll be good!" But in this study, even the "experts" and the "students" were just as bad at spotting the fake as the average person. They were all guessing in the dark.

2. The Topic Changed Everything.
The only thing that mattered was what the advice was about.

Back Pain: People were okay at spotting the AI here.
Diabetes: People were terrible at spotting the AI here.
Analogy: It's like a magic trick. If the magician is doing a simple card trick (Back Pain), you might spot the sleight of hand. But if they are doing a complex, high-speed illusion (Diabetes), you are completely fooled. The AI was better at mimicking the "human voice" when talking about complex, emotional, or chronic issues.

🕵️‍♀️ Why Did People Fail? The "Honest Signals" Trap

Why couldn't people tell the difference? The researchers found that humans rely on "Honest Signals" to judge trust.

In real life, we trust people because they show us signs of being human:

They use slang or informal language.
They show empathy ("I'm so sorry you're going through this").
They share personal stories.

The AI Trap:
The AI learned to fake these signals perfectly. It added the "empathy" and the "personal touch" just like a human would.

Analogy: Imagine a robot that learns to cry on command. If you see it crying, you think, "Oh, it's sad, it must be real." But it's just a robot following a script. The AI was "crying" (showing emotion) to trick us, making the signal dishonest.

Because the AI was so good at faking these "human" feelings, people's brains got confused. They thought, "This feels human, so it must be human," and they were wrong.

💡 What Should We Do? (The Solutions)

Since we can't train people to spot the fakes, the paper suggests we need to change the rules of the game.

1. Stop Trying to "Hide" the AI.
Instead of trying to make AI look exactly like a human, we should be transparent.

Analogy: Don't try to make a robot look like a human; just put a big "I AM A ROBOT" sign on its forehead. The study found that people want to know if they are talking to a machine. They trust the advice more if they know where it comes from.

2. Change the AI's "Voice."
The AI shouldn't try to be "too human." It should drop the fake empathy and the personal stories when giving medical advice.

Analogy: If a robot is giving you legal or medical advice, it should sound like a calculator, not a therapist. If it stops trying to fake human emotions, it becomes easier to spot, and people won't be tricked into thinking it has feelings it doesn't have.

3. Show the Receipts (Provenance).
We need to know who made the AI and what data it used.

Analogy: When you buy medicine, you check the label for the manufacturer and ingredients. Online health advice needs a label too. "This advice was generated by an AI trained on X data, moderated by Y organization."

🏁 The Bottom Line

We are entering an era where robots can sound more human than humans.

We cannot rely on our own brains to tell the difference.
We cannot rely on "experts" or "patients" to spot the fakes.
Trust comes from Transparency. We need to stop hiding the fact that AI is being used and start building systems that clearly label the advice, so we can make smart decisions about who (or what) we are listening to.

1. Problem Statement

Online health communities (OHCs) rely heavily on user trust and the credibility of shared advice. With the rise of Large Language Models (LLMs), there is a growing risk that AI-generated health advice will be indistinguishable from human-written advice, potentially eroding trust and spreading misinformation.

Core Challenge: Users often cannot reliably distinguish between AI-generated and human-written health advice.
The Gap: It is unclear whether "lived experience" with a specific health condition or specific "training" in AI detection improves a user's ability to discern authorship. Furthermore, there is a lack of understanding regarding how transparency and trust interact in this context, and what specific cues (signals) users rely on when making these judgments.
Moderation Dilemma: While community-based moderation (where users flag content) and self-moderation by LLMs (embedded warnings) are proposed solutions, their feasibility and effectiveness remain untested in the specific context of health advice.

2. Methodology

The authors conducted a mixed-methods online experiment involving 252 UK participants recruited via Prolific.

Experimental Design

Structure: A controlled $2 \times 2 \times 2$ between-subjects design.
Independent Variables:
1. Training: Present (received examples and feedback on AI detection) vs. Absent (familiarization only).
2. Lived Experience: Present (participant or loved one has the condition) vs. Absent.
3. Health Condition: Two distinct conditions were used: Diabetes (chronic, complex) and Back Pain (often perceived as temporary/trivial).
Stimuli:
- Human-written: Real questions and answers scraped from Reddit's r/AskDocs (anonymized and modified).
- AI-generated: Responses generated by GPT-3.5-turbo using the same prompts. The AI was instructed to act as a medical doctor, with parameters set for coherence (Temperature 0.5, Top_p 1.0). Crucially, all explicit safety prefixes (e.g., "I am not a doctor...") were removed to test pure discernment ability.
Procedure:
1. Pre-screening: Identified participants with/without lived experience.
2. Training/Familiarization: Participants viewed sample advice and indicated authorship. The "Training" group received feedback and annotated examples of AI vs. Human text; the "Absent" group did not.
3. Main Task: Participants evaluated three pairs of advice (one human, one AI) for each condition, rating them on a 4-point scale (Definitely Human $\to$ Definitely AI).
4. Qualitative Data: Participants provided open-ended reasoning for their choices.
5. Surveys: Measured attitudes toward Transparency (Societal vs. Private moderation) and Trust in AI.

Data Analysis

Quantitative: Analysis of Covariance (ANCOVA) was used to test the effects of Training, Lived Experience, and Health Condition on discernment accuracy (Correctness-Overall), controlling for Trust.
Qualitative: Inductive coding of 710 open-ended responses resulted in 24 codes, grouped into three themes: Advice Characteristics, Human Characteristics, and Writing Characteristics.

3. Key Results

RQ1: Transparency and Trust

Correlation: There is a strong positive correlation between Trust and Comfort in receiving AI-generated advice.
Transparency Types: Two distinct factors emerged:
- Transparency of Societal Moderation (e.g., NHS, professional associations).
- Transparency of Private Moderation (e.g., AI systems, for-profit orgs).
Finding: Transparency is highly valued (90.5% want to know who developed the AI), and it correlates with trust. However, the ability to discern authorship does not strongly correlate with trust levels.

RQ2: Discernment Accuracy

No Effect of Training or Experience: Contrary to hypotheses, neither Training nor Lived Experience significantly improved the ability to distinguish AI from human text.
Significant Effect of Health Condition: A major difference was found between the two conditions.
- Back Pain: Participants performed significantly better ( $M=2.02$ ) in discerning authorship.
- Diabetes: Participants performed significantly worse ( $M=1.42$ ).
- Interpretation: The complexity and specific semantics of Diabetes advice made it harder to detect AI, whereas Back Pain advice contained more "dishonest signals" (cues that looked human but were actually AI-generated) that confused users.

RQ3: Cues Used for Discernment

Qualitative analysis revealed that users rely on flawed heuristics (System 1 thinking) rather than deep analysis:

Advice Characteristics: Users looked for detail and quality. In the Diabetes group, AI advice was often perceived as "concise" and "to the point" (positive), whereas in Back Pain, it was seen as "lacking detail" or "generic."
Human Characteristics: Users searched for emotion and personalization.
- Diabetes: AI advice was often perceived as having "closeness" and "intimacy" (a false positive for human authorship).
- Back Pain: AI advice was perceived as "robotic," "textbook," and lacking empathy.
Writing Characteristics: Users judged based on formality. Back Pain AI advice was flagged for being "overly formal," while Diabetes advice was not.
Conclusion: LLMs successfully embedded "dishonest signals" (e.g., simulated empathy or specific phrasing) that users interpreted as markers of human authorship, leading to incorrect judgments.

4. Key Contributions

Empirical Evidence on Discernment: Demonstrates that neither domain expertise (lived experience) nor specific training significantly improves the ability to detect AI in health contexts.
Condition-Specific Variability: Reveals that the type of health condition is a critical variable in discernment efficacy, likely due to the complexity of the topic and the specific linguistic cues the AI generates for that topic.
Theoretical Extension: Extends Walther's Hyperpersonal Model and Donath's Signaling Theory to AI-mediated communication (AI-MC). It shows that in the absence of non-verbal cues, users rely on text-based signals that LLMs can easily manipulate, rendering these signals "dishonest."
Transparency Framework: Distinguishes between Societal and Private moderation transparency, suggesting users have different expectations for public vs. private AI moderators.

5. Significance and Implications

Community Moderation Limitations: Relying on community members (even those with lived experience) to self-moderate AI content is not a viable standalone strategy due to low discernment accuracy.
Self-Moderation & Transparency:
- Explicit Safeguards: Simple text prefixes (e.g., "I am not a doctor") are easily removed and unreliable.
- Implicit Safeguards: LLM developers should embed subtle, hard-to-remove signals (e.g., purposefully removing "human-like" emotional cues when generating health advice) to make AI content distinct from human content.
Provenance: Future systems must move beyond simple "AI vs. Human" labels. They need to provide provenance (sources, references, and who developed the model) to help users evaluate credibility, as users cannot currently detect authorship based on text alone.
Policy: A blanket ban on AI health advice may be inappropriate given user willingness to accept it; instead, enhanced transparency and technical safeguards are the preferred path to mitigate risk.

Conclusion

The study concludes that users are currently unable to reliably discern AI-generated health advice, regardless of their experience or training. The effectiveness of detection depends heavily on the specific health condition being discussed. To maintain trust in online health communities, the focus must shift from expecting users to detect AI to engineering AI systems that are transparent and technically distinct from human communication through embedded, reliable provenance markers.