Quality Perceptions and Intended Engagement in Response to AI-Generated and AI-Assisted News

Technical Summary: Quality Perceptions and Intended Engagement in Response to AI-Generated and AI-Assisted News

Problem Statement
The integration of Artificial Intelligence (AI) into news production, particularly through Large Language Models (LLMs), presents a critical challenge for understanding audience reception. While AI offers potential efficiencies, public skepticism remains high; prior research suggests audiences often distrust AI-generated content, particularly in political contexts, and prefer human-authored news. However, existing literature often conflates pre-existing attitudes with actual content evaluation, frequently assuming that aversion stems from perceived lower quality. Furthermore, most prior studies predate the capabilities of modern generative AI, and few have disentangled initial quality assessments (made without knowledge of authorship) from subsequent reactions to disclosure. This study addresses these gaps by examining whether audiences perceive differences in quality between human-written, AI-assisted, and fully AI-generated news when authorship is unknown, and how disclosure of AI involvement influences engagement intentions.

Methodology
The authors conducted a preregistered online between-subjects survey experiment in German-speaking Switzerland (N = 599). Participants were recruited via quotas for age and gender and randomly assigned to one of three conditions:

1. Human-written: Excerpts from original articles by journalists.
2. AI-assisted: Original articles rewritten by ChatGPT without information loss.
3. AI-generated: Short articles generated entirely by ChatGPT based only on the original title and lead.

Experimental Flow:

1. Quality Assessment (Blind): Participants read two 150-word excerpts and rated them on three dimensions of quality: expertise (clarity, coherence, comprehensiveness, conciseness, well-written), readability (boring, enjoyable, interesting, lively, pleasing), and credibility (biased, fair, objective) using a 1–5 Likert scale. Crucially, they performed this rating before knowing how the text was produced.
2. Disclosure: Participants were informed of the actual production method (human, AI-assisted, or AI-generated).
3. Engagement Intentions: Participants reported their willingness to (a) continue reading the specific article and (b) read AI-generated news in the future.

Key Results

• Perceived Quality (Blind): There were no statistically significant differences in perceived expertise, readability, or credibility across the three conditions. Equivalence tests (TOST) confirmed that readers evaluated AI-assisted and AI-generated articles as statistically equivalent in quality to human-written articles. Consequently, hypotheses H1–H6 (predicting lower quality ratings for AI conditions) were rejected.
• Short-Term Engagement (Post-Disclosure): Upon learning of AI involvement, participants in both the AI-assisted and AI-generated conditions reported a significantly higher willingness to continue reading the specific article compared to the human-written control group (moderate-to-large effect sizes, Cohen's d = 0.53 and 0.61, respectively).
• Long-Term Attitudes: Despite the increased immediate interest, there was no significant difference in the general willingness to read AI-generated news in the future across any of the three conditions. Hypotheses H9 and H10 (predicting increased future willingness) were rejected.
• AI-Assisted vs. AI-Generated: Exploratory analysis found no significant differences between the AI-assisted and fully AI-generated conditions across any measure.

Key Contributions

1. Decoupling Quality from Source: The study provides empirical evidence that, in the era of LLMs, the message quality (credibility, readability, expertise) of AI-generated news is perceived as comparable to human journalism when the source is unknown. This challenges the assumption that aversion to AI news is driven by perceived inferiority of the text itself.
2. The "Curiosity" Effect of Disclosure: The research demonstrates a partial decoupling of content quality from source acceptance. While disclosure did not degrade the perceived quality of the text, it triggered a short-term increase in engagement intentions. The authors interpret this not as genuine acceptance or trust-building, but as a curiosity-driven response or heightened scrutiny (an "automation heuristic") where readers want to re-examine the text to detect AI traces.
3. Distinction Between Situational and Sustained Interest: The findings highlight that a single exposure to AI-generated content, even with disclosure, is insufficient to shift broader, long-term attitudes toward AI in journalism. Situational interest (driven by novelty or incongruity) did not translate into sustained individual interest or future reading intentions.

Significance and Claims
The paper claims its findings contribute to a nuanced understanding of the relationship between AI transparency, reader perceptions, and journalistic credibility. It argues that while AI-generated news can match human news in perceived message quality, the disclosure of AI use does not necessarily "build trust" or "enhance engagement" in a meaningful, long-term sense. Instead, disclosure may momentarily spike curiosity without altering deeper skepticism or reading habits.

The authors remain modest in their conclusions, noting that the study measures self-reported intentions rather than actual behavior and relies on a single national sample. They emphasize that their design isolates the effect of disclosure on engagement but does not prove sustained acceptance. The study suggests that the "automation heuristic" remains a powerful force, where readers may judge the content as competent but remain hesitant about the source, indicating that transparency alone may not resolve the tension between AI efficiency and audience expectations of human accountability in journalism.