Towards Gaze-Informed AI Disclosure Interfaces: Eye-Tracking Attentional and Cognitive Load While Reading AI-Assisted News

Technical Summary: Towards Gaze-Informed AI Disclosure Interfaces

Problem Statement
As generative AI becomes integrated into journalism, news organizations face the challenge of designing AI-use disclosures that inform readers without imposing unnecessary attentional or cognitive burdens. While prior research has extensively examined the impact of disclosures on trust and credibility, the specific effects of disclosure design on readers' attentional and cognitive load remain underexplored. This gap is critical because higher cognitive load can reduce information retention and increase susceptibility to misinformation. Furthermore, current disclosures are non-standardized, ranging from minimal one-line labels to detailed transparency statements, yet it is unclear how these variations and the specific role of AI (editing vs. content generation) influence real-time reader processing.

Methodology
The authors conducted a 3 × 2 × 2 mixed-factorial laboratory study involving 34 participants (after exclusions for non-compliance). The study manipulated three variables:

1. AI Disclosure Detail (Between-subjects): No disclosure, one-line disclosure, and detailed disclosure.
2. News Type (Within-subjects): Politics and Lifestyle.
3. AI Role (Within-subjects): AI-assisted editing (minimal tweaks) vs. Partial AI content generation (substantial drafting).

Stimuli and Procedure:
Participants read 12 news articles (derived from real sources like BBC and CNN, modified using ChatGPT-4o) via a locally hosted web interface. Disclosures were placed at the top of the article. The study utilized a Latin square rotation to ensure counterbalancing across conditions.

Measures:
To capture load, the study employed a multimodal approach:

• Subjective: NASA-TLX (focusing on mental demand, effort, and frustration) administered after each article.
• Objective (Eye-Tracking): A Tobii Pro Fusion (120 Hz) recorded:
  • Cognitive Load: Pupil diameter.
  • Attentional Load: Total fixation duration and saccade count (reflecting visual search and processing difficulty).
• Qualitative: Semi-structured interviews were conducted to gather preferences and interpretations of the disclosures.

Key Results
The study revealed a significant divergence between attentional and cognitive load metrics:

1. Attentional Load (Eye-Tracking):

   • One-line disclosures resulted in significantly higher attentional load compared to both no disclosure and detailed disclosures. This was evidenced by increased fixation durations and saccade counts.
   • This effect was most pronounced for AI-edited content and lifestyle articles.
   • Detailed disclosures did not impose additional attentional burden compared to the no-disclosure baseline, despite containing significantly more text.
   • The increased load for one-line disclosures was not attributable to text complexity, as AI-edited articles (which had higher readability scores) still triggered higher load under one-line conditions.

2. Cognitive Load (NASA-TLX & Pupil Diameter):

   • No significant differences were found across any disclosure conditions (none, one-line, detailed) for either NASA-TLX scores or pupil diameter.
   • This indicates that while readers' visual attention patterns changed, their perceived mental effort and physiological arousal (cognitive effort) remained stable regardless of disclosure detail.

3. Qualitative Insights:

   • Participants found one-line disclosures ambiguous, leading to uncertainty about the AI's specific role.
   • Detailed disclosures were perceived as more transparent but were often viewed as too long, with participants drawing parallels to "cookie banners" that are skipped.
   • A strong preference emerged for "detail-on-demand" designs (e.g., a short label with an expandable option), allowing readers agency over the level of information.

Key Contributions

• Empirical Evidence of Attentional Cost: The study demonstrates that brief, ambiguous AI labels (one-line) trigger increased visual scrutiny and information-seeking behavior (higher fixation/saccades), whereas detailed labels do not.
• Methodological Validation: The research highlights that subjective measures (NASA-TLX) and physiological indicators of cognitive effort (pupil diameter) may fail to capture processing differences that are readily detected by attentional metrics (fixation/saccades). This underscores the necessity of eye-tracking for evaluating transparency interfaces.
• Theoretical Application: The findings support Information-Gap Theory, suggesting that one-line disclosures create an information gap (awareness of AI use without understanding its role) that drives visual search, whereas detailed disclosures resolve this uncertainty.

Significance and Claims
The paper claims that its findings inform the design of gaze-informed adaptive disclosure interfaces. The authors argue that because one-line disclosures increase attentional load (potentially reducing information retention) while detailed disclosures do not, but are often ignored due to length, a "detail-on-demand" approach is optimal.

The authors propose that real-time gaze sensing could enable interfaces to dynamically adjust transparency levels. For instance, if a reader exhibits high attentional load (indicating confusion or scrutiny) while viewing a brief disclosure, the system could automatically expand the disclosure to provide the necessary detail, thereby balancing transparency with cognitive and attentional burden. The study concludes that such adaptive designs are a promising path for user-centric news interfaces, though it acknowledges limitations regarding the homogeneity of the sample (high AI literacy) and the controlled lab setting.