Leveraging Human Reading Behavior for Keyphrase Extraction: A Webcam-based Eye-tracking Corpus

Technical Summary: Leveraging Human Reading Behavior for Keyphrase Extraction

Problem Statement
Keyphrase extraction (KPE) is a fundamental task in information retrieval, yet existing methods predominantly rely on text-internal cues such as term frequency, syntactic patterns, and contextual semantics. These approaches largely overlook the connection between keyphrases and human reading behavior, specifically how readers allocate attention during comprehension. While eye-tracking studies have established that fixation patterns correlate with cognitive processing, applying these insights to KPE is hindered by two main factors: the scarcity of eye-tracking data for Chinese academic texts and the high cost of traditional laboratory-grade eye-tracking equipment. Furthermore, existing open-source corpora (e.g., ZuCo, GECO) are primarily based on general content like news or novels, lacking the domain-specific terminology and structural conventions of academic literature, particularly in Library and Information Science (LIS).

Methodology
To address these gaps, the authors developed a comprehensive framework involving data collection, corpus construction, and model integration:

1. Lightweight Data Collection Platform: The study utilized a cost-effective, webcam-based approach integrating the open-source SearchGazer JavaScript library with a Flask-based backend. This platform allows for real-time gaze inference using consumer-grade webcams, eliminating the need for specialized hardware.
2. Corpus Construction (CLIS-ET): The authors constructed the Chinese LIS Eye-Tracking Corpus (CLIS-ET). Data was collected from ten LIS graduate students and senior undergraduates reading 320 abstracts from core Chinese LIS journals. The experiment involved a nine-point calibration before each abstract and a post-reading comprehension check.
3. Feature Extraction: The study focused on character-level eye-tracking metrics to avoid word segmentation issues. Three primary features were extracted and averaged across participants:
   • First Fixation Duration (FFD): The duration of the initial gaze on a character.
   • Fixation Number (FN): The total count of glances at a character.
   • Total Fixation Duration (TFD): The cumulative time spent gazing at a character.
     Data was filtered to retain valid fixations (16.7ms to 1500ms), and invalid character data (where >50% of participants lacked valid data) was discarded.
4. Model Integration: The eye-tracking features were incorporated into two categories of KPE models:
   • Recurrent Neural Networks: BiLSTM, BiLSTM+CRF, Attention-based BiLSTM (Att-BiLSTM), and Att-BiLSTM+CRF. Features were used as external inputs or attention indicators.
   • Pre-trained Language Models: BERT, RoBERTa, MacBERT, and Randeng-T5-784M.
5. Evaluation: Experiments were conducted on two datasets: a smaller eye-tracking dataset (Abstract-320) and a larger general KPE dataset (Abstract-5190) derived from the same journal sources. Performance was evaluated using F1 scores at top-3, top-5, and top-10 keyphrase extractions.

Key Contributions

1. Accessible Data Collection Method: The paper introduces a lightweight, scalable platform for collecting eye-tracking data using standard webcams, significantly lowering the barrier for domain-specific reading behavior research.
2. CLIS-ET Corpus: The release of the first domain-specific eye-tracking corpus for Chinese LIS academic abstracts, providing character-level metrics (FFD, FN, TFD) for cognitive KPE research.
3. Empirical Validation of Cognitive Signals: The study systematically evaluates how individual and combined eye-tracking features influence KPE performance across various model architectures, demonstrating that human reading behavior provides complementary signals to textual features.

Results

• Feature Efficacy: Integrating eye-tracking features consistently improved KPE performance across both datasets and model types.
  • On the Abstract-320 dataset, the FFD feature yielded the most significant improvement for BERT-based models (up to +2.83%).
  • On the larger Abstract-5190 dataset, the FN feature combined with RoBERTa achieved the highest performance (44.51% F1@5).
• Feature Combinations: The combination of Fixation Number and Total Fixation Duration (FN+TFD) achieved the best performance on the Att-BiLSTM+CRF model. Generally, combinations of FFD+FN and FN+TFD provided more stable improvements than using single features or the full triplet (FFD+FN+TFD).
• Statistical Significance: Paired t-tests on the Abstract-320 dataset confirmed that eye-tracking features exert statistically significant effects across multiple architectures. Specifically, FN+TFD and FFD+TFD showed consistent significance, suggesting that fixation frequency and duration capture complementary aspects of reading behavior.
• Dataset Scale Effects: The study observed that early-processing features (like FFD) offer greater informational gain when data is limited, whereas cumulative-processing features (like TFD) better support deep semantic modeling in larger datasets.

Significance and Claims
The paper claims that incorporating human reading behavior into computational models bridges the gap between statistical heuristics and cognitive processes. By demonstrating that lightweight webcam-based eye-tracking can effectively enhance KPE, the authors argue for a more accessible, cost-effective approach to gathering cognitive data for NLP. The findings suggest that eye-tracking features serve as valuable auxiliary signals that reflect the depth of semantic processing and the importance of lexical items, thereby improving the interpretability and performance of keyphrase extraction systems. The study positions this work as a step toward building human-centered information retrieval tools that align more closely with actual reader attention patterns.