Controlling Reading Ease with Gaze-Guided Text Generation

Technical Summary: Controlling Reading Ease with Gaze-Guided Text Generation

Problem Statement
Controlling the readability of generated text is critical for applications ranging from second-language acquisition to accessibility for individuals with communication impairments. While Large Language Models (LLMs) have been applied to text adaptation using prompting or fine-tuning, these methods often lack transparency and precise control, frequently resulting in outputs misaligned with desired readability levels. Existing approaches struggle to provide a direct, interpretable signal for steering generation toward specific cognitive processing loads.

Methodology
The authors propose a gaze-guided text generation framework that steers LLM outputs to elicit specific reading behaviors (e.g., increased or decreased reading time) by integrating eye-tracking data into the decoding process. The system operates as an ensemble of two unidirectional models:

1. Language Model (LM): An off-the-shelf, instruction-tuned LLM (Llama 3.2, 3B parameters) that generates the probability distribution for the next token based on the preceding context.
2. Gaze Model: A fine-tuned GPT-2 model (124M parameters) trained to predict aggregated eye-tracking measures (specifically word-level First-Pass Reading Time, or FPRT) for candidate token sequences.

The Decoding Mechanism
The method modifies the standard beam search decoding process through a scoring re-ranking mechanism:

• The LM generates the top $k$ candidate tokens (where $k=8$).
• The Gaze model predicts the FPRT for each candidate sequence.
• A user-defined gaze weight ($w$) is applied to the gaze score.
• The final total score for a token is calculated as:
  $$\text{Total Score} = \text{Token Log Probability} + (w \times \text{Normalized Gaze Score})$$
• A positive gaze weight favors tokens predicted to induce longer reading times (higher difficulty), while a negative weight favors tokens with shorter reading times (lower difficulty). A weight of 0 disables the gaze guidance.

To address tokenization mismatches between the LM and the gaze model (which operates on subwords), the authors employ beam search. This allows the system to "revise" partial word predictions as the gaze model's tokenization stabilizes, preventing reliance on unreliable subword scores.

Key Contributions

1. Novel Control Mechanism: The introduction of a method using eye-tracking data as a direct signal to guide LLM text generation at inference time, avoiding the need for full model fine-tuning.
2. Dataset Release: The creation and public release of a new eye-tracking-while-reading dataset containing texts generated via this gaze-guided method, alongside the raw and processed eye-tracking data from the evaluation study.
3. Empirical Validation: A comprehensive analysis combining linguistic feature extraction, eye-tracking metrics, and human subjective ratings to demonstrate the method's effectiveness.

Experimental Results
The authors conducted an eye-tracking study with 24 participants (12 native English speakers [L1] and 12 non-native speakers [L2]) reading stories generated with gaze weights of $-2$, $0$, and $+2$.

• Text Characteristics: Increasing the gaze weight resulted in texts with fewer but longer words, lower word frequency (higher Zipf scores), shorter sentences, and higher lexical diversity. Extreme weights ($\pm 5$) caused repetitive failure modes.
• Reading Behavior: The method successfully altered reading times. Positive gaze weights increased First-Pass Reading Times (FPRT), while negative weights decreased them. The effect was statistically significant for both L1 and L2 readers ($p < 0.001$), though the magnitude was stronger for L2 readers.
• Linguistic Drivers: Statistical analysis revealed that the changes in reading behavior were primarily driven by lexical processing features, specifically word length and word prevalence (familiarity). When controlling for these features, the gaze weight effect became insignificant for L1 readers but remained significant for L2 readers, suggesting L2 readers faced additional processing challenges beyond simple lexical familiarity.
• Subjective Perception: Readers perceived texts with higher gaze weights as more difficult. Naturalness ratings decreased for L2 readers with positive weights, likely due to perceived difficulty rather than poor fluency. Interestingness ratings remained largely unaffected.
• Model Performance: The gaze model's predictions correlated moderately with observed reading times ($r \approx 0.55–0.59$), performing slightly better on low-prevalence words.

Significance and Claims
The paper claims that gaze-guided text generation is an effective, modular, and efficient approach to controlling reading ease without compromising text quality. The authors emphasize that the method successfully steers LLMs toward texts that are objectively easier or harder to read, as measured by cognitive effort (FPRT) and subjective difficulty.

The authors maintain a modest scope regarding the limitations of their findings:

• The observed effects are largely attributable to low-level lexical features (word length and frequency) rather than complex syntactic or semantic structures. This is attributed to the choice of FPRT as the target metric, which is known to reflect early cognitive processing.
• The approach is currently limited to English, as the training data for the gaze model is English-only.
• While the method shows promise for text simplification and personalized educational material, the authors note that their current implementation does not fully address syntactic or discourse-level simplification required for broader accessibility applications.
• The study did not include primary user groups for accessibility (e.g., individuals with specific communication impairments), limiting direct claims about the method's utility in those specific contexts.

The work represents a foundational step toward personalized text generation adapted to specific reader groups by leveraging eye-tracking data as a feedback signal for decoding strategies.