Identification of fixations and saccades in eye-tracking data using adaptive threshold-based method

1. Problem Statement

Eye-tracking data analysis relies on distinguishing between fixations (stable gazes) and saccades (rapid eye movements). While this distinction is fundamental to cognitive research, clinical diagnostics, and human-computer interaction, current methods face significant challenges:

• Reliance on Fixed Thresholds: Standard algorithms (e.g., I-VT, I-DT) use static, ad-hoc thresholds (e.g., velocity > 30°/s) that ignore inter-task and inter-individual variability in oculomotor dynamics.
• Sensitivity to Noise: Fixed thresholds are highly sensitive to measurement noise, blinks, and tracking loss. Even moderate Gaussian noise can cause rapid degradation in classification accuracy, leading to biased statistics.
• Lack of Adaptability: Existing adaptive methods often require complex modeling (e.g., Hidden Markov Models) or detailed distribution assumptions, making them computationally expensive or difficult to implement.

The authors aim to develop a parsimonious, adaptive thresholding method that optimizes classification performance based on the specific characteristics of the data and task, while maintaining robustness against noise.

2. Methodology

2.1 Data Collection

• Participants: 15 adults with high-quality eye-tracking data.
• Tasks: Two distinct paradigms were used:
  1. Free Viewing (Random Pixel): Viewing unstructured pixel fields.
  2. Visual Search (Waldo): Locating targets in complex "Where's Waldo?" scenes.
• Hardware: EyeLink Portable Duo at 1000 Hz sampling rate.
• Reference Labels: The study used the standard EyeLink parser (velocity 30°/s, acceleration 8000°/s²) as a practical benchmark for ground truth, acknowledging it is a threshold-based classifier itself.
• Noise Simulation: To test robustness, the authors superimposed tunable Gaussian noise ($\epsilon \sim N(0, \sigma^2)$) onto the raw trajectories, testing noise levels ($\sigma$) from 0 to 50 pixels.

2.2 Classification Algorithms Evaluated

The study evaluated three threshold-based algorithms, introducing adaptive versions for each:

1. I-VT (Velocity-Threshold Identification): Classifies based on instantaneous velocity ($V_i$).
2. I-AVT (Angular Velocity-Threshold Identification): A variant using effective angular velocity ($V^{eff}_i = V_i \cdot \cos(\theta_i - \theta_{i-1})$), combining speed with directional persistence.
3. I-DT (Dispersion-Threshold Identification): Classifies based on the spatial dispersion of gaze points within a moving time window.

2.3 The Core Innovation: K-Ratio Optimization

The authors introduced a novel metric called the K-ratio to determine optimal thresholds adaptively.

• Concept: The K-ratio compares the empirical number of fixation-to-saccade transitions observed in the data against the expected number of transitions if the labels were randomly distributed (independent of state).
• Formula:
  $$K\text{-ratio} = \frac{n_{F \to S}}{n_S(1 - n_S)}$$
  Where $n_{F \to S}$ is the frequency of fixation-to-saccade transitions, and $n_S$ is the proportion of saccades.
• Optimization Strategy: The optimal threshold is the one that minimizes the K-ratio.
  • A K-ratio < 1 indicates that states (fixations/saccades) tend to cluster (temporal coherence).
  • Minimizing this ratio ensures that the algorithm selects thresholds that maximize the separation between states, reducing spurious switching caused by noise.

3. Key Results

3.1 Baseline Performance (Noise-Free)

• I-VT achieved the highest overall accuracy (90–93%) and the best balance between fixation and saccade detection (highest Saccade F1-score: ~0.51–0.54).
• I-AVT showed high fixation accuracy but significantly under-detected saccades.
• I-DT produced the most fixation-dominated segmentation, often merging short saccades into fixations, resulting in low saccade recall.
• Task Dependence: Optimal thresholds varied significantly between the "Random Pixel" and "Waldo" tasks, validating the need for task-specific tuning.

3.2 Robustness to Noise

• Fixed Thresholds: Performance degraded rapidly as noise increased. At high noise levels ($\sigma \ge 30$ px), accuracy for fixed-threshold I-VT and I-AVT dropped below 20%, with most data misclassified as saccades due to inflated velocity estimates.
• Adaptive Thresholds (K-ratio): Re-optimizing thresholds at each noise level significantly mitigated performance loss.
  • I-DT (Adaptive): Demonstrated superior noise robustness, maintaining accuracy > 81% even at extreme noise levels ($\sigma = 50$ px).
  • Trade-off: This robustness came with a precision-recall trade-off. Adaptive I-DT maintained high precision for fixations (conservatively classifying uncertain points as fixations) but suffered from very low saccade recall (F1 < 0.11) at high noise.
  • I-VT (Adaptive): Improved significantly over fixed thresholds but remained more sensitive to noise than I-DT.

3.3 Participant Variability

• The optimal thresholds derived via K-ratio minimization showed high intra-participant consistency (Intraclass Correlation Coefficients, ICC, ranging from 0.68 to 0.85) across trials, confirming that the method captures stable individual oculomotor characteristics rather than random noise.
• However, thresholds varied significantly between participants, justifying the need for individualized or task-specific adaptation rather than a universal global threshold.

4. Key Contributions

1. Adaptive K-Ratio Method: Introduced a simple, model-free statistical metric to automatically tune thresholds for any threshold-based eye-tracking algorithm, minimizing unnecessary state transitions.
2. Comprehensive Noise Analysis: Provided a systematic evaluation of how Gaussian noise affects I-VT, I-AVT, and I-DT, demonstrating that fixed thresholds are insufficient for noisy or mobile data.
3. Algorithm Benchmarking: Established that while I-VT is the best general-purpose algorithm for clean data, I-DT with adaptive thresholds is the most robust choice for noisy environments, provided the user accepts a bias toward fixation detection.
4. Practical Guidelines: Offered actionable advice for researchers:
   • Use I-VT for high-quality data where saccade detection is critical.
   • Use Adaptive I-DT for noisy data or applications where identifying stable fixations is the primary goal.
   • Avoid fixed thresholds when data quality varies or noise is present.

5. Significance and Conclusion

This paper addresses a critical bottleneck in eye-tracking research: the gap between theoretical algorithmic simplicity and the messy reality of experimental data. By demonstrating that a Markovian approximation (K-ratio) can effectively adapt thresholds without complex modeling, the authors provide a scalable solution for improving classification accuracy.

The findings highlight that no single algorithm is universally superior; the choice depends heavily on data quality (noise level) and analytical priorities (saccade detection vs. fixation stability). The proposed adaptive method allows researchers to optimize their analysis pipelines dynamically, reducing bias in cognitive and clinical studies caused by rigid, one-size-fits-all thresholds. Future work suggested includes integrating K-ratio tuning with advanced denoising techniques and exploring its application in learning-based classifiers.