Sure About That Line? Approaching Confidence-Based, Real-Time Line Assignment in Reading Gaze Data

1. Problem Statement

The paper addresses a critical bottleneck in assistive reading technologies (e.g., gaze-driven word highlighting, read-aloud tools) and eye-tracking research: real-time, reliable line assignment in multi-line reading scenarios.

• The Core Challenge: Eye-tracking data, particularly from remote or webcam-based systems, suffers from vertical noise (drift, slope, spatial jitter). This makes it difficult to determine which line of text a user is currently reading based solely on the $y$-coordinate of a fixation.
• Limitations of Existing Solutions:
  • Post-hoc methods: Most current algorithms (e.g., Dynamic Time Warping, clustering) require the entire reading session to be recorded before processing. They cannot support real-time interaction.
  • Heuristic/Constraint-based methods: Real-time systems often restrict natural reading behavior (e.g., disallowing regressions/re-reading or forcing strict sequential progression) to avoid errors.
  • Learning-based methods: Deep learning approaches (Transformers, CNNs) often require large labeled datasets, GPU acceleration, and pre-processing, making them unsuitable for low-latency, plug-and-play assistive devices.
• Specific Vulnerability: Children and beginning readers exhibit more variable gaze patterns, frequent regressions, and higher noise levels, causing existing algorithms to fail or require manual correction.

2. Methodology: CONF-LA

The authors propose CONF-LA (Confidence-score-based Online Fixation-to-Line Assignment), a principled, low-latency approach that integrates spatial evidence with behavioral priors.

Core Algorithmic Principles

CONF-LA treats line assignment as a probabilistic inference problem rather than a deterministic geometric mapping. It operates on the principle: "Only assign a fixation to a line if the confidence score exceeds a formal threshold; otherwise, defer the decision."

The system models the relationship between lines and gaze position using a Non-Homogeneous Hidden Markov Model (HMM):

1. Hidden States ($X_i$): Discrete line indices.
2. Observations ($Y_i$): Continuous $y$-position of the fixation.
3. Emission Probability: A Gaussian distribution centered on the line's $y$-coordinate. The model allows for an adaptive shift ($\mu_{shift}$) to account for vertical drift, which updates dynamically based on recent correction trends.
4. Transition Probability: A context-sensitive matrix ($L$) encoding behavioral priors of skilled reading:
   • Standard Reading: High probability of staying on the same line ($L_{l,l} = 0.85$).
   • Line Transitions: Moderate probability of moving to the next line ($L_{l,l+1} = 0.1$).
   • Regressions: Low probability of moving back ($L_{l+1,l} = 0.03$).
   • Context Updates: The transition matrix dynamically adjusts based on saccade velocity (high velocity suggests a return sweep/line change) and sudden $x$-position changes (large horizontal jumps suggest a line transition).

Inference and Decision Logic

The algorithm processes data in segments of size $S$ (where $S=1$ is pure real-time, and $S>1$ allows for local smoothing):

1. Posterior Calculation: For each fixation, it computes a posterior probability distribution over all possible lines.
2. Confidence Scoring: It calculates a confidence score ($c_i$) as the maximum posterior probability.
3. Thresholding:
   • If $c_i \geq C$ (threshold): The fixation is assigned to the line with the highest posterior.
   • If $c_i < C$: The assignment is deferred. The original $y$-coordinate is kept, and the fixation remains unassigned.
4. Resolution of Deferred Cases:
   • Local Temporal Filter: A short-horizon forward-backward pass within the segment resolves ambiguities.
   • Neighbor Voting: If a fixation remains unassigned, the algorithm checks neighboring fixations. If a sufficient percentage of neighbors agree on a specific line with high confidence, the unassigned fixation is updated.

3. Key Contributions

1. Confidence-Based Deferral Strategy: Unlike prior work that forces an assignment for every fixation (leading to errors), CONF-LA explicitly models uncertainty. It only corrects data when the model is "sure," preserving raw data for ambiguous cases.
2. Real-Time Viability: The algorithm achieves a mean per-fixation latency of 0.348 ms on standard CPUs, closing the gap between offline and online performance.
3. Behavioral Priors without Training: The model uses hardcoded, context-sensitive transition rules derived from reading theory (Rayner, 1998) rather than requiring large-scale supervised training. This makes it layout-agnostic and plug-and-play.
4. Robustness to Regressions: The method explicitly supports re-reading and backward jumps, a feature often excluded in real-time assistive tools.
5. Open-Source Implementation: The authors provide open-source code and benchmarking against the standard Carr et al. (2022) dataset.

4. Results and Evaluation

The authors evaluated CONF-LA on the Carr et al. dataset (adults and children) and the MECOde dataset, comparing it against 9 baseline algorithms (e.g., attach, chain, cluster, DTW).

• Accuracy vs. Latency:
  • Ad Hoc (Real-time, $S=10$): CONF-LA achieved 95% accuracy for adults and 94.2% for children, significantly outperforming the baseline attach algorithm (87.7% / 87.7%).
  • Post Hoc ($S=N$): Performance was comparable to state-of-the-art post-hoc methods (96.6% vs. 93.5% for attach), demonstrating that the online approach does not sacrifice accuracy for speed.
• Noise and Distortion Resilience:
  • CONF-LA showed invariance to segment size, maintaining high accuracy even when reducing the look-ahead buffer.
  • It performed exceptionally well under regression distortions (within-line and between-line), where other algorithms failed.
  • It demonstrated robustness in low-fidelity data (simulated webcam conditions with 30Hz sampling and spatial noise), outperforming baselines by ~10-11%.
• Generalization: When tested on the MECOde dataset without re-tuning parameters, CONF-LA maintained superior performance over the baseline, proving its ability to generalize across different experimental setups.
• Limitations: The algorithm showed susceptibility to extreme slope and drift distortions, likely due to the fixed nature of the Gaussian emission model. Additionally, the confidence scores were found to be miscalibrated (overconfident at high accuracy, underconfident at low), limiting their direct use in downstream probabilistic tasks without further calibration.

5. Significance and Future Impact

• Enabling Assistive Technologies: CONF-LA provides the first robust, real-time solution for line assignment that supports natural reading behaviors (regressions, skips) without requiring restrictive UI constraints or heavy machine learning infrastructure.
• Interpretability: By outputting confidence scores and explicit uncertainty, the system allows developers to build interfaces that adapt to user reliability (e.g., delaying highlighting if the system is unsure).
• Accessibility: The low-latency, CPU-friendly nature of the algorithm makes it accessible for deployment on consumer-grade devices (laptops, tablets) and low-cost eye trackers, democratizing access to gaze-based reading support for children and neurodivergent users.
• Future Directions: The authors suggest moving toward fully parametrized probabilistic models (e.g., using Dirichlet distributions for transitions) to improve confidence calibration and enable adaptive parameter tuning based on individual user behavior.

In summary, CONF-LA represents a shift from "forceful correction" to "confidence-aware inference," offering a stable, real-time foundation for the next generation of gaze-based reading support systems.