Hierarchical Resource Rationality Explains Human Reading Behavior

This paper proposes a hierarchical resource-rational framework that unifies eye movement control and comprehension during reading by positing that the brain optimizes these processes across nested time scales to maximize understanding while minimizing cognitive and temporal costs, a model that successfully replicates a wide range of human reading behaviors.

The Gist

Imagine your brain is a smart, budget-conscious project manager running a high-stakes construction site. The "project" is reading a text, and the "resources" you have are limited: you only have so much attention, so much short-term memory, and only a limited amount of time before the deadline hits.

This paper argues that the way humans read isn't random or purely mechanical. Instead, it's a series of strategic, rational decisions made by this project manager to get the most "understanding" out of the text while spending the least amount of "effort" and "time."

Here is how the paper breaks this down using simple analogies:

1. The Core Idea: "Resource Rationality"

Think of reading like shopping with a strict budget.

• The Goal: You want to buy the most valuable items (comprehension).
• The Constraint: You only have $20 (your limited attention and time).
• The Strategy: You don't buy everything. You skip the cheap, obvious items (easy words) to save money for the expensive, complex ones (hard words). If you realize you missed a crucial item, you go back and buy it (regression), but only if it's worth the extra cost.

The paper calls this Resource Rationality. It means your eyes and brain are constantly calculating: "Is looking at this word again worth the time it takes?"

2. The Three-Level Management Team (Hierarchy)

The authors propose that your brain doesn't just look at letters one by one. It uses a three-tiered management team to handle the job, much like a corporation:

• Level 1: The Foreman (Word Level)

  • Job: Identifying individual words.
  • Strategy: If a word is short and common (like "the"), the foreman glances at it quickly. If it's long and weird (like "uncharacteristically"), the foreman spends more time staring at it to make sure they got the letters right. They are trying to reduce uncertainty.
  • Analogy: Checking a cheap, familiar tool vs. inspecting a complex, unfamiliar machine part.

• Level 2: The Site Supervisor (Sentence Level)

  • Job: Making sure the sentence makes sense.
  • Strategy: This supervisor decides whether to skip ahead (if the sentence is easy) or regress (go back and re-read) if the sentence got confusing.
  • Analogy: If you're reading a recipe and the instructions are clear, you skip the next step. But if you realize you missed a crucial ingredient, you go back and re-read that step immediately.

• Level 3: The CEO (Text Level)

  • Job: Understanding the whole story or article.
  • Strategy: This is the big picture. The CEO decides which paragraphs to read, which to skim, and when to stop. They build a "mental map" of the text in long-term memory.
  • Analogy: A movie director deciding which scenes to keep in the final cut to tell the best story, rather than filming every single second.

3. The "Time Pressure" Experiment

The researchers tested this by making people read under a ticking clock (30 seconds, 60 seconds, 90 seconds).

• When time is tight (30s): The "Project Manager" panics. They stop checking details. They skip words, stop re-reading, and just try to get through the text as fast as possible. They sacrifice accuracy for speed.
• When time is plenty (90s): The manager relaxes. They check the difficult words, go back to fix mistakes, and build a deeper, more accurate understanding.

The Result: The computer model they built acted exactly like the humans. It knew when to skip and when to re-read based on how much time was left. This proves that humans aren't just "bad at reading under pressure"; they are optimizing their behavior to survive the deadline.

4. Why This Matters

Before this paper, scientists thought reading was either:

1. Just eyes moving based on word length (mechanical).
2. Just the brain building meaning (cognitive).

This paper says: It's both, and it's a calculation.

Your eyes move exactly the way they do because your brain is trying to solve a math problem: Maximize Understanding + Minimize Effort.

The Big Takeaway

Reading isn't a passive activity where information just flows into your brain. It is an active, strategic game. Your eyes are like a spotlight that you are constantly moving, deciding exactly where to shine it to get the most value for the energy you spend.

If you ever find yourself skipping a paragraph or re-reading a sentence, you aren't being lazy or confused. You are your brain's "Project Manager" making a smart, rational trade-off to get the job done with the resources you have.

Technical Summary

Here is a detailed technical summary of the paper "Hierarchical Resource Rationality Explains Human Reading Behavior."

1. Problem Statement

Reading is a complex, knowledge-intensive activity requiring the coordination of perception (eye movements), memory, and action under strict constraints of attention, working memory, and time. While existing theories have separately explained eye-movement control (focusing on lexical processing and local uncertainty reduction) and comprehension (focusing on memory construction and schema integration), a unified framework linking the two has been missing.

• The Gap: Current models either treat eye movements as reflexive responses to lexical features or treat comprehension as a separate process, failing to explain how evolving comprehension goals dynamically guide eye movements (e.g., skipping, regressions, rereading) in a resource-constrained environment.
• The Challenge: How can a computational model account for the adaptive trade-offs humans make between reading speed, accuracy, and effort across different time scales (word, sentence, text) without relying on hard-coded heuristics?

2. Methodology

The authors propose a Hierarchical Resource-Rational Control framework, implemented as a computational agent using Deep Reinforcement Learning (DRL) within a Partially Observable Markov Decision Process (POMDP) structure.

A. Core Theoretical Framework: Resource Rationality

The model posits that reading is an optimization problem where an agent seeks to maximize expected comprehension utility while minimizing cognitive and temporal costs (eye movement effort, time). The agent operates under uncertainty, maintaining probabilistic beliefs about word identity, sentence meaning, and text coherence.

B. Hierarchical Architecture (Three Nested Levels)

The model decomposes reading into three interacting controllers, each operating at a distinct temporal and informational scale:

1. Word-Level Controller (Lexical):
   • Goal: Reduce uncertainty in word identity.
   • Mechanism: Decides fixation locations within a word to sample letters. It uses Bayesian inference to update beliefs over candidate words based on visual input (foveal/parafoveal) and priors (frequency/predictability).
   • Action: Fixate specific letter positions or terminate recognition.
2. Sentence-Level Controller:
   • Goal: Integrate words into a coherent sentence representation.
   • Mechanism: Balances reading speed against information gain. It maintains a "comprehension belief" (integration quality) and a "prediction belief" (likelihood of upcoming words).
   • Action: Choose to move forward, skip a word (if prediction is high), regress (if comprehension belief is low), or terminate.
3. Text-Level Controller:
   • Goal: Construct a global "gist" and maintain text coherence.
   • Mechanism: Coordinates long-term memory construction. It monitors the coherence of the evolving text representation.
   • Action: Decide which sentence to read next or regress to earlier sentences to repair coherence failures.

C. Implementation Details

• Learning: Each controller is trained independently using Deep Reinforcement Learning (PPO algorithm) to learn policies that maximize cumulative reward (comprehension score minus effort/time costs).
• Observations: The agent receives noisy visual input (partial letters) and internal state updates (memory appraisals).
• Time Pressure: The model was extended to handle explicit time constraints by incorporating "remaining time" into the state space, forcing the agent to learn speed-accuracy trade-offs.
• Data: The model was validated against five datasets, including a new dataset collected by the authors for reading under time pressure (30s, 60s, 90s conditions), as well as established datasets for word recognition (Kliegl et al.), sentence reading (ZuCo 1.0), and text comprehension (McNamara et al.).

3. Key Contributions

1. Unified Mechanism: First computational model to jointly explain how comprehension guides eye movements and how eye movements support comprehension under a single principle (resource rationality).
2. Emergent Behavior: Demonstrates that complex reading behaviors (skipping, regressions, rereading) are not hard-coded rules but emerge naturally from the agent's optimization of utility under resource constraints.
3. Hierarchical POMDP Formalization: Successfully formalizes reading as a nested hierarchy of POMDPs, bridging the gap between low-level motor control and high-level cognitive goals.
4. New Empirical Data: Provides a new dataset of English reading under varying time pressures, capturing the joint dynamics of gaze behavior and memory formation.

4. Key Results

The model successfully replicated a wide range of human reading phenomena across all levels:

• Word Level:
  • Reproduced the effects of word length, frequency, and predictability on gaze duration.
  • Longer words increased fixation time; frequent/predictable words reduced it. The model achieved high correlation ($R^2 \approx 0.73-0.85$) with human data.
• Sentence Level:
  • Accurately predicted skipping rates (higher for short, frequent, predictable words) and regression rates (higher for difficult words).
  • Showed that regressions are strategic investments to repair comprehension, not random noise.
• Text Level:
  • Replicated the effects of prior knowledge and text coherence on recall scores.
  • Demonstrated targeted regressions: The agent selectively reread sentences with low comprehension scores to restore coherence, mirroring human behavior.
• Time Pressure Adaptation:
  • Under tight time limits (30s), the model adopted a "quantity-over-quality" strategy: faster saccades, increased skipping, reduced regressions, and lower comprehension accuracy.
  • Under loose limits (90s), it shifted to "quality-over-quantity": slower reading, more regressions, and higher accuracy.
  • The model matched human adaptations in both eye-movement metrics (scanpaths, heatmaps) and comprehension outcomes (MCQ accuracy, free recall).
• Ablation Studies:
  • Removing resource limits (unlimited memory) led to "super-human" performance but eliminated strategic eye movements.
  • Removing hierarchical structure (myopic agents) caused the model to stall locally or fail to advance through the text, confirming that hierarchical control and bounded resources are essential for human-like behavior.

5. Significance and Implications

• Theoretical Unification: The paper provides a principled account of how high-level cognitive goals (comprehension) and low-level motor actions (eye movements) are coordinated. It suggests that reading is a goal-directed, active information sampling process rather than a passive reflex.
• Generalizability: The hierarchical resource-rational framework is proposed as a general mechanism for other knowledge-intensive human behaviors (e.g., writing, sports, skill learning) where perception, memory, and action must be coordinated under uncertainty.
• AI and HCI: The approach offers a blueprint for building interpretable AI agents that learn human-like behaviors through optimization rather than imitation of massive datasets. It has direct applications in Human-Computer Interaction (HCI) for designing adaptive interfaces and in education for modeling reading proficiency and impairment (e.g., dyslexia).
• Cognitive Science: It validates the "Resource Rationality" hypothesis, showing that human cognitive limitations (memory, attention) are not bugs but features that drive adaptive, optimal behavior in complex environments.

In summary, this work establishes that human reading behavior is a boundedly optimal solution to a sequential decision-making problem, where the brain hierarchically allocates limited resources to maximize understanding while minimizing effort.