Human gaze behaviors track abstract stimulus categories

This study demonstrates that human gaze behaviors encode abstract categorical information independent of physical stimulus features, paralleling non-human primate findings by showing that oculomotor signals reflect learned behavioral states and category distinctions.

The Gist

Imagine your eyes are like tiny, involuntary spies. Even when you think you're just staring straight ahead at a screen, your eyes are actually whispering secrets about what your brain is thinking.

This paper is about discovering that human eye movements carry a hidden code for abstract thoughts, specifically how we sort things into categories.

Here is the breakdown of the research using simple analogies:

The Big Idea: The "Leaking" Brain

For a long time, scientists thought our eyes were like a camera lens: they just pointed at what we were looking at. But this study suggests the eyes are more like a leaky faucet. Even if you are trying to be still, tiny drips of water (tiny eye movements) are leaking out, and those drips reveal what's inside the pipe (your brain).

The researchers wanted to know: If you teach a person a new, arbitrary rule for sorting things (like "tilted left is Blue, tilted right is Red"), can we tell which rule they are thinking about just by watching their eyes?

The Experiment: The "Secret Rule" Game

The researchers played a game with three groups of people:

1. The Training: They showed people a bunch of lines tilted at different angles. They didn't tell them the rule. Instead, they said, "Guess if this is Category A or Category B." If you guessed right, you got a "Correct!" cheer; if wrong, a "Try again."

   • The Twist: The "rule" was different for every person. For one person, a line tilted slightly left might be Category A. For another, that same tilt might be Category B. The brain had to learn a specific, invisible boundary.

2. The Test: While people played the game, the researchers recorded their eye movements with super-precise cameras.

What They Found: The Eyes Betray the Mind

1. The Eyes Know the Category (Even Before You Answer)
In the first experiment, people had to click a button to say "A" or "B." The researchers found that they could look at the eye movement data and guess, with better-than-chance accuracy, whether the person was thinking "Category A" or "Category B."

• Analogy: It's like watching a magician's hands. Even before the trick is revealed, the way they hold the card gives away the answer. The eyes were "leaking" the category information.

2. It's Not About the Buttons (The "Delayed" Test)
Critics might say, "Maybe their eyes just moved toward the 'A' button on the keyboard!"
To prove this wrong, the researchers used a Delayed Match-to-Category game (Experiment 2).

• The Setup: A shape appears (Sample). Then, a blank screen for a second (Delay). Then, a second shape appears (Test). The person has to say if they are the same category.
• The Catch: During the "Delay" period, the person doesn't know what the answer will be yet. They can't prepare to press a button.
• The Result: Even during that blank waiting period, the eyes were still moving in a way that revealed the category of the first shape.
• Analogy: Imagine you are holding a secret note in your pocket. Even before you decide to show it to anyone, your hand is twitching in your pocket in a specific way that reveals you're holding a note. The eyes were twitching with the "memory" of the category.

3. It's About the Task, Not Just the Picture (The "Color" Test)
In the third experiment, they showed the exact same pictures but changed the rules.

• Task A: Sort by the tilt of the lines (the abstract rule).
• Task B: Ignore the tilt! Just count if there are more black or white bars.
• The Result: When people were sorting by tilt, their eyes revealed the "tilt category." When they were sorting by color, their eyes revealed the "color category."
• Analogy: Imagine looking at a painting of a red apple. If you are a fruit expert, your eyes might focus on the stem. If you are a painter, your eyes might focus on the redness. The painting didn't change, but your attention changed, and your eyes followed your brain's new focus.

The "Secret Sauce": It's a Whisper, Not a Shout

The researchers found that you can't just look at one thing, like "how many times they blinked" or "how far they looked left." It's a complex pattern.

• Analogy: It's not like a single word being shouted. It's like a whole song being hummed very quietly. You can't hear the melody if you only listen to one note, but if you listen to the whole song, you can recognize the tune. The brain's "category" signal is a subtle mix of many tiny eye movements.

Why Does This Matter?

This study bridges a gap between humans and monkeys. We already knew monkeys' brains have "category neurons" in the part of the brain that controls eye movements. This study proves humans have the same thing.

It suggests that our brains don't just have a "thinking" part and a "moving" part. Instead, the part that moves our eyes is also busy doing our thinking. Our eyes are a window into our abstract thoughts, proving that how we look at the world is deeply connected to how we understand it.

In short: Your eyes are constantly broadcasting your brain's decisions. If you know how to read the signal, you can tell what category your brain is thinking about, even if you haven't said a word or pressed a button yet.

Technical Summary

1. Problem Statement

Categorization is a fundamental cognitive ability involving the grouping of stimuli based on behavioral relevance. While neurophysiological studies in non-human primates (NHPs) have established that abstract category information is encoded in oculomotor structures (specifically the Superior Colliculus and Lateral Intraparietal area) and manifests as small, uninstructed eye movements, it remains unknown whether this phenomenon exists in humans.

The central research question is: Does human gaze behavior encode abstract, rule-defined categorical information independent of low-level physical stimulus features and overt motor responses? Previous human studies have shown gaze reflects spatial working memory or stimulus features, but it is unclear if gaze tracks the decision state (the abstract category label) rather than just the sensory input.

2. Methodology

The authors conducted three experiments using high-resolution eye-tracking (SR Research Eyelink 1000 Plus, 500–1000 Hz) and machine learning decoding techniques.

General Experimental Design:

• Stimuli: Circular apertures containing flickering iso-oriented bars (30 Hz).
• Task: Participants learned to classify stimuli into "Category 1" or "Category 2" based on an arbitrary, participant-specific boundary (e.g., stimuli tilted clockwise vs. counter-clockwise relative to a random boundary).
• Training: Participants underwent a training phase to learn their specific boundary via trial-and-error feedback.
• Decoding Approach: The primary analysis used multivariate decoding (Linear Discriminant Analysis and Support Vector Machines) to predict category membership from trial-wise gaze position data ($x, y$ coordinates).
  • Time-resolved decoding: Analyzed gaze on a millisecond-by-millisecond basis.
  • Epoch-based decoding: Analyzed fixations within specific time windows (e.g., sample period, delay period).
  • Cross-validation: Leave-one-block-out or train/test splits to prevent overfitting.

Experiment 1: Speeded Categorization

• Task: Participants viewed oriented stimuli and pressed a key to indicate the category.
• Goal: Determine if category can be decoded from gaze during a standard speeded task.
• Controls: Ambiguous trials (stimuli exactly on the boundary) were used to test if decoding was driven by motor preparation (looking at the response key) rather than category.

Experiment 2: Delayed Match-to-Category (DMC)

• Task: A sample stimulus appeared, followed by a 1-second blank delay, then a test stimulus. Participants judged if the sample and test belonged to the same category.
• Goal: Dissociate category encoding from motor preparation. Since the correct response is undefined until the test stimulus appears, any category signal during the sample or delay period cannot be attributed to response planning.

Experiment 3: Task-Dependence and Feature Generalization

• Task: Participants performed two tasks with identical visual stimuli:
  1. Orientation Task: Categorize based on the learned orientation boundary.
  2. Color Task: Judge whether the display contained more white or black bars (ignoring orientation).
• Goal: Determine if gaze patterns reflect the current task set (abstract category) or fixed stimulus-driven biases. Cross-task decoding was also performed (training on one task, testing on the other).

3. Key Results

Experiment 1: Decoding Abstract Categories

• Robust Decoding: Category membership could be reliably decoded from gaze position above chance levels (50%), particularly between 0.7s and 1.4s after stimulus onset.
• Accuracy Correlation: Decoding accuracy was significantly higher on correct trials compared to all trials, suggesting gaze reflects the internal decision state.
• Motor Confounds Ruled Out: On ambiguous trials (where no category information exists), the classifier trained on unambiguous trials failed to predict the participant's motor response (key press). This indicates gaze shifts were not simply directed toward the response keys.
• Parameter Analysis: No single eye movement parameter (saccade count, amplitude, or direction) fully explained the decoding. Instead, category information was distributed across multiple weak parameters (vertical direction and amplitude were significant predictors).
• Distance-to-Boundary: Decoding accuracy was flat across different angular distances from the category boundary, ruling out a simple stimulus-driven gradient (e.g., larger eye movements for more extreme angles).

Experiment 2: Temporal Dissociation

• Delay Period Encoding: Category-selective gaze patterns emerged ~0.4s after the sample stimulus and persisted through the blank delay period (500–1500 ms).
• Significance: Since no motor response could be planned during the delay, these results prove that gaze tracks the maintained internal representation of the category, not just immediate sensory input or imminent motor output.
• Predictors: In this task, saccade rate was the primary predictor, further highlighting that the specific oculomotor features carrying category information are task-dependent.

Experiment 3: Task Dependence and Feature Generalization

• Task Specificity: Gaze patterns successfully decoded orientation categories during the orientation task but failed to do so during the color task (despite identical stimuli).
• Cross-Task Failure: A classifier trained on the orientation task could not decode orientation categories from gaze data collected during the color task.
• Color Decoding: Interestingly, gaze patterns also decoded the "color category" (white vs. black majority) during the orientation task, suggesting the oculomotor system can simultaneously track multiple feature dimensions, but the dominant signal depends on task relevance.
• Conclusion: The decoded signal reflects the observer's behavioral state and task set, not fixed stimulus-evoked biases.

4. Key Contributions

1. Human Analogue to NHP Findings: Provides the first behavioral evidence in humans that abstract, rule-defined category information is encoded in oculomotor behavior, paralleling findings in the Superior Colliculus of non-human primates.
2. Dissociation of Cognition and Motor Output: Demonstrates that category-selective gaze patterns exist even when motor responses are temporally decoupled (Experiment 2), confirming these are cognitive signals rather than motor preparation artifacts.
3. Task-Dependent Multiplexing: Shows that the oculomotor system multiplexes abstract decision variables. The specific gaze features carrying category information change based on the task (e.g., vertical direction in Exp 1 vs. saccade rate in Exp 2), supporting a "subspace alignment" model where cognitive and motor codes transiently align.
4. Methodological Advancement: Establishes that multivariate decoding of gaze position can serve as a non-invasive readout of internal categorical states, even when univariate analyses of individual eye movement parameters fail to show significant effects.

5. Significance

• Theoretical Implications: These findings challenge strictly modular views of cognitive function. They suggest that abstract decision variables are embedded within sensorimotor circuits (specifically oculomotor networks) rather than being confined to association cortices.
• Mechanism of Multiplexing: The results support the hypothesis that the brain uses a mechanism where cognitive states (category decisions) project weakly onto motor subspaces, creating small, "uninstructed" eye movements that carry high-level information.
• Clinical and Applied Potential: This research suggests that eye-tracking could be used as a non-invasive tool to monitor internal cognitive states, decision confidence, and category learning in real-time, potentially aiding in the diagnosis of cognitive disorders or the development of brain-computer interfaces that rely on naturalistic behavior.
• Conserved Architecture: The similarity between human gaze behaviors and NHP neurophysiology suggests a conserved computational architecture across primates for integrating abstract cognition with motor control.