The People's Gaze: Co-Designing and Refining Gaze Gestures with General Users and Gaze Interaction Experts

This paper presents a two-phase methodology that combines co-design workshops with non-expert users and expert refinement to develop a grounded, intuitive set of 32 gaze gestures and design principles for hands-free interaction on gaze-enabled devices.

The Gist

Imagine you are holding a smartphone, but you want to control it without touching the screen. Maybe your hands are full of groceries, or perhaps you have a disability that makes using your fingers difficult. You want to use your eyes to tap, swipe, and click.

This is the dream of "gaze interaction." But there's a big problem: Your eyes are always looking at things. If the phone thinks every time you look at a button you want to press it, you'd accidentally delete your photos or close your apps constantly. This is known as the "Midas Touch" problem (everything you look at turns to gold/gets clicked).

This paper, titled "The People's Gaze," is a story about how the researchers solved this problem by mixing two very different groups of people: regular users (like you and me) and eye-tracking experts.

Here is the story of how they built a new language for your eyes.

🎨 Part 1: The "People's" Idea (The Co-Design Workshop)

The researchers gathered 20 regular people who had never designed a gaze gesture before. They gave them a simple task: "If you could control your phone with your eyes, what would you do?"

What the regular people came up with:

• The "Finger" Metaphor: Most people treated their eyes like a finger. They wanted to "swipe" left to go back or "tap" an icon to open it. They wanted the eye to act exactly like a touchscreen.
• The "Pause" and the "Blink": Realizing that just looking isn't enough, the regular users intuitively invented a two-step system.
  • Step 1: Stare at something for a moment (like a "hover" on a mouse).
  • Step 2: Blink to confirm the action (like a "click").
  • Analogy: It's like walking up to a door. You don't just walk through it; you pause, check if it's open, and then push. The "stare" is the check, and the "blink" is the push.
• The SOS Idea: Some people thought about emergencies. "If I'm in trouble and can't use my hands, I should be able to blink five times fast to call for help."

The Result: They came up with 102 different ideas. Many were creative, but some were physically impossible for human eyes to do without getting tired.

🔬 Part 2: The "Expert" Filter (The Peer Review)

Next, the researchers brought in 4 scientists who study how eyes move. They looked at the 102 ideas and said, "Hold on a minute. Let's talk about how your eyes actually work."

The Experts' Reality Check:

• Eyes are not fingers: Your eyes are built for jumping (saccades), not drawing. You can easily look from point A to point B in a straight line. But trying to draw a perfect circle or a spiral with your eyes? That's like trying to write a perfect cursive 'S' with a pencil while someone is shaking your hand. It's exhausting and inaccurate.
• The "Reading" Problem: If you ask someone to swipe their eyes from left to right, their eyes might just do that naturally because they are reading a sentence. The phone can't tell the difference between "reading" and "commanding."
• The Solution: The experts kept the best ideas but tweaked them.
  • They kept the "Stare + Blink" system because it solves the "Midas Touch" problem.
  • They threw away the complex circles and spirals.
  • They kept simple, straight-line movements (like a quick flick left or right) because that's what eyes do best.
  • They added a rule: The more dangerous the command, the harder the gesture.
    • Analogy: Changing the volume should be easy (a quick look). Turning off the phone or deleting a file should be hard (a complex sequence) so you don't do it by accident.

🏆 The Final Result: 32 New Gestures

After mixing the "People's" intuition with the "Experts'" science, they ended up with 32 perfect gestures.

The Big Takeaways (The "Rules of the Road"):

1. The "Activate-Then-Confirm" Grammar:
   Don't just look and click. Look, wait a split second (activate), then do something else (blink or move) to confirm. This separates "looking" from "doing."

   • Analogy: It's like a safety switch on a power tool. You have to hold a trigger and press a button to make it work.

2. Respect the Eye's Anatomy:
   Use straight lines and corners. Don't ask the eye to draw loops.

   • Analogy: Ask a horse to run in a straight line, not to dance a waltz.

3. Match the Effort to the Risk:
   Simple gestures for simple tasks. Complex gestures for dangerous tasks.

   • Analogy: You don't need a heavy, complicated key to open a garden shed, but you need a complex security code to open a bank vault.

4. Use Familiar Landmarks:
   Start gestures from the edges of the screen (like swiping from the side on a phone) because that feels natural.

🌟 Why This Matters

This paper proves that you don't need to be a scientist to design good technology. Regular people have great instincts about what feels natural. But they need experts to tell them what is physically possible.

By combining User Creativity (the "People's Gaze") with Scientific Reality (the "Expert Filter"), the researchers created a set of eye-gestures that are:

• Intuitive: You can learn them quickly.
• Safe: You won't accidentally delete your photos.
• Comfortable: Your eyes won't get tired.

This is a huge step forward for making phones and computers accessible to everyone, especially those who cannot use their hands. It turns the "Midas Touch" problem into a "Midas Control" solution.

Technical Summary

Here is a detailed technical summary of the paper "The People's Gaze: Co-Designing and Refining Gaze Gestures with General Users and Gaze Interaction Experts."

1. Problem Statement

As eye-tracking technology transitions from specialized labs to consumer mobile devices (e.g., iOS 18, Apple Vision Pro), there is a critical gap in interaction design. Current gaze gesture vocabularies are predominantly expert-designed and often fail to align with how non-expert users naturally move their eyes.

• The Core Challenge: Transforming raw eye-tracking data into expressive, discoverable, and user-friendly gestures is difficult due to the "Midas Touch" problem (distinguishing intentional commands from natural viewing) and the biomechanical limitations of the oculomotor system (eyes are optimized for ballistic saccades, not smooth tracing).
• The Gap: Existing gesture sets lack ecological validity for mobile contexts, often ignoring user mental models, ergonomic constraints, and the need for compositional structures to ensure intentionality.

2. Methodology

The study employs a two-phase mixed-method approach to bridge the gap between user intuition and technical feasibility.

Phase 1: Participatory Co-Design (Bottom-Up)

• Participants: 20 non-expert users recruited from the UK, divided into four groups (A, B, C, D).
• Procedure: Four co-design workshops were conducted using a 9-point spatial grid (3×3 anchor points) to standardize gaze paths and ensure technical feasibility.
• Process:
  1. Introduction: Participants were briefed on gaze interaction constraints (jitter, drift) and design principles.
  2. Ideation: Groups designed gestures for common mobile commands (e.g., open, return, zoom, copy) using sketches and rationales.
  3. Evaluation: Participants rated their own and peer groups' designs on five dimensions: Memorability, Ease of Execution, Mapping Clarity, Perceived Robustness, and Overall Impression.
• Output: 102 initial concepts were generated, which were deduplicated and filtered to 59 unique gestures representable on the 9-point grid.

Phase 2: Expert Peer Review (Top-Down)

• Participants: 4 domain experts in gaze interaction and eye-tracking research.
• Procedure: Modeled after academic peer review, experts independently evaluated the 59 user-generated gestures based on usability, novelty, and learnability.
• Consensus Building: A structured panel discussion (moderated by an Area Chair) resolved disagreements, leading to a final curated set.
• Output: The set was refined down to 32 validated gestures deemed ergonomic, distinct, and technically viable.

3. Key Contributions

A. The "Compositional Gaze Grammar"

The study identifies and formalizes a critical interaction pattern discovered by non-experts and validated by experts: Activate-then-Confirm.

• Structure: A two-stage sequence consisting of an Activation Primitive (e.g., a short dwell time to "arm" the interface) followed by a Confirmation Primitive (e.g., a blink, a specific scanpath, or a tap-stroke).
• Significance: This grammar explicitly separates "looking" (passive) from "acting" (intentional), effectively solving the Midas Touch problem by providing a discrete commit signal in a continuous input modality.

B. A Validated Gesture Vocabulary

The paper presents a set of 32 mobile-optimized gaze gestures derived from over 100 user ideas. Unlike previous expert-only sets, these gestures are grounded in user metaphors (e.g., swiping, tapping) but refined for ocular biomechanics.

C. Four Design Principles

The authors distill their findings into a framework for future gaze interaction design:

1. Establish a Clear Grammar of Intent: Use an activate-then-confirm syntax to distinguish passive observation from active commands.
2. Prioritize Physiological Reality over Naive Metaphor: Favor linear, saccade-friendly strokes (cardinal directions) over complex curves or spirals, as the eye is not designed for smooth tracing.
3. Match Interaction Cost to Operational Consequence: Apply "proportional complexity"—simple gestures for low-risk actions (e.g., next page) and complex, multi-step gestures for high-risk actions (e.g., shutdown) to prevent accidental triggers.
4. Anchor in Familiar Contexts: Ground gestures in existing mental models (e.g., "X" to close) and spatial landmarks (screen edges/corners) to reduce the learning curve.

4. Key Results & Findings

• User Intuition vs. Expert Reality:
  • Users naturally gravitated toward metaphorical designs (e.g., copying = clockwise rotation) and edge-anchored gestures (mimicking touch swipes). They independently invented the "dwell + blink" structure to manage intentionality.
  • Experts rejected gestures that were ergonomically unsound (e.g., spirals, long curves) or prone to false positives (e.g., single horizontal sweeps that mimic reading). They prioritized distinctiveness and saccadic efficiency.
• The "Expert-Novice Gap":
  • There was a significant divergence in rankings. Users preferred short, single-stroke gestures for fluency, while experts penalized them for lack of distinctiveness against natural eye noise.
  • However, the study found that with minimal scaffolding (a 9-point grid and brief instruction), non-experts converged on the same foundational principles (compositionality) that experts advocate, suggesting the design barrier is surmountable for general users.
• Quantitative Outcomes:
  • Phase 1: 59 unique gestures retained after deduplication.
  • Phase 2: 32 gestures retained after expert filtering.
  • Top-Ranked Gestures: Included directional flicks, "X" shapes for closing, and blink-based confirmations.
  • Lowest-Ranked Gestures: Complex spirals, continuous zooming paths, and gestures requiring precise curve tracing.

5. Significance

• Paradigm Shift: Moves gaze interaction design from a top-down, researcher-driven approach to a participatory, user-grounded methodology.
• Practical Application: Provides a ready-to-use, expert-validated gesture set specifically optimized for mobile devices, addressing constraints like handheld usage, lighting variability, and lack of tactile feedback.
• Accessibility: Offers a robust foundation for hands-free interfaces, crucial for users with motor impairments who rely on gaze as a primary control channel.
• Future Research: Establishes a collaborative framework (Co-Design + Peer Review) that can be applied to other emerging input modalities, proving that non-specialists can contribute meaningfully to complex interaction design when provided with appropriate scaffolding.

In conclusion, the paper demonstrates that by combining the semantic intuition of general users with the ergonomic and technical rigor of experts, it is possible to create gaze gesture vocabularies that are both intuitive to learn and robust enough for real-world deployment.