Do Developers Read Type Information? An Eye-Tracking Study on TypeScript
This eye-tracking study involving 26 undergraduate students reveals that developers do not directly look at type annotations more frequently during code comprehension or bug localization tasks in TypeScript, suggesting that while types may serve as documentation, they are not actively read in the manner previously hypothesized.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are reading a cookbook. Some recipes have little sticky notes on the ingredients list saying "2 cups of flour" or "chicken, boneless." These notes are like type annotations in computer code. They tell the programmer (the cook) exactly what kind of data (ingredient) is being used.
For years, experts have believed that these "sticky notes" act as a built-in instruction manual. The theory is that when a programmer gets stuck or tries to understand a recipe, they look at these notes to help them out.
The Big Question:
Do programmers actually look at these notes when they are reading code, or do they just ignore them?
To find out, the researchers in this paper set up a "kitchen" experiment. They didn't just ask programmers what they thought they did; they used eye-tracking glasses (like high-tech glasses that record exactly where your eyes are looking) to watch 22 students as they tried to understand code and find bugs in it.
Here is what they discovered, broken down into simple concepts:
1. The "Ghost Note" Phenomenon
The Expectation: If the sticky notes are helpful, people should look at them often, especially when they are confused.
The Reality: The eye-tracking data showed that programmers barely looked at the type annotations.
- The Analogy: It's like having a map with a bright red "You Are Here" sticker on it, but the driver keeps staring at the road and the GPS screen, completely ignoring the sticker. Even when the notes were there, the programmers' eyes didn't linger on them any more than when the notes were missing.
- The Finding: The presence of these notes didn't change how the programmers read the code. They didn't look at the notes more often, and they didn't go back to re-read them when they got stuck.
2. The "Memory" Test
The Expectation: Maybe the notes act as a memory aid? If a programmer has a lot of things to remember (high working memory), maybe they use the notes less because they can hold the info in their head. If they have a hard time remembering, maybe they rely on the notes more.
The Reality: The researchers tested the students' memory (like asking them to remember a list of letters while doing math).
- The Finding: It didn't really matter how good their memory was. Whether they had a "super memory" or a "normal memory," they still didn't look at the type notes significantly more or less. The notes didn't seem to act as a crutch for their memory in the way the researchers hoped.
3. The "It Depends on the Recipe" Factor
The Finding: The biggest thing that changed how the programmers read the code wasn't the notes; it was the code itself.
- The Analogy: Some recipes are just easier to read than others. If a recipe is messy or confusing, the cook spends more time staring at it. If it's clear, they breeze through. The researchers found that the specific piece of code mattered way more than whether it had "sticky notes" on it. Some code was just harder to understand, regardless of the annotations.
4. The "I Love Notes" Paradox
The Twist: After the experiment, the researchers asked the students, "Do you like these sticky notes?"
- The Result: The students said YES! They overwhelmingly said type annotations are helpful for writing code and understanding it. They think they are useful.
- The Paradox: This is the confusing part. The students say the notes are great, but their eyes prove they aren't looking at them. It's like saying, "I love reading the nutrition label," but then never actually looking at it when you buy the cereal.
What Does This Mean?
The paper concludes that while programmers believe type annotations are like helpful in-code documentation, they don't actually read them in the way we might expect. They don't stare at them, and they don't go back to them when they get stuck.
Who should care?
- Tool Builders: If programmers aren't looking at the notes, maybe the tools (like the software editors they use) need to show this information in a different, more obvious way, rather than just printing it on the screen.
- Teachers: If students aren't reading the notes, teachers might need to teach them how to read and use this information, rather than just assuming they will pick it up naturally.
- The Community: It suggests that just adding these notes to code isn't a magic fix for understanding; the notes need to be used in a specific way that we haven't quite figured out yet.
In short: Programmers love the idea of type annotations, but in practice, their eyes tell a different story: they are mostly ignoring them.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.