Q&A or Document-Based? The Effects of Interface Type on How Screen Reader Users Access Interconnected Documents
This study reveals that while blind and low-vision users often prefer and overestimate the effectiveness of Question-Answer Interfaces for exploring interconnected documents, Document-Based Interfaces actually enable them to access more distinct sources, construct more accurate mental models, and apply knowledge more effectively.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
For people who are blind or have low vision, the internet is a world built for the eyes, not the ears. To navigate it, they rely on screen readers, software that converts text on a screen into spoken words or braille. For decades, this has meant a methodical process of listening to documents one by one, following links, and piecing together information from a structured layout. But a new kind of tool has arrived: conversational artificial intelligence. These systems allow users to ask questions and receive direct answers, promising to skip the tedious work of reading entire files. The hope is that this technology will make information access faster and easier. However, a critical question remains: does getting a quick answer mean you truly understand the whole picture? Researchers are beginning to wonder if the convenience of conversation might actually hide gaps in knowledge, leaving users with a sense of knowing without the substance to back it up.
To investigate this, a team of researchers at the University of Victoria and Singapore Management University conducted a study with sixteen blind and low-vision participants. They wanted to see how these users built their understanding of complex, unfamiliar topics when using two very different tools. The first tool was a traditional document interface, where users navigated a collection of twenty-five interconnected text files and diagrams, much like browsing a website or reading a set of linked encyclopedia entries. The second tool was a conversational interface powered by a large language model, where users could ask open-ended questions and receive synthesized answers drawn from the same collection of documents. To ensure the results were fair, the researchers created two entirely fictional worlds, named Solana and Dominion, complete with their own histories, governments, and cultures. This ensured that no participant had any prior knowledge to rely on; they had to learn everything from scratch.
The participants spent time exploring these fictional worlds using both interfaces. In one session, they used the traditional document system, and in another, they used the conversational chatbot. After each session, the researchers asked the participants to map out what they had learned, drawing connections between different ideas, and then to solve a problem that required them to apply what they had discovered. The results revealed a surprising disconnect between what the users felt they had achieved and what they had actually accomplished. When using the traditional document interface, participants visited a wider variety of documents and constructed mental models that were larger, more detailed, and significantly more accurate. They made fewer mistakes in their understanding and were better able to apply their knowledge to new situations. In contrast, when using the conversational interface, participants visited fewer distinct documents. While the chatbot helped them make quick connections between a smaller set of topics, their overall understanding was shallower, and they made more errors in their mental maps of the worlds.
Perhaps the most striking finding was that the participants did not realize this difference. Despite performing better with the traditional document system, many of them felt that the conversational interface had helped them learn more, explore more deeply, and feel more confident in their answers. They perceived the chatbot as a more effective tool for building knowledge, even though the evidence showed the opposite. The researchers suggest that the conversational style creates a sense of ease and coverage that can be misleading. Because the chatbot synthesizes information into a smooth, direct answer, it feels like a complete picture, but it may actually be skipping over the broader context that a user would encounter by navigating the documents themselves. The traditional interface, while sometimes more demanding and requiring more effort to track connections, forces the user to engage with the structure of the information, leading to a more robust and accurate understanding.
This study highlights a potential risk in the rapid adoption of conversational artificial intelligence for information access. While these tools offer undeniable convenience and can be excellent for finding specific facts, they may inadvertently encourage a superficial engagement with complex information spaces. For blind and low-vision users, who often rely on technology to level the playing field, there is a danger that a tool which feels easier and faster might actually leave them with a weaker grasp of the material. The research suggests that the design of these systems matters deeply; if the goal is deep understanding and the ability to apply knowledge, the structured, sometimes more laborious path of traditional document navigation may still be the superior choice. The findings serve as a reminder that efficiency should not be confused with comprehension, and that the way we access information shapes what we actually know.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.