Towards Adaptive and Accessible Subtitled Media: A Systematic Review, Taxonomy and Design Considerations

Imagine sitting in a darkened room, watching a film on a screen. For most viewers, the story unfolds through sound and sight, but for millions of people with hearing or vision difficulties, the experience is incomplete without text appearing on the screen. These words, known as subtitles or closed captions, translate spoken dialogue and describe sounds into written language, acting as a vital bridge to the audio world. For decades, the way these words appear has remained largely the same: white text on a black background, fixed at the bottom of the screen, appearing in three lines at a time. This "one-size-fits-all" approach works for many, but for people with specific disabilities, it can create barriers. A person with a cognitive processing difference might find the text moves too fast, while someone with a visual impairment might struggle to read small letters against a busy background. The question researchers have begun to ask is whether we can change how these words look, move, and appear to better suit the unique needs of every viewer.

A team of researchers at the University of Dundee set out to answer this question by looking at the entire history of scientific work on the subject. They gathered and analyzed 147 academic papers published over a 29-year period, from 1995 to 2024. Their goal was not just to count how many studies existed, but to understand what kinds of changes scientists had tried, who they had tested them on, and what those tests revealed. By organizing this vast collection of research, they created a new map of the field, revealing where progress has been made and, more importantly, where significant gaps remain.

The researchers found that the vast majority of studies focused on two specific groups: people who are deaf or hard of hearing, and people who are blind or have low vision. These communities have been the primary focus because changing the visual style of text directly helps those with hearing loss, and adjusting audio cues or screen descriptions helps those with vision loss. However, the review uncovered a stark reality: other groups are largely being left behind. The study found almost no research dedicated to people with intellectual or developmental disabilities, neurodivergent individuals, or those with mental health conditions. Even more surprisingly, the review found zero papers (0.0%) specifically addressing people with aphasia, a condition that affects the ability to understand or produce language, despite millions of people worldwide living with it. The authors suggest that this oversight is a missed opportunity, as these groups often face unique challenges with standard subtitles that current designs do not address.

Another major finding concerns where and how people watch these adapted videos. The research showed that most studies were conducted on desktop computers and laptops, with very few experiments taking place on televisions, smartphones, or large cinema screens. This is a significant limitation because the way we consume media has shifted dramatically. People now watch content on a wide variety of devices, from small phones to massive theater screens, yet the science behind subtitle design has not kept pace. The review suggests that what works on a small computer monitor might not work on a large cinema screen or a virtual reality headset. Furthermore, the contexts in which people watch were mostly limited to entertainment and education, with very little attention paid to how subtitles function in social or public settings, such as a busy café or a museum.

When it came to the actual methods used to test these ideas, the researchers noticed a heavy reliance on questionnaires and surveys. While these tools are useful for gathering opinions, they often happen in a controlled lab setting or a participant's home, rather than in the real world. The study found that very few researchers used methods like observing people in their daily lives or running long-term studies to see how adaptations hold up over time. This means that while we have many ideas about how to improve subtitles, we have less evidence about how they actually work in the messy, unpredictable reality of everyday life. The authors argue that future research needs to move beyond the lab and involve a much wider range of people, including older adults and caregivers, to truly understand the needs of the diverse population that relies on these tools.

The team also developed a new way to categorize the different types of changes made to subtitles. They looked at four main areas: where the text appears on the screen, where the text is located relative to the video (such as on a separate device), what the text actually says, and how the text looks in terms of font, color, and size. Their analysis showed that while researchers have been experimenting with all four areas, the focus has been uneven. Changing the content of the text, such as simplifying sentences or adding punctuation, has been the most common area of study. In contrast, moving subtitles to different locations, like a companion device or a virtual reality space, has been explored far less frequently. This imbalance suggests that while we are good at tweaking the words themselves, we are slower to rethink the physical space where those words appear.

Ultimately, this comprehensive review serves as a call to action for the scientific community and media creators. It highlights that the current approach to subtitles is too rigid and too focused on a narrow set of users. The authors propose that to create truly accessible media, we must expand our thinking. This means designing subtitles that can adapt to different screen sizes, from small phones to large cinema displays, and creating systems that can be customized for a much broader spectrum of human needs. By involving a more diverse group of people in the design process and testing these solutions in real-world environments, we can move toward a future where everyone, regardless of their abilities, can fully enjoy the stories and information presented on their screens. The path forward requires not just technical innovation, but a fundamental shift in who we consider when we design the way we see and hear the world.