Bridging communication between hearing and visually impaired individuals via speech recognition and synthesis
This paper presents a cost-effective, software-only Speech Recognition and Synthesis Tool (SRST) developed in MATLAB that facilitates bidirectional communication between hearing and visually impaired individuals by converting speech to high-contrast text and text to synthesized speech, achieving approximately 90% operational efficiency without requiring specialized hardware.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of communication as a giant, bustling party where everyone is trying to talk to each other. For most people, this is easy: you speak, your friend hears, they speak back, and you see their lips move. But for some guests, the party is a bit broken. If you can't hear the music (hearing impaired), the conversation feels like watching a silent movie where you miss the jokes. If you can't see the screen (visually impaired), you might hear the music but have no idea what the movie is about because you can't read the subtitles.
This paper lives in the corner of science called "Assistive Technology," which is basically the field of building tools to help people with disabilities join the party. To understand what these researchers did, you need to know about two main tricks computers use to talk to humans. The first is Speech Recognition, which is like a super-fast translator that listens to your voice and turns it into written words on a screen. The second is Speech Synthesis, which is the opposite: it takes written words and uses a computer voice to read them out loud. Usually, these tools are designed for people who can hear but can't see, or vice versa. But what happens when a person who can't hear tries to talk to a person who can't see? They are stuck in a loop where one can't hear the answer, and the other can't read the question. This paper tries to fix that specific broken loop.
The researchers, a team from Aksum University in Ethiopia, built a digital bridge called the Speech Recognition and Synthesis Tool (SRST). Think of it as a two-way walkie-talkie that doesn't just transmit sound; it instantly translates sound into text and text back into sound. Their goal was to create a system where a hearing-impaired person could speak into a microphone, have their words instantly appear as big, clear text on a screen for a blind person to "hear" via a computer voice, and then have that blind person speak back, with their voice turning into text for the hearing-impaired person to read.
The cool part about this project is that they didn't build any fancy new robots or expensive hardware. Instead, they wrote a clever software program that runs on a standard computer using a regular microphone and speaker. It's like turning a normal laptop into a magic communication device. The system works in two main directions. On one side, it listens to a blind person's voice. It chops the sound into tiny pieces, analyzes the unique patterns of the speech (like a fingerprint for sound), and matches those patterns to a list of words it has been taught. Once it figures out what was said, it prints the words on the screen in high-contrast text so the hearing-impaired person can read them. On the other side, the hearing-impaired person types a message. The computer then takes those letters and stitches together tiny pre-recorded chunks of human speech (called "diphones") to create a new voice that reads the message aloud to the blind person.
The team tested their creation by having people speak into the system in a noisy university lab. They found that the system works pretty well, getting the message right about 90% of the time. It was slightly better at recognizing isolated words (around 91.2%) than it was at understanding sentences in a quiet room (89.5%), and it dropped a bit more when there was background noise (84.1%). The authors suggest that the few mistakes happened because some words sound very similar to the computer's ear, like twins in a crowd.
They also discovered that the system is flexible. Because they didn't hard-code complex grammar rules, the system could handle a mix of languages, successfully recognizing words from English, Tigrigna, and Amharic in the same conversation. However, they admit the system isn't perfect yet. It sometimes gets confused by sudden loud noises (like a chair scraping) that aren't speech, and it can be a little slow if the computer has to check too many words at once.
The researchers conclude that while this is a working prototype and not a finished commercial product, it proves that a low-cost, software-only solution can successfully bridge the gap between these two communities. They suggest that in the future, adding a translation layer could let people speak different languages and still understand each other, and using better microphones could help the system ignore background noise. For now, though, they have built a solid foundation—a digital handshake—that shows how technology can help two people who experience the world differently finally have a conversation.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.