Imagine you are trying to write a song about your day, but you've never learned music theory, you don't play an instrument, and the words are stuck in your throat. Now, imagine doing this while living in a world that is designed for people who hear, making it even harder to express your feelings. This is the daily reality for many Deaf and Hard-of-Hearing (DHH) individuals.
This paper introduces a new digital tool called SoulNote that acts like a "musical diary" to help these individuals turn their daily struggles and joys into songs, using Artificial Intelligence (AI) as a co-pilot.
Here is the story of the research, broken down into simple concepts and analogies.
🎵 The Problem: The "Silent" Wall
For many DHH people, the world feels like a room where everyone is speaking a language they can't quite hear. This can lead to feeling isolated or like they can't fully share their inner world.
- The Old Way: Previous tools tried to help them make music, but they were like one-time workshops. You might make one song, feel a little better, and then stop. It was like writing a single letter and never writing again.
- The Missing Piece: Real emotional healing often comes from repetition. Just like a therapist helps you unpack your feelings over many sessions, or like keeping a daily journal, the researchers wanted to see what happens if you make music every day over a long period.
🤖 The Solution: SoulNote (The Musical Co-Pilot)
The team built SoulNote, a web-based system that acts like a friendly, patient guide. Think of it as a musical translator that turns your thoughts into a song, even if you don't know a single note of music.
It works in four simple steps, like a recipe:
- The Chat (The Conversation): You talk to an AI chatbot. Instead of asking technical questions like "What key should I use?", the AI asks gentle, story-based questions like, "What image comes to mind when you think about your day?" or "What scene are you seeing?"
- The Lyrics (The Words): Based on your story, the AI writes a draft of lyrics. You can edit them, change words, or ask the AI to make them more hopeful. It's like having a ghostwriter who listens to your voice.
- The Music (The Sound): The AI then turns those lyrics into a full song. It picks the instruments, the mood (sad, happy, calm), and the speed.
- The Gallery (The Archive): All your songs are saved in a personal library. You can listen to them later, watch them with colorful visualizations (since sound is visualized as colors and shapes for DHH users), and reflect on how you felt.
🌱 The Experiment: 12 Weeks of "Musical Journaling"
The researchers asked 12 DHH participants to use SoulNote for three weeks, creating about 12 songs total. They didn't just look at the songs; they looked at how the people changed.
Here is what happened, using some metaphors:
1. The "Third-Person" Mirror (Self-Insight)
When people write about their pain, they often get stuck in the pain. But when the AI turns their story into a structured song with lyrics, it's like stepping out of the movie and watching it on a screen.
- The Result: Participants could look at their own problems from a distance. One person said, "Seeing my feelings written down in a song made it easier to accept that I'm okay." It helped them stop fighting their feelings and start understanding them.
2. The "Emotional Gym" (Emotion Regulation)
The AI suggested different musical styles based on what the person was feeling.
- The Analogy: Imagine you are sad. Usually, you might just listen to sad music. But the AI might say, "Let's try a bright, upbeat drum beat to see if it lifts your energy," or "Let's try a slow piano to help you calm down."
- The Result: Participants discovered they could choose how to feel. They learned that music isn't just something you listen to; it's a tool you can use to fix your mood, like a mental vitamin.
3. The "Bridge to Others" (Attitude Change)
Before the study, many participants were afraid to talk about their feelings with family or friends.
- The Shift: After practicing expressing themselves through songs for weeks, they became braver. One participant said, "I used to stay silent when I was upset at work. Now, I can explain my feelings to my colleagues."
- The Metaphor: The songs acted as a training ground. By practicing their "emotional voice" with the AI, they gained the confidence to speak up in the real world.
🧪 The "Secret Sauce" Test
To prove that the combination of Chat + Lyrics + Music was special, the researchers ran a small test comparing four groups:
- Just writing a diary (Text only).
- Just chatting with the AI (No music).
- Just listening to AI music (No chat).
- SoulNote (The full mix).
The Winner: The full SoulNote experience was the clear champion. It was the only one that made people feel truly understood and helped them understand themselves the best. It showed that talking, writing, and listening to music together creates a powerful loop for healing.
⚠️ The Safety Net
The researchers were very careful. They worked with music therapists to make sure the AI didn't accidentally trigger bad memories or make people feel worse. The AI was programmed to be supportive, not to dig up trauma, but to help people process what they were already feeling.
🏁 The Big Takeaway
This paper shows that Generative AI can do more than just make cool tech demos. For marginalized communities like the Deaf and Hard-of-Hearing, it can be a daily companion for self-discovery.
By turning a song into a daily habit, SoulNote helped people move from "I can't express myself" to "Here is my story, and I am learning to live with it." It transforms the act of songwriting from a one-time performance into a lifelong practice of self-care.