From Dyad to Triad: Eliciting XAI Requirements in Stroke Rehabilitation

The Invisible Coach and the Silent Patient

Imagine you are learning to ride a bike, but instead of a parent holding the seat, a robot is guiding you. This robot can see your wobbles and knows exactly when you're about to fall. But here's the catch: the robot never speaks. It just tilts the handlebars or stops the pedals. You might get frustrated, wondering, "Did I pedal too hard? Is the battery low? Is the robot broken?" This is the world of Explainable AI (XAI). In simple terms, XAI is the art of making computer programs tell us why they made a decision, rather than just showing us the result.

Now, imagine this robot is helping a stroke survivor relearn how to move their arm. A stroke is like a power outage in the brain's wiring, often leaving people unable to speak clearly or process complex ideas quickly. For years, doctors have been the "coaches" in this scenario, explaining to patients why an exercise failed and how to fix it. But as technology steps in to help, the team changes from a duo (doctor and patient) to a trio (doctor, patient, and robot). The big question is: How do you ask a person who is struggling to find the right words what kind of "explanation" they need from a robot? If you ask them, "Do you prefer a contrastive explanation or a counterfactual one?" they might just stare at you. This paper dives into that exact problem, trying to figure out how to listen to patients who can't always speak for themselves, so the robots they use don't just work, but actually make sense to them.

---

From "What?" to "Why?"

This paper is about a clever experiment designed to solve a communication puzzle. The researchers wanted to know what stroke survivors actually need to understand from the AI systems helping them recover. But there was a hurdle: many of these patients have aphasia, a condition that makes it hard to speak or understand complex language. If you ask them directly, "What kind of transparency do you want from this algorithm?" they might get stuck, or worse, just say "yes" to whatever you suggest because they don't want to be difficult.

The team, led by Param Rajpura and Yogesh Kumar Meena, decided to stop asking abstract questions and start showing stories. They created a video-based scaffolding protocol. Think of "scaffolding" like the temporary wooden supports builders use to help a wall get built. Here, the "wood" was a series of short, 2-minute videos showing a character named Sunita trying to use a brain-computer interface (BCI) to move her arm. The videos showed realistic problems—like the machine working sometimes but not others, or giving confusing feedback—but they didn't show the solution. Instead, the videos stopped right at the moment of confusion, asking the viewers, "What would help Sunita here?"

To get the most out of these videos, the researchers used four special tricks, or "scaffolding approaches," to help the patients express their needs:

1. The "Familiar System" Bridge (Analogical Bridging): Instead of talking about "neural signals" or "impedance," the facilitators compared the AI to things everyone knows. For example, they compared a bad brain signal to a dropped mobile phone call. Suddenly, the idea of "fixing the signal" made perfect sense to a patient who had never heard of EEG technology.
2. The "Third-Person" Shield (Projective Personas): Asking a patient directly, "Are you worried about your privacy?" can feel scary or embarrassing. So, the researchers asked, "What would a skeptical person worry about?" This allowed patients to voice their fears without feeling like they were admitting weakness.
3. The "A or B" Choice (Binary Forcing): Sometimes, open-ended questions are too heavy. The researchers simplified things: "Do you want a tiny summary or a huge report?" or "Should the machine tell you what to do, or should a human explain it?" This reduced the mental load and let patients make clear choices.
4. The "Silence is Golden" Rule (Extended Response Time): They gave patients 15 to 30 seconds of silence after asking a question. For someone with aphasia, that extra time is the difference between a blank stare and a thoughtful answer.

What They Found: A Crowd of Different Voices

The study involved three stroke survivors (two of whom had moderate-to-severe aphasia) and three family caregivers. The results were surprising and showed that patients are not a single, uniform group.

• They want different levels of detail: One patient wanted a tiny, simple report ("Just tell me if I did it"), while another wanted a detailed, play-by-play breakdown of every single movement. If the designers had just asked one person, they might have built a system that annoyed the other.
• They trust different "coaches": When the doctor said one thing and the machine said another, some patients trusted the doctor, while others trusted the machine's numbers because they felt more "objective." Interestingly, no matter who they trusted, they all said they would work harder if they felt supported.
• They want different kinds of help: Some patients wanted the machine to talk directly to them so they could fix their own mistakes. Others preferred that a family member or human explained the error first, because they didn't feel confident talking to a machine yet.

The Hidden Trap: The "Expert" Bias

The paper also shines a light on a tricky problem: the researchers themselves can accidentally mess things up. By watching their own interactions, they found three ways they unintentionally steered the answers:

1. The "More is Better" Bias: When a patient asked for a simple report, the facilitator sometimes summarized it as a "full report" because they assumed complexity was what the AI could do. This accidentally erased the patient's actual preference for simplicity.
2. The "Yes, I knew it" Bias: When a patient gave a surprising answer (like saying they wanted the machine to know their thoughts), the facilitator sometimes quickly moved on to the next expected answer, missing a valuable insight.
3. The "Doctor in the Room" Effect: Because a doctor was present, patients sometimes said they trusted the doctor more than the machine, even if they actually preferred the machine. The presence of authority changed the answer.

The Takeaway

This paper suggests that we cannot just assume we know what patients need. We can't design a robot coach based on what a doctor thinks a patient needs, or what a software engineer imagines is best. The study shows that with the right tools—like videos, simple analogies, and plenty of time—we can hear the real voices of patients who struggle to speak.

However, the authors are careful to say this is a formative study, meaning it's a first step to test the method, not a final solution for the whole world. They only worked with six people in one center in India. They aren't claiming to have solved the problem for every stroke survivor everywhere. Instead, they are offering a protocol (a recipe) and a set of warnings (a checklist of biases to avoid) for anyone trying to design AI for people with communication barriers.

The big lesson? Trustworthy AI isn't just about making the code smart; it's about making sure the people using it can actually understand what the code is saying. And to do that, we have to learn how to listen in a whole new way.