Designing Safe and Accountable GenAI as a Learning Companion with Women Banned from Formal Education
This paper presents a participatory design study with women in Afghanistan that reveals their need for GenAI to function as a safe, accountable mentor rather than just an information source, while demonstrating that co-designing such systems not only addresses critical privacy and pedagogical risks but also significantly empowers users' aspirations and agency.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where a library is locked, a classroom is closed, and the only way to learn is through a window you can't always open. This is the reality for many women in Afghanistan, who have been banned from formal education. They are hungry to learn, to code, and to build careers, but they are doing it alone, in the dark, and often under the watchful eyes of family members who might not approve.
This paper is about a group of researchers who sat down (virtually) with 20 of these women to ask: "If you could have a magical AI friend to help you learn, what would it look like, and how do we make sure it doesn't get you in trouble?"
Here is the story of their journey, explained simply.
1. The Problem: Learning in a "Glass House"
Usually, when you learn something new, you have a teacher, classmates to ask questions, and a safe space to make mistakes. For these women, that community is gone. They are trying to learn programming and job skills on shared family phones, with spotty internet, and under strict rules.
If they use a normal AI chatbot, it's like trying to study in a glass house.
- The Risk: If the AI saves their chat history, or if the phone is shared with a brother or father who sees the screen, the woman could be in serious danger.
- The Mismatch: The AI might suggest, "Go to a coffee shop to study!" or "Work on a project about dating apps!" In their context, these suggestions are not just unhelpful; they are dangerous and culturally impossible.
- The Trap: If the AI just gives the answer to a coding problem, the woman feels smart for a minute, but she isn't actually learning. It's like being given the answer key to a test; you pass the test, but you don't know how to drive the car.
2. The Solution: The "Ghost Mentor"
The researchers asked the women to imagine their perfect AI companion. They didn't want a robot that just spits out facts. They wanted a Ghost Mentor.
Here is what this Ghost Mentor needs to do:
- Be a "Shadow" Friend: The women wanted an AI that feels like a peer sitting next to them, but one that leaves no footprints. It needs to be able to delete its own tracks instantly. Imagine a chalkboard that erases itself the moment you look away. No registration, no saved history, just pure, safe help.
- Be a "Patient Teacher," Not a "Cheat Code": Instead of saying, "Here is the code you need," the AI should say, "Let's break this problem down. What do you think happens if we try this step?" It needs to guide them through the thinking process so they actually learn, rather than just copying the answer.
- Be a "Local Guide": The AI needs to know the rules of the house. It shouldn't suggest going to a public internet café. It should know that the user only has 15 minutes before they have to cook dinner, so it should offer micro-lessons that can be paused and picked up later. It should speak their local language but also gently teach them English, which is needed for global jobs.
- Be a "Career Bridge": Since they can't go to job interviews in person, the AI should act as a simulator. It can pretend to be a client on a website like Upwork, letting the women practice writing proposals and negotiating prices in a safe, private space.
3. The Magic of "Dreaming Together"
The most surprising part of the study wasn't just about designing the AI. It was about what happened to the women during the design process.
The researchers didn't just ask, "What features do you want?" They said, "Let's imagine your future together."
When the women got to design this future, something magical happened. Their hope levels skyrocketed.
- Before the session: They felt stuck, like there were no doors open.
- After the session: They felt like they could see a path forward. They felt more powerful (agency) and saw more ways to reach their goals (avenues).
It's like being in a dark room and someone hands you a flashlight. You don't just see the wall; you suddenly see the whole room, and you realize there are doors you didn't know existed.
4. The Big Lesson: Safety First, Answers Second
The main takeaway from this paper is that for people living in dangerous or restrictive situations, safety is more important than speed.
A "good" AI for a student in a free country might be one that gives the fastest answer. But a "good" AI for a woman in Afghanistan must be one that:
- Protects her identity (so she doesn't get in trouble).
- Respects her reality (so it doesn't suggest impossible things).
- Teaches her to think (so she doesn't become dependent).
The Analogy of the "Safe House"
Think of this GenAI not as a search engine, but as a Safe House.
- It's a place where you can practice your skills without fear of being watched.
- It's a place where the furniture (the advice) fits your specific room size (your constraints).
- It's a place where you can rehearse for the outside world (jobs) without actually stepping outside yet.
The researchers concluded that if we build AI with this kind of "accountability"—where we care about the user's safety and real-life context as much as the code itself—we can help women not just learn, but truly imagine and build a future for themselves.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.