Studying People to Study AI: Expert Perspectives on the Epistemic Fit and Barriers of Human Research in AI Safety & Ethics

Imagine you are trying to build a robot that can talk, think, and help people. You want to make sure this robot is safe and won't accidentally hurt anyone or say something mean. To do this, scientists have been running tests. But here is the tricky part: most of these tests are like playing a video game where the robot talks to another robot. They simulate how a human might react, or they use math to check if the robot is following the rules. It's fast, cheap, and easy to run a million of these "robot-vs-robot" games in a day.

However, there is a big problem with only playing video games. Just because a robot passes a test against another robot doesn't mean it will be safe when it meets a real, messy, emotional human being. Real humans are complicated. They get confused, they feel emotions, and they might trick the robot in ways a simulation never could. This paper asks a simple but huge question: Why are so many scientists still playing video games with robots instead of actually talking to real people to see what happens?

---

The Great Robot Test: Why We Need Real People, Not Just Simulations

This paper is like a detective story investigating a strange gap in the world of AI safety. The researchers, a team of experts from universities like the University of Toronto and McGill, wanted to understand why the field of AI Safety and Ethics (let's call it AISE for short) is so hesitant to use real human beings in their experiments.

Think of AISE as a giant workshop where people are trying to build the safest, most ethical robots possible. Inside this workshop, there are different groups of experts. Some are Technical wizards who love code, math, and running simulations. Others are Sociotechnical detectives who care about how people actually feel and interact with technology. There are also Governance folks who think about rules and laws, and Normative thinkers who worry about what is "right" or "good."

The paper's main discovery is that while everyone in the workshop agrees that talking to real humans is super important, the Technical group is mostly ignoring it. They prefer their fast, clean, computer simulations. The authors found that this isn't just because the Technical group prioritizes efficiency; it's because they have a different way of thinking about what counts as "proof."

The "Video Game" vs. The "Real World"

The authors describe a major divide. The Technical researchers often treat humans like variables in a math equation. They want to measure things with a ruler and a stopwatch. If they can't measure it perfectly, they often don't trust it. They worry that talking to real people is "messy."

Imagine you are trying to test a new bridge.

• The Simulation Approach: You build a perfect digital model of the bridge on a computer. You run a million wind storms through it. The computer says, "It's 100% safe!" This is fast and cheap.
• The Human Research Approach: You actually build the bridge and ask real people to walk across it. Maybe a few people get scared, maybe the wind feels different than the computer said, and maybe the bridge sways in a way the model didn't predict.

The paper suggests that AISE is currently obsessed with the computer model. They are running billions of simulations (like the "video games" mentioned earlier) to see if AI is safe. But the authors argue that this is risky. If you only test your bridge in a computer, you might miss the fact that real people panic when the wind blows, or that the bridge makes a weird noise that scares them.

The "Human-Washing" Trap

One of the most interesting things the paper warns about is something the authors call "human-washing." This is like "green-washing," where a company puts a leaf on a bottle to make it look eco-friendly without actually being green.

"Human-washing" happens when researchers say, "Hey, we talked to some people!" just to check a box. But they didn't really listen to them, or they didn't ask the right questions. They just added a few humans to their study to make it look like they were doing real research, but they didn't actually change their methods. The authors say this is dangerous because it gives a false sense of safety. It's like putting a fake fire extinguisher on the wall; it looks like you're prepared, but if a fire starts, it won't work.

The Barriers: Why Don't They Just Do It?

You might wonder, "If everyone agrees human research is good, why don't they just do more of it?" The paper found three big walls stopping them:

1. The Money and Time Wall: Talking to real humans is expensive and slow. The authors note that a computer simulation might cost less than $1,000 to run. But a serious study with real people? That can cost $300,000 or more. Plus, it takes months to find the right people, get permission from ethics boards, and run the tests. It's like the difference between drawing a picture of a cake and actually baking a giant one for a party.
2. The Mentor Wall: Many young researchers want to study real people, but their bosses (mentors) tell them not to. The paper shares stories of students whose advisors said, "Don't do that, it's not 'real' science," or "Just stick to the code." This makes young scientists afraid to try human research because they think it will hurt their careers.
3. The "Messy Data" Wall: Technical researchers often find human data frustrating. Humans don't always answer questions the same way. They lie, they get tired, or they say something weird. To a computer scientist who loves perfect numbers, this feels like a bug in the system. But the authors argue that this "messiness" is actually the most important part of the data because it's real life.

The Gap Between "Wanting" and "Doing"

The researchers surveyed 93 experts and interviewed 17 of them in depth. They found a huge gap. Almost everyone said they wanted to do more human research. They agreed it was valuable. But when they looked at what people were actually doing, the numbers were much lower.

It's like a gym where everyone says, "I really want to get fit and lift weights," but when you look at the equipment, no one is actually lifting anything. They are just reading books about weights. The Technical group was the least likely to lift the weights; they preferred the books (simulations).

What Should We Do?

The paper doesn't just point out the problem; it offers a few ideas to fix it:

• Stop the "Human-Washing": We need to make sure that when we do include humans, we do it properly. We need to respect their time and listen to what they say, not just use them as a prop.
• Change the Money: Funding groups (the people who give out grants) need to realize that human research costs more. They should give more money to these projects so they aren't crushed by the high price tag.
• Mix the Teams: We need more teams where the code-wizards and the people-experts work together from the start. Right now, they often work in separate rooms. The authors suggest that if a Technical researcher wants to build a safe AI, they need to talk to a Sociotechnical researcher who understands how humans actually behave.

The Bottom Line

This paper suggests that the field of AI Safety is currently too focused on perfect computer models and not enough on the messy, real world of human beings. While simulations are useful, they can't replace the truth that only comes from talking to real people. If we want to build AI that is truly safe, we need to stop treating humans like variables in a math problem and start treating them like the experts on their own lives.

The authors aren't saying simulations are bad; they are saying that relying only on simulations is like trying to learn how to swim by reading a book about water. You need to get wet. And right now, the AI safety community is mostly staying dry, reading the books, and hoping the water is safe.