AI-assisted Script Management for Requirements Elicitation Interviews
This paper presents and evaluates an AI-assisted script management workflow for requirements elicitation interviews, demonstrating through a quasi-experimental study that it outperforms training-only approaches by generating higher-quality scripts, enabling deeper follow-up questioning, and producing more refined goal models despite covering fewer topics.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a detective trying to solve a mystery, but instead of a crime scene, you are interviewing a witness to figure out what they need from a new video game or a shopping app. In the world of software engineering, this detective work is called "requirements elicitation." The goal is to ask the right questions to uncover exactly what the user wants before the programmers start building. If you miss a clue or ask the wrong question, the final product might be broken, confusing, or just plain useless.
To do this well, interviewers usually prepare a "script"—a list of questions designed to cover all the important bases. But here's the tricky part: real conversations are messy. The witness might go off on a tangent, or the detective might get so nervous they forget to ask the most important question. It's a balancing act between sticking to the plan and listening to what the person is actually saying. For a long time, experts have tried to train humans to get better at this, but what if we could give the detective a smart, invisible assistant that whispers hints in their ear while they talk? That is the big question this paper explores: Can artificial intelligence (AI) help humans ask better questions and get better answers during these interviews?
The researchers at Carnegie Mellon University decided to test this by building a digital "co-pilot" for interviewers. They created a two-step system. First, the AI acts as a planner, reading a company's big-picture business goals (like "make people watch more videos") and turning them into a smart list of interview questions. Second, during the actual interview, the AI acts as a live assistant. It listens to the conversation, tracks which questions from the list have been asked, and pops up on the screen with suggestions for follow-up questions if the interviewer gets stuck or needs to dig deeper.
To see if this worked, they set up a showdown between two groups of interviewers. One group was a bunch of smart graduate students who had taken a class on how to interview people (the "Training-Only" group). The other group was a similar bunch of students who had no special training but were given the AI co-pilot to help them (the "AI-Assisted" group). It was like comparing a veteran detective who knows all the rules against a rookie detective who has a super-smart GPS and a reference guide.
The results were surprising and showed that the AI didn't just help; it changed the whole style of the interview. The AI-generated scripts were much higher quality, scoring 92.8 out of 100 compared to 74.8 for the human-written ones. But the real magic happened during the interviews. The AI-assisted interviewers didn't try to rush through a long list of topics. Instead, they focused on fewer topics but went much deeper. They asked about 3.4 follow-up questions for every topic they discussed, whereas the trained humans only asked about 1.15.
Think of it like digging for treasure. The trained humans were like people with a wide net, casting it over a huge area and catching a few fish from many different spots. The AI-assisted interviewers were like people with a single, deep hole; they stayed in one spot and dug down, uncovering much more detailed and specific "treasure" (which the researchers call "low-level requirements"). Because they dug deeper, the final map of what the users wanted was more detailed and useful.
However, the paper is careful not to say the AI is a magic wand that fixes everything. The researchers found that the AI's "follow-up question" suggestions weren't always perfect; the interviewers often ignored the specific words the AI suggested because they felt a bit stale or slow. Instead, the AI's real superpower was the "topic tracker." It was like a friendly dashboard that told the interviewer, "Hey, you haven't asked about the login screen yet," or "You've been talking about payment for a while, maybe dig deeper here?" The interviewers loved this feature, rating it as the most useful part of the system.
So, what's the bottom line? The study suggests that giving an untrained person a smart AI assistant can actually produce better results than giving a trained person no help at all. The AI helped the interviewers stay focused, ask more follow-up questions, and uncover deeper details about what users actually need. But it also showed that AI isn't a replacement for human judgment; the best results came when the AI handled the heavy lifting of tracking and planning, while the human interviewer used that support to have a more natural, deep conversation. It's not about the AI doing the work for you; it's about the AI holding the flashlight so you can see the clues more clearly.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.