🤖 AI

SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery

This paper introduces SparkMe, a multi-agent LLM system that formulates adaptive semi-structured interviewing as an optimization problem to balance predefined topic coverage with emergent theme discovery, demonstrating superior performance in both automated benchmarks and human studies compared to existing approaches.

Original authors: David Anugraha, Vishakh Padmakumar, Diyi Yang

Published 2026-02-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: David Anugraha, Vishakh Padmakumar, Diyi Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a complex case. You have a checklist of 10 specific clues you must find (like "Where were you at 8 PM?" or "Did you see a red car?"). But you also know that the real breakthrough might come from something the witness mentions casually, something you didn't even think to ask about (like "Oh, by the way, the suspect was humming a specific tune").

The problem? You only have 30 minutes, and you can't ask the witness 100 questions. If you stick strictly to your checklist, you might miss the big clue. If you just chat freely, you might forget to ask the 10 things you were hired to find.

This is the exact challenge researchers face when trying to understand people's experiences. They need to cover a specific list of topics (like "How does AI affect your job?") but also stay open to surprising new ideas that pop up during the conversation. Doing this with real humans is slow and expensive. Doing it with AI is easy, but current AI interviewers are either too robotic (sticking rigidly to the script) or too chaotic (going off on tangents and forgetting the main point).

Enter SparkMe.

What is SparkMe?

Think of SparkMe as a super-intelligent, multi-person interview team powered by AI. Instead of one AI trying to do everything at once, SparkMe splits the job into three specialized roles, working together like a pit crew in a race:

The Interviewer (The Face): This is the AI that actually talks to you. It asks the questions, listens to your answers, and keeps the conversation flowing naturally.
The Scribe (The Note-Taker): This AI is constantly writing down a "map" of the conversation. It tracks which checklist items you've already covered and summarizes what you've said so far. It knows exactly what's been done and what's left.
The Strategist (The Crystal Ball): This is the magic part. Every few minutes, this AI pauses the real conversation and runs a simulation. It asks itself: "If I ask Question A next, where will the conversation go? If I ask Question B, will we discover something new?"

The "Crystal Ball" Analogy

Imagine you are playing a game of "20 Questions."

Old AI Interviewers just pick the next question from a list. They don't think ahead.
SparkMe's Strategist is like a grandmaster chess player. Before making a move, it simulates the next three moves in its head. It calculates: "If I ask about 'AI ethics' now, the user might mention 'bias in hiring.' That's a new, valuable topic we didn't plan for! Let's go there."

It calculates a "Score" for every possible path:

Did we cover the checklist? (Good)
Did we find a new, interesting insight? (Better)
Did we waste time asking too many questions? (Bad)

It picks the path with the highest score.

Why Does This Matter?

The researchers tested SparkMe against other AI interviewers by interviewing hundreds of "fake" people (simulated users) and then 70 real people from different jobs (teachers, engineers, HR, etc.).

The Results:

Better Coverage: SparkMe was better at making sure they asked all the required checklist questions.
Better Discovery: It found new, surprising insights that the other AIs missed. For example, while interviewing teachers, SparkMe didn't just ask "Do you use AI?" It discovered that teachers were using AI to stress-test their own lesson plans by asking the AI to find holes in their logic. That's a deep, emergent insight that a rigid script would have missed.
Less Fatigue: Because SparkMe is so efficient at planning, it got all the information it needed in fewer questions, so the interview didn't drag on.

The Bottom Line

SparkMe is like giving a researcher a smart, adaptive compass. It doesn't just follow a map; it knows when to stick to the path and when to explore a hidden trail that leads to a treasure. It balances the need for structured data with the human need for spontaneous, deep conversation, making it a powerful tool for understanding how technology is changing our world.

In short: SparkMe is the AI interviewer that knows when to follow the script and when to throw the script away to find the real story.

1. Problem Statement

Qualitative data collection via semi-structured interviews is critical for product design and policy-making but is constrained by the time and availability of human experts. While Large Language Models (LLMs) have been proposed to automate these interviews, existing systems suffer from a fundamental limitation: they lack a principled mechanism to balance systematic coverage of a predefined topic guide with adaptive exploration of emergent themes.

Current LLM interviewers typically rely on prompt engineering to implicitly manage this trade-off. However, without explicit optimization, these models struggle to:

Systematically cover all predefined subtopics.
Identify and pursue "emergent subtopics" (relevant, participant-introduced insights not in the original guide).
Minimize interview cost (length) to avoid over-burdening the interviewee.

2. Methodology

The authors propose SparkMe, a multi-agent LLM system that formulates adaptive interviewing as a utility optimization problem.

A. Formalization of the Task

The authors define the interview objective as a utility function $U$ that balances three competing goals:
$U(Q,R,S) = \alpha C(R|Q,S) - \beta L(Q) + \gamma E(R|Q)$

Where:

$C$ (Coverage): The fraction of predefined subtopics ( $S$ ) explored in sufficient depth.
$E$ (Emergence): The discovery and coverage of emergent subtopics ( $S_{emg}$ )—content relevant to core topics but introduced organically by the participant and not in the original guide.
$L$ (Cost): The burden on the interviewee, measured by the number of turns or questions asked.
$\alpha, \beta, \gamma$ : Weights adjustable by the researcher to prioritize coverage, cost-efficiency, or discovery based on study goals.

The goal is to find the question sequence $Q^*$ that maximizes the expected utility given the stochastic nature of participant responses.

B. System Architecture: SparkMe

SparkMe employs a multi-agent architecture inspired by dual-process theories of cognition (separating fast reaction from slow deliberation). It consists of three agents:

Interviewer Agent (IA): Operates at every turn. It conducts the conversation, selecting actions to probe depth, explore emergence, or transition to new subtopics based on the current state.
Agenda Manager (AM): Operates synchronously with the IA. It maintains a shared "Interview Agenda" that tracks:
- Coverage status of predefined subtopics.
- Accumulated notes and summaries.
- Hierarchical organization of information.
Exploration Planner (EP): Operates asynchronously every $k$ $k$ turns (e.g., every 2 turns). This is the core innovation. The EP performs deliberative planning via simulated conversation rollouts:
- It simulates multiple future conversation trajectories (rollouts) of length $h$ (horizon).
- For each trajectory, it predicts hypothetical user responses using an LLM.
- It calculates the expected utility gain ( $\Delta U$ ) for each rollout, estimating how much coverage and emergence would be achieved versus the cost incurred.
- It selects the trajectory with the highest expected utility and suggests specific question directions or emergent subtopics to the IA.

This mechanism allows SparkMe to proactively identify high-value emergent themes (e.g., a participant mentioning "AI hallucinations as a stress test") and steer the conversation toward them, rather than just reacting to the last turn.

3. Key Contributions

Objective Formulation: A tractable, customizable utility function for semi-structured interviewing that explicitly quantifies the trade-off between topic guide coverage, emergent theme discovery, and interaction cost.
SparkMe System: A multi-agent framework incorporating a deliberative planning step (simulated rollouts) to optimize interview trajectories. It is released as open-source.
Comprehensive Evaluation:
- Automated Benchmarking: Evaluation against 4 baselines (StorySage, LLM-Roleplay, Interview-GPT, MimiTalk) using 200 simulated user agents.
- Human User Study: A study with 70 participants across 7 professions (e.g., researchers, engineers, educators) to validate real-world utility and insight quality.

4. Results

A. Automated Evaluation (Simulated Agents)

Coverage & Utility: SparkMe achieved the highest predefined subtopic coverage (0.977) and overall utility (1.017), outperforming all baselines. It reached peak coverage in 38 turns, whereas other systems required up to 72 turns or failed to converge.
Emergent Discovery: Unlike baselines which had 0 covered emergent subtopics, SparkMe successfully identified and covered emergent themes (e.g., specific AI risk management strategies) due to the EP module.
Efficiency: SparkMe maintained high coherence and flow while asking questions at a readable complexity level (Flesch-Kincaid grade level), proving that optimization for coverage/emergence does not degrade user experience.

B. Human User Study (70 Participants)

Content Quality: Domain experts rated SparkMe significantly higher than the best baseline (MimiTalk) on Coverage (4.37 vs. 3.14), Depth (4.06 vs. 3.09), and Correctness (4.23 vs. 3.46).
Interaction Quality: SparkMe was rated highly for clarity, adaptiveness, and comfort, demonstrating that the multi-agent planning did not introduce unnatural conversational friction.
Emergent Insights: Peer validation confirmed that SparkMe elicited novel, profession-specific insights (e.g., "informal social norms regarding AI ethics" or "impact of AI-specific roles on team dynamics") that were not captured by the predefined guide or the baseline systems.

5. Significance and Impact

Methodological Shift: The paper moves LLM interviewing from "prompt engineering" to "optimization-based design," providing a rigorous framework for balancing competing research goals.
Scalability: SparkMe demonstrates that automated systems can conduct high-quality, adaptive qualitative research at scale, potentially democratizing access to deep user insights for policy and product teams.
Discovery Mechanism: By explicitly modeling and optimizing for "emergence," SparkMe addresses a critical gap in automated qualitative research: the ability to find the "unknown unknowns" that human experts typically discover through intuition and adaptability.
Open Science: The release of the code, datasets, and evaluation protocols allows the community to replicate, benchmark, and customize the system for diverse domains.

In conclusion, SparkMe represents a significant advancement in AI-driven qualitative research, successfully bridging the gap between the structured rigor of surveys and the adaptive depth of human-led interviews.