Technical Summary: Adaptive Interviewing for Persona Simulation in LLMs

Problem Statement

Accurately simulating the decisions of a specific individual remains a significant challenge for Large Language Models (LLMs). While LLMs can reproduce population-level opinion distributions when conditioned on coarse demographic prompts, they often fail to capture the individual-level nuances required for specific decision simulation. Current approaches typically rely on static persona descriptions (e.g., demographic profiles or short bio summaries), which miss the dynamic values, experiences, and contextual cues necessary for realistic individual modeling. Furthermore, existing research suggests that such static conditioning often fails to reliably reproduce the opinions of specific groups, even when models are explicitly steered toward those identities.

Methodology

The authors propose an adaptive interviewing framework designed to elicit persona-relevant information through a structured, multi-turn dialogue rather than relying on pre-defined static attributes. The methodology consists of three distinct stages:

1. Core Questions (Stage 1): The model generates ten open-ended, concrete questions spanning ten empirically grounded psychological domains (e.g., behavioral priorities, decision logic, fears, relational boundaries, and life narrative). These questions are designed to invite specific examples rather than abstract self-assessments.
2. Adaptive Follow-ups (Stage 2): Based on the participant's responses to the core questions, the model dynamically generates 5–6 follow-up questions. These are targeted at clarifying ambiguities, exploring emotionally salient segments, or deepening understanding of latent values and decision-relevant cues.
3. Synthesized Summary (Stage 3): A concise personality summary is distilled from the full conversation.

Evaluation Setup:
The framework is evaluated using a decision simulation task based on moral dilemma scenarios (drawing from CNI-style trolley problems and altruism vs. loyalty paradigms). The study involves 20 participants who undergo the three-stage interview and subsequently complete standardized assessments (Big Five, MBTI) and 25 moral dilemma questions (ground truth).

The LLM's ability to simulate participant decisions is tested under three conversational contexts:

• Core-10: The model receives only the responses to the initial ten core questions.
• Full Interview: The model receives the complete transcript, including both core responses and adaptive follow-up exchanges.
• Personality Summary: The model receives the distilled summary of the full dialogue.

The evaluation measures prediction accuracy (exact match and relaxed metrics for ordinal tasks) and analyzes reasoning traces to determine if model decisions are grounded in evidence derived from the conversational context.

Key Contributions

1. Adaptive Interviewing Framework: The introduction of a dynamic, three-stage dialogue system that generates follow-up questions based on user responses to gather richer, decision-relevant persona information.
2. Evaluation Protocol: A novel setup for individual decision simulation using moral dilemma scenarios, comparing model performance across varying levels of conversational context (Core-10, Full Interview, and Summary).
3. Reasoning-Trace Analysis: A granular analysis of 340 categorical decision items demonstrating that adaptive follow-up functions as a selective grounding mechanism. The study shows that follow-up-derived evidence is incorporated in approximately 40% of full-interview traces, and predictions grounded in this evidence are more accurate than those grounded only in core responses (45.5% vs. 39.3%).

Results

• Aggregate Accuracy: Contrary to the intuition that "more context equals better simulation," aggregate accuracy does not monotonically increase with conversational richness. The Personality Summary condition achieved the highest aggregate accuracy (0.393), followed by Core-10 (0.379) and Full Interview (0.365). The authors attribute this to the summary providing a compact representation that reduces noise for certain tasks.
• Task-Dependent Performance:
  • Categorical Tasks: Discrete judgments (e.g., multiple-choice dilemmas, MBTI inference) perform best with condensed summaries.
  • Ordinal Tasks: Likert-style judgments and ranking tasks benefit from richer conversational context. While exact-match accuracy was lower for the Full Interview condition on Likert scales, allowing for a one-point tolerance revealed significantly better ordinal calibration (74.3% vs. 57.9% for Core-10).
• Selective Grounding: The primary finding is that the benefit of adaptive interviewing is selective. In the Full Interview condition, 40% of predictions incorporated follow-up-derived evidence. When the model explicitly grounded its reasoning in this follow-up evidence, accuracy improved (45.5%) compared to predictions grounded only in core interview content (39.3%).
• Error Analysis: Qualitative analysis identified systematic interpretive biases in LLMs, including a "Normative Rule Bias" (defaulting to procedural fairness), "Prosocial Harmony Bias" (favoring cooperative outcomes), and "Over-responsibility Bias" (seeking compromise solutions), which often led to errors when users deviated from these normative patterns.

Significance and Claims

The paper claims that effective persona simulation requires not only the elicitation of richer evidence but also the reliable grounding of model reasoning in that user-specific evidence.

• Mechanism over Booster: Adaptive interviewing acts less as a uniform accuracy booster and more as an information-availability mechanism. Its value is realized only when the model meaningfully incorporates the additional cues elicited through follow-up questions into its reasoning process.
• Task-Adaptive Representations: The optimal persona representation depends on the downstream task. Systems should avoid uniform compression; instead, they should utilize condensed summaries for discrete categorical tasks and full conversational histories for ordinal calibration and preference ranking.
• Evaluation Shift: The authors argue that evaluating persona simulation should move beyond strict exact-match accuracy to include reasoning grounding and task-specific calibration metrics.

The study concludes that while richer elicitation protocols are necessary, they are insufficient on their own; improvements depend on whether downstream inference mechanisms can reliably anchor predictions in the specific evidence provided by the user.