🤖 AI

Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?

This study demonstrates that large language models can effectively construct detailed individual-level "digital twins" from existing socio-economic panel data, achieving up to 78.8% accuracy and strong correlation with real respondents by optimizing information depth, embedding methods, and reasoning modes, thereby showing that twin-based market research is now limited by construction decisions rather than data availability.

Original authors: Leonard Kinzinger, Jochen Hartmann

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Leonard Kinzinger, Jochen Hartmann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict how a specific person would answer a survey question. Usually, you have to ask them directly, which is slow and expensive. This paper asks a bold question: Can we build a "digital clone" of a real person using only the data companies already have sitting in their databases?

Think of a company's database like a giant, messy attic filled with years of notes, receipts, and old surveys about their customers. The researchers wanted to see if they could take a random person's scattered notes from this attic and use an AI to create a "digital twin" that answers new questions just like the real person would.

Here is how they did it and what they found, explained simply:

1. The Ingredients: Building the Clone

To build these digital twins, the researchers used data from the German Socio-Economic Panel (SOEP). Imagine this as a massive, 40-year-long diary kept by over 28,000 real people. It covers everything from their jobs and income to their family life and opinions.

They picked 500 people from this diary and tried to build a digital version of each one. To do this, they tested a "recipe" with four main ingredients:

The Brain (The AI): They used three different open-source AI models (think of them as three different types of super-smart calculators).
The Memory (How much info to feed it): They tested feeding the AI just basic facts (like age and gender) versus feeding it the entire diary. They measured this "information depth" like filling a bucket with water.
The Format (How to tell the story): They tried two ways to present the data:
- The Summary: A short, neat paragraph summarizing the person's life.
- The Raw Chat: A long, messy transcript of every question the person was ever asked and how they answered.
The Thinking Style: They tested if the AI should just spit out an answer immediately, or if it should "think out loud" (write down its reasoning) before answering.

2. The Experiment: The "Taste Test"

The researchers created 2.1 million digital responses. They then asked these digital clones 183 questions that the real people had already answered in the past. The goal was to see if the digital clone's answer matched the real person's answer.

They measured success in two ways:

Accuracy: Did the clone give the exact same answer? (e.g., If the real person said "Yes," did the clone say "Yes"?)
Ranking: Did the clone understand who is more likely to say what? (e.g., If Person A is usually more optimistic than Person B, does the clone reflect that difference?)

3. The Big Discoveries

The "Goldilocks" Zone of Information
You might think "more data is always better." The researchers found this is only partly true.

The First Sip: Adding a little more info (beyond just basic demographics) helped a lot.
The Sweet Spot: The biggest jump in accuracy happened when they added the top 75% of the most interesting questions.
Diminishing Returns: Adding the final 25% of the data (the last bit of the bucket) didn't help much. It cost a lot of computer power but barely improved the results.
Analogy: It's like seasoning a soup. Adding salt helps a lot at first. Adding a little more helps. But dumping in the whole shaker at the end just makes it salty without making it taste much better.

The Power of Raw History
Surprisingly, feeding the AI the raw chat history (the messy transcript of past questions) worked better than giving it a neat summary.

Analogy: It's like trying to guess what your friend will order for dinner. A summary says, "My friend likes Italian." But a chat history says, "Last Tuesday they ordered pizza, but when they were tired they got salad, and they hate cilantro." The messy history gave the AI more clues to get it right.

Thinking Helps, But Not for Accuracy
When the AI was told to "think out loud" before answering, it got better at ranking people (understanding the differences between individuals), but it didn't necessarily get the exact answers right more often.

Analogy: It's like a student who writes out their math steps. They might get the final number wrong, but they understand the logic better than a student who just guesses the answer.

4. The Bottom Line

The study proves that you don't need to run special, expensive interviews to build good digital twins. You can build them using the heterogeneous, messy data companies already have (like loyalty program history or old surveys).

Best Result: The most accurate digital twin got the right answer 78.8% of the time.
The Takeaway: The biggest bottleneck for using these digital twins isn't how you design the data collection; it's just having enough data volume and picking the right AI model.

In short, if you have a digital attic full of customer notes, you can now build a surprisingly accurate "digital ghost" of your customers to test ideas, without needing to interview them one by one.

Technical Summary: Synthetic Personalities from Heterogeneous Panel Data

1. Problem Statement

While Large Language Models (LLMs) offer a promising avenue for scaling market research, existing "digital twin" literature suffers from two critical limitations regarding operational relevance for marketing practice:

Data Source Mismatch: Most detailed individual-level twins are built on bespoke, purpose-collected datasets (e.g., multi-wave surveys designed specifically for twin construction or long-form AI interviews). This ignores the reality that firms already possess vast, heterogeneous panel data accumulated through CRM systems, loyalty programs, and repeat surveys, which lack a twin-oriented design and feature uneven information density.
Methodological Gap: While embedding methods, reasoning modes, and information depth have been examined anecdotally, no study has systematically compared these construction choices on a shared dataset and evaluation set to determine which combinations drive specific performance metrics.

This paper addresses whether detailed individual-level twins can be constructed from pre-existing, heterogeneous panel data and identifies the optimal construction-method grid for doing so.

2. Methodology

Data and Sample

The study utilizes the German Socio-Economic Panel (SOEP), a 41-year longitudinal household panel representing the German population.

Source: SOEP-Core v40 EU Edition (2023 wave), covering 10 thematic domains (e.g., employment, health, family, income).
Sample: A fixed random subsample of 500 participants drawn from 16,055 filtered records.
Data Structure: Each participant has 949 usable question-answer pairs, split into:
- 38 core demographic items (baseline context).
- 728 persona-context questions (used for construction).
- 183 held-out evaluation questions (used exclusively for testing prediction accuracy).

Construction-Method Grid

The authors evaluate a 3 × 5 × 2 × 2 grid (60 cells) varying four construction dimensions:

LLMs (3): Three open-weights models runnable on a single GPU (as of May 2026):
- Qwen 3 (30B-A3B)
- Gemma 4 (26B-A4B)
- Ministral 3 (14B)
- Note: All inference was performed locally on an NVIDIA H100 to ensure reproducibility and compliance with data licenses.
Information Depth (5): Cumulative levels of persona context, ranked by normalized Shannon entropy (a measure of a question's ability to differentiate between individuals):
- Basic Demographics only.
- Demographics + Top 25% of entropy-ranked questions.
- Demographics + Top 50%.
- Demographics + Top 75%.
- Demographics + 100% (All available persona-context data).
Embedding Method (2):
- Narrative Summary: A dense, Chain-of-Density compressed description of the participant's history.
- Dialog Input: Raw question-and-answer pairs formatted as alternating user/assistant chat turns.
Reasoning Mode (2):
- Non-Thinking: Direct response.
- Thinking: Explicit internal reasoning trace (via dedicated checkpoints or structured prompting) before the final response.

Evaluation Metrics

Performance was measured against ground truth on the 183 held-out questions using three metrics established in prior twin literature:

Accuracy: A score (0–1) measuring proximity to the human answer, adapted for nominal, ordinal, and metric scales.
Rank-Order Correlation: Fisher-z transformed Pearson correlation ( $r$ ) measuring whether the twin reproduces the relative ranking of participants (heterogeneity) rather than just the mean.
Dispersion Ratio: The ratio of twin response variance to human response variance, assessing if the twin captures the spread of human behavior or collapses toward the mean.

3. Key Results

Performance Benchmarks

Best Cell: The highest accuracy (78.8%) and rank-order correlation ( $r = 0.590$ ) were achieved by Gemma 4 and Qwen 3 respectively, both in the Dialog + Thinking + 100% Depth configuration.
Overall Scale: These performance levels are comparable to leading published twins built on purpose-collected data, demonstrating that pre-existing panel data is a viable substrate for high-fidelity twins.

Impact of Construction Choices

Information Depth: Both accuracy and correlation rose monotonically with information depth but exhibited diminishing returns.
- The 75% entropy quartile captured the bulk of the signal (approx. 584 of 728 items) at a moderate token cost.
- Moving from 75% to 100% depth yielded only marginal gains in accuracy (+0.6 pp on average) and sometimes reduced correlation gains, establishing the 75% level as a cost-efficient Pareto point.
Embedding Method: Switching from a narrative summary to a raw dialog history consistently raised hold-out accuracy across all models and reasoning modes at the 100% depth.
Reasoning Mode: Explicit thinking modes significantly increased rank-order correlation (reproducing individual heterogeneity) but did not consistently improve raw accuracy scores.
Model Performance:
- Qwen 3 led on average across all three metrics.
- Gemma 4 achieved the single best accuracy cell.
- Ministral 3 showed lower absolute performance but followed similar trends.

Ablation Findings

Empty Persona: When persona content was removed, accuracy remained flat (~0.65–0.66) regardless of depth, and correlation dropped to zero. This confirms that performance gains are driven by the specific individual data, not just instruction following.
Entropy Ranking: The specific ranking of questions by entropy had less impact than hypothesized; random selection of quartiles produced statistically indistinguishable results from entropy-ranked selection.

4. Contributions and Significance

The paper claims three primary contributions:

Data Viability: It demonstrates that detailed individual-level twins can be successfully constructed from pre-existing, heterogeneous panel data (like SOEP) rather than requiring bespoke, end-to-end data collection. This opens the digital twin paradigm to firms with existing CRM and loyalty data.
Systematic Method Mapping: It provides the first systematic comparison of construction levers (embedding, reasoning, depth) on a shared dataset, identifying which choices drive which metrics.
Actionable Guidelines: The study locates a clear Pareto point at the 75% entropy quartile, offering practitioners a rule for context budgeting that balances token costs with predictive performance. It further suggests that raw dialog history and explicit thinking modes are critical for maximizing heterogeneity and accuracy, respectively.

The authors conclude that the barrier to twin-based market research is no longer data design, but rather item volume, model selection, and a small set of construction-level decisions that this paper maps.