💻 computer science

When Do LLM Personas Support Visualization Design? A Cross-Model Study of Color Assignment and Chart Choice

This study demonstrates that while LLM personas can reveal some stable personality-driven patterns in visualization design tasks like color assignment and chart choice, their outputs are heavily dependent on model configuration and task context, positioning them as useful exploratory probes rather than reliable substitutes for human participants.

Original authors: Shahreen Salim, Klaus Mueller

Published 2026-07-03✓ Author reviewed ⓘ

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shahreen Salim, Klaus Mueller

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an architect trying to design a new building. Before you hire a team of real people to give you feedback, you decide to ask a group of highly advanced, super-smart robots to pretend to be different types of people. You tell Robot A, "Pretend you are a calm, organized person," and Robot B, "Pretend you are a chaotic, emotional person." Then you ask them, "What color should the walls be?" or "What kind of chart looks best for this data?"

This paper is a report card on how well those robots actually act like the different people they are pretending to be. The researchers wanted to know: Do these "persona" robots actually change their answers based on the personality you give them, or are they just following a script that ignores the personality?

Here is what they found, broken down into simple ideas:

1. The "Color Test": It Depends on the Robot Model

The researchers asked three different versions of a robot (GPT-4o-mini, GPT-4.1-mini, and GPT-5-mini) to pick colors for two types of ideas:

Concrete ideas: Things you can touch, like a Banana or a Carrot.
Abstract ideas: Things you can't touch, like Chaos or Serenity.

The Results:

The "Banana" Problem: When asked about concrete things (like a banana), the robots mostly ignored the personality. A "chaotic" robot and a "calm" robot both picked yellow for a banana. The real-world object was too strong; the robot knew a banana is yellow no matter what.
The "Chaos" Surprise: When asked about abstract things (like "Chaos"), the robots did start to act differently based on the personality. A "chaotic" robot might pick a wild, bright red for the concept of Chaos, while a "calm" robot might pick a soft blue.
The Robot Model Matters: This was the biggest shock. One robot model (GPT-4.1-mini) was very good at acting like the different personalities. Another model (GPT-4o-mini) completely ignored the personality instructions for colors. A third model (GPT-5-mini) was somewhere in the middle.

The Takeaway: You can't just ask any robot to pretend to be a person and expect it to work. The specific "brain" inside the robot changes the results.

2. The "Chart Test": The Job Matters More Than the Person

Next, they asked the robots to choose the best type of chart (like a pie chart, a line graph, or a bar chart) for different jobs:

Showing a family tree (Hierarchy).
Showing stock prices over time (Time Series).
Comparing two teams (Comparison).

The Results:

The "Context" King: No matter what personality the robot had, if the job was "showing a family tree," almost every robot picked a Tree Map. If the job was "stock prices," they all picked a Line Chart. The task was so obvious that the personality didn't change the top choice.
The "Group Hug" Effect: When the researchers grouped the robots into personality clusters (like "The Organized Group" vs. "The Emotional Group"), the rankings became very stable. The groups agreed on the best chart.
The "No-Persona" Baseline: The researchers also asked the robots to pick a chart without giving them a personality. Surprisingly, the robots picked the exact same top chart 8 out of 9 times as they did when they were pretending to have a personality.

The Takeaway: The personality of the robot mostly changed how much they liked a chart (giving it a 5-star vs. a 4-star rating), but it rarely changed which chart they thought was the best. The type of data being shown was the boss, not the personality.

3. The Big Conclusion: Robots are "Explorers," Not "Replacements"

The authors conclude that using AI personas is a great way to explore ideas early in the design process. It's like using a sketch to get a rough idea of what might work.

However, you cannot use these robots as a replacement for real human participants.

If you want to know exactly how a real human will feel, you still need to ask a real human.
If you use a robot, you have to test it with multiple different robot models to make sure the results aren't just a glitch in one specific robot's brain.
You have to be careful about what you are asking. If you ask about concrete things (like fruit), the robot's "personality" won't show up. If you ask about abstract things, it might.

In short: AI personas are like a fun, quick "what-if" game for designers, but they aren't a crystal ball that can perfectly predict human behavior. You still need real people for the final answer.

Technical Summary: When Do LLM Personas Support Visualization Design?

Problem Statement
Large Language Model (LLM) personas are increasingly utilized to simulate diverse user profiles during early-stage visualization research, offering a cost-effective alternative to human studies for exploring design alternatives like color assignment and chart preferences. However, a critical methodological risk remains: it is unclear whether persona-conditioned outputs reflect stable personality effects or are merely artifacts of specific model configurations, prompt defaults, or task semantics. This study investigates whether conditioning LLMs on Big Five personality profiles produces systematic, stable differences in visualization choices, or if these signals are fragile and dependent on the underlying model and task type.

Methodology
The authors conducted two experiments across three model configurations: GPT-4o-mini, GPT-4.1-mini, and GPT-5-mini, using a consistent set of 43 unique Big Five personality profiles derived from state-level U.S. data.

Experiment 1 (Color Assignment): The study tested whether personality conditioning alters the colors assigned to six concepts (three concrete: Banana, Strawberry, Carrot; three abstract: Serendipity, Serenity, Chaos). For each concept-persona pair, 30 color samples were generated per model.
- Data Processing: RGB outputs were converted to CIELCH color space. Between-persona variation was quantified using average pairwise Hellinger distance across hue histograms.
- Analysis: A Mantel permutation test assessed the correlation between Big Five personality distance (L1 distance) and color distance. A two-way ANOVA decomposed hue variance into contributions from model identity and persona.
- Note on GPT-5-mini: As temperature was not exposed, variability was induced via eight interpretive lenses (e.g., literal, metaphorical) across fixed seeds.
Experiment 2 (Chart Preferences): Building on Alves et al.'s work, this experiment evaluated chart-idiom preferences across three task contexts (hierarchy, time series, comparison) using 12 chart types.
- Simulation: The 43 profiles were resampled into 60 persona-conditioned runs. Personas were aggregated into three trait-aligned clusters (Organized/Stable, Sociable/Cooperative, Emotionally Reactive) based on L1 distance to prototype vectors.
- Analysis: Preferences were analyzed via individual-trait OLS regressions and cluster-level aggregation methods (Majority Vote, Instant Runoff Voting, Apriori Support, Borda Count). A "no-persona" baseline was established to determine if top choices were driven by context rather than personality. Cross-model agreement was measured using Kendall's $\tau$ .

Key Results

Model Configuration Dependence: Personality-color coupling is highly dependent on the specific model.
- GPT-4o-mini: Showed no significant association between personality distance and hue distance for any of the six concepts.
- GPT-4.1-mini: Demonstrated consistent coupling across all six concepts.
- GPT-5-mini: Showed partial coupling (significant for only two of six concepts), with effects sensitive to hue-bin resolution.
Concept Type Influence: The signal strength varies by concept type. For abstract concepts, personality explained significantly more hue variance ( $\eta^2 = 0.038$ ) than model identity ( $\eta^2 = 0.008$ ). For concrete concepts, both effects were small and comparable, suggesting concrete concepts are constrained by default color associations.
Stability of Aggregation: In chart choice, aggregating personas into trait-aligned clusters stabilized top-idiom rankings for hierarchy and time-series contexts across all models. However, the comparison context showed lower stability, particularly for the "Emotionally Reactive" cluster.
Context vs. Persona: A no-persona baseline recovered the same top-ranked chart idiom as the persona-conditioned runs in 8 out of 9 model-context cells. This indicates that task context (e.g., "hierarchy" vs. "time series") drives rank-1 selection more strongly than personality conditioning. Persona prompting primarily modulates absolute rating levels (e.g., the "Emotionally Reactive" cluster rated idioms ~1 point lower on a 7-point scale) rather than reordering the top choice.

Significance and Claims
The paper positions LLM personas as exploratory probes for visualization design rather than substitutes for human participants. The authors argue that persona-conditioned outputs are not a single, stable property of prompting but are contingent on model configuration, concept type, and analysis granularity.

The study motivates three specific methodological practices for future research:

Multi-model testing: Relying on a single model configuration may yield false negatives (as seen with GPT-4o-mini) or artifacts.
Concept-type disaggregation: Pooling abstract and concrete concepts can obscure distinct patterns of persona influence.
No-persona baselines: Researchers must report baseline results to distinguish between context-driven defaults and genuine personality effects.

The authors explicitly state that these findings do not claim LLM personas can replace human studies; rather, they highlight the need for small-scale human calibration to determine which observed patterns reflect genuine user preferences and which are model artifacts.

1. The "Color Test": It Depends on the Robot Model

2. The "Chart Test": The Job Matters More Than the Person

3. The Big Conclusion: Robots are "Explorers," Not "Replacements"

Technical Summary: When Do LLM Personas Support Visualization Design?

More like this