Investor risk profiles of large language models

This paper evaluates how three large language models (GPT, Gemini, and Llama) generate and adjust investor risk profiles in response to standardized questionnaires and persona-based prompts, revealing distinct baseline tendencies and varying degrees of adaptability across the models.

The Gist

Imagine you are walking into a bank to open an investment account. Before the banker gives you any advice, they sit you down with a questionnaire. They ask things like: "How old are you?", "How much money do you have?", and "If your investments dropped 20% tomorrow, would you panic and sell, or buy more?"

Based on your answers, the bank creates a "Risk Profile." This is like a personality badge that tells the bank: "This person is a cautious turtle," or "This person is a daring cheetah." The bank then uses this badge to decide what investments are safe for you.

Now, imagine that instead of a human banker, you are talking to a super-smart computer brain (an AI) like GPT, Gemini, or Llama. You might think, "Great! The AI will just read my answers and give me the perfect advice."

But this paper asks a very important question: "What is the AI's own personality before we even ask it anything? And if we tell it to pretend to be someone else, does it actually change its mind?"

Here is the story of what the researchers found, explained simply.

1. The AI's "Default" Personality

The researchers asked three different AI models to pretend to be an investor and answer the standard bank questionnaire. They didn't tell them who to be; they just asked them to answer.

It turns out, the AIs have their own hidden "default settings," just like a new phone comes with a default wallpaper.

• Gemini (The Steady Hand): This AI was the most consistent. It answered the same way every time. Its personality was "Moderate." It wasn't too scared of risk, but it wasn't crazy either. Think of it as a sensible middle-aged accountant who always wears a tie.
• Llama (The Cautious Turtle): This AI tended to be very conservative. It was scared of losing money. If you asked it to invest, it would probably suggest keeping your money in a savings account. It's the "safety first" type.
• GPT (The Wild Card): This AI was the most aggressive (willing to take more risks), but it was also the most unpredictable. Sometimes it acted like a risk-taker, sometimes like a conservative. It was like a moody teenager who changes their mind every hour.

The Big Takeaway: If you just ask an AI for advice without giving it details about you, it will give you advice based on its own hidden personality, not yours.

2. The "Acting Class" Test (Prompt Engineering)

The researchers then tried something fun. They told the AIs: "Okay, stop being yourself. Pretend you are a 20-year-old with no money," or "Pretend you are a 50-year-old millionaire."

They wanted to see if the AIs could "act" like different people.

• Did they change? Yes! When the researchers told the AIs to be "Risk-Averse" (scared of losing money), the AIs gave answers that were much more conservative. When they told them to be "Risk-Seeking" (adventurous), the AIs got bolder.
• Did they act their age? Yes. When told to be in their 20s, all three AIs said they were willing to take big risks. When told to be in their 50s, they became much more careful.
• Did they act rich? Yes. When told they were wealthy, the AIs said they could afford to take bigger risks. When told they were poor, they got scared.

The Analogy: Imagine an actor on stage.

• Without a script (Default): The actor just talks about their own life.
• With a script (Persona): If you hand the actor a script that says, "You are a 20-year-old gambler," they will act like a gambler.
• The Catch: Even with the script, the actor still has their own "voice." The paper found that while the AIs did change their answers, they didn't all change by the exact same amount. Some actors (like Llama) were harder to convince to change their mind than others (like GPT).

3. Why This Matters to You

This paper is a warning label for the future of financial advice.

In the future, you might talk to an AI to get investment advice. You might say, "I am a 30-year-old teacher with $50,000." The AI will try to act like a 30-year-old teacher.

However, the paper warns us that:

1. The AI has a hidden bias. Even if you give it your details, its "default" personality might still peek through.
2. Not all AIs are the same. If you ask GPT for advice, you might get a different answer than if you ask Gemini, even if you give them the exact same details about yourself.
3. Consistency is key. If you ask the same AI the same question twice, it might give you two different answers (especially GPT).

The Bottom Line

Think of these AI financial advisors as newly hired interns. They are very smart and can learn to act like different types of investors if you tell them to. But they also have their own quirks and habits.

If you use them for your money, you need to know that:

• They aren't perfect mirrors of you; they are mirrors with a slight tint of their own personality.
• You need to be very specific in your instructions (prompts) to get the advice you actually want.
• You shouldn't trust just one AI; it's smart to check a few different ones to see if they agree.

The researchers conclude that while these AIs are getting better at giving personalized advice, we need to be careful and not treat them like human experts just yet. They are powerful tools, but they are still learning how to wear the "financial advisor" hat properly.

Technical Summary

Here is a detailed technical summary of the paper "Investor risk profiles of large language models."

1. Problem Statement

As Large Language Models (LLMs) increasingly serve as financial advisors for retail investors, a critical gap exists in understanding how these models interpret and internalize investor risk profiles. The investor risk profile is a foundational concept in financial advising, determining asset allocation (e.g., exposure to cryptocurrencies or leveraged products) based on two dimensions: risk preference (willingness to take risk) and risk capacity (ability to take risk).

While traditional methods rely on standardized questionnaires to measure these profiles, it remains unclear whether LLMs possess a consistent "default" risk persona or if they can accurately adjust their advice when prompted with specific user personas (e.g., age, wealth, experience). The study addresses the lack of empirical data on the consistency, bias, and adaptability of leading LLMs in generating investor risk profiles.

2. Methodology

The researchers employed a quantitative experimental design using a standardized financial questionnaire to evaluate three prominent LLMs: GPT-4o, Gemini 1.5 Pro, and Llama 3.1 (70b).

• Instrument: The study utilized the Charles Schwab Investor Profile Questionnaire, selected for its public availability and explicit scoring methodology. The questionnaire consists of 7 questions:
  • Time Horizon (2 questions): Investment period and spending period after withdrawal.
  • Risk Tolerance (5 questions): Investment knowledge, willingness to take risk, current holdings, reaction to losses, and acceptable outcomes.
• Experimental Design:
  • Default Profile: Each model was prompted 100 times with a neutral persona ("Suppose you are an individual investor...") to establish a baseline default risk profile.
  • Persona Assignment: The researchers tested the models' adaptability by modifying the prompt to assign specific attributes:
    • Risk Appetite: Risk-averse, risk-neutral, risk-seeking.
    • Demographics: Age groups (20s, 30s, 40s, 50s).
    • Economic Status: Below average, average, above average wealth.
    • Experience: No experience, some experience, professional experience.
  • Data Collection: For each condition, 100 iterations were run, with new conversations initiated for each iteration to minimize context carryover.
• Statistical Analysis:
  • Metrics: Mean and median scores for Time Horizon and Risk Tolerance were calculated.
  • Hypothesis Testing: ANOVA (for means) and Kruskal-Wallis (K-W) tests (for medians) were used to determine statistical significance. The K-W test was prioritized in cases where variance homogeneity (Levene's test) or normal distribution assumptions were violated.

3. Key Contributions

• Baseline Profiling: The study establishes the first empirical "default" risk profiles for GPT, Gemini, and Llama, revealing inherent biases in their self-assessment.
• Persona Sensitivity Analysis: It quantifies the extent to which LLMs adjust their risk recommendations based on specific user attributes (age, wealth, experience), validating the efficacy of persona prompting in financial contexts.
• Model Comparison: It provides a comparative framework showing that different LLM architectures exhibit distinct behavioral patterns regarding risk tolerance and consistency.

4. Key Results

A. Default Risk Profiles (Without Specific Personas)

• General Trend: All three models default to a long-term investment horizon.
• Risk Tolerance Differences:
  • Gemini: Exhibits a moderate risk level with highly consistent responses (low variance).
  • Llama: Skews toward a conservative risk profile.
  • GPT: Appears moderately aggressive but demonstrates the greatest variability in responses.
• Statistical Significance: ANOVA and K-W tests confirmed that the differences in risk scores between the models are statistically significant (p < 0.01) across most questions.

B. Impact of Assigned Personas

When specific attributes were assigned, the models adjusted their profiles, though the magnitude and direction of adjustments varied:

1. Risk Appetite:

   • All models successfully reflected assigned risk levels (Risk-averse $\approx$ 10 score, Risk-neutral $\approx$ 20, Risk-seeking $\approx$ 35-40).
   • Anomaly: GPT correlated higher risk tolerance with a longer time horizon. Conversely, Gemini assigned a lower time horizon score to the "risk-seeking" persona, suggesting a unique interpretation of the relationship between risk and time.

2. Age:

   • All models assigned the highest risk scores to the 20s and decreased scores as age increased.
   • GPT showed the most sensitivity to age, with the largest range in risk scores between the 20s and 50s.

3. Wealth:

   • All models exhibited a positive correlation between wealth and risk tolerance; "above average" wealth resulted in significantly higher risk scores.

4. Investing Experience:

   • A strong relationship was observed where "no investing experience" resulted in significantly lower risk scores compared to "some" or "professional" experience.
   • Time horizon scores were less affected by experience levels.

C. Statistical Consistency

• Gemini remained the most consistent model, often showing identical answers across iterations (resulting in "nan" for variance-based tests in some persona categories).
• GPT and Llama showed significant variance, particularly in GPT's responses to age and wealth variations.

5. Significance and Implications

• Viability for Robo-Advisory: The study confirms that LLMs can be guided via prompt engineering to align with specific investor preferences, making them viable candidates for personalized financial advice.
• Need for Caution: The existence of distinct default biases (e.g., GPT's inherent aggressiveness vs. Llama's conservatism) and inconsistencies (especially in GPT) poses a risk. If a user does not explicitly define their persona, the LLM may provide advice based on its own "hallucinated" profile rather than the user's actual situation.
• Prompt Engineering Criticality: The results underscore that the quality of financial advice from LLMs is heavily dependent on the precision of the prompt. Relying on default settings without explicit persona assignment could lead to mismatched risk profiles.
• Future Research: The findings suggest a need for standardized evaluation benchmarks for LLMs in financial domains and further investigation into why different models interpret the relationship between risk, time, and wealth differently.