Calibrating Behavioral Parameters with Large Language Models

This paper proposes a framework to use large language models (LLMs) as calibrated measurement instruments for behavioral finance parameters, demonstrating that profile-based calibration can correct systematic rationality biases in LLMs to produce agent-based models that accurately replicate empirical asset pricing patterns.

The Gist

The Big Idea: Turning AI into a "Psychology Lab in a Box"

Imagine you are a scientist trying to study how humans make mistakes with money. To do this, you usually have to hire hundreds of people, pay them to sit in a room, and hope they don't get bored or lie to you. It’s slow, expensive, and messy.

This paper proposes a radical shortcut: Instead of using humans to study human mistakes, let’s use AI as a high-tech "tuning knob" to simulate those mistakes.

The researchers aren't trying to see if AI is human. Instead, they are treating the AI like a scientific instrument—sort of like a thermometer. A thermometer doesn't "act" like heat; it just reacts to it in a predictable way so you can measure it. The researchers found that if they "tell" the AI to act a certain way (using specific instructions), they can precisely dial up or dial down specific human biases, like greed, fear, or the tendency to follow the crowd.

---

The "Radio Dial" Analogy

Think of a Large Language Model (like ChatGPT) as a massive, complex radio.

• The Baseline (The "Static" Problem): If you just turn the radio on without touching anything, it doesn't sound like a human. It sounds "too perfect"—it’s too rational, too polite, and too logical. It lacks the "human noise" of mistakes and emotions.
• The Calibration (Turning the Dials): The researchers discovered they could turn specific "dials" on this radio.
  • If they turn the "Loss Aversion" dial up, the AI starts acting like someone who is terrified of losing $10.
  • If they turn the "Herding" dial up, the AI starts acting like a person who buys whatever everyone else is buying.
  • If they turn the "Extrapolation" dial up, the AI starts acting like a gambler who thinks a winning streak will last forever.

By turning these dials, they can create "Digital Humans" that have the exact same psychological settings as real people.

---

What did they find? (The Good, the Bad, and the "Not Quite Human")

The researchers tested eight different "human glitches" (biases). Here is how the AI performed:

1. The Successes (The "Digital Twins"):
For things like Loss Aversion (fear of losing), Herding (following the crowd), and Extrapolation (predicting the future based only on the recent past), the AI was a superstar. They could dial these up until the AI's behavior matched real-world human data almost perfectly.

2. The "Almost There" (The "Logic vs. Feeling" Gap):
For some things, the AI was close but not quite there. It could understand the logic of a bias, but it lacked the gut feeling.

3. The Failures (The "No Heart" Problem):
The AI failed at things that require true emotion or social ego.

• The Disposition Effect: Humans often hold onto "loser" stocks because it hurts our pride to admit we were wrong. The AI doesn't have "pride," so it couldn't replicate this well.
• Representativeness: Humans get swept up in "cool stories" (like a flashy new tech company). The AI is too focused on the math and the facts to get "hyped" by a good story.

---

Why does this matter? (The "Flight Simulator" for Finance)

Why go through all this trouble? Because once you have a "calibrated" AI, you can build a Flight Simulator for the Stock Market.

Instead of guessing how a market crash might happen, economists can build a digital world populated by thousands of these "calibrated" AI agents. They can say, "What happens to the economy if everyone suddenly becomes 50% more fearful of losing money?"

Because the AI is predictable and easy to "tune," researchers can run these simulations thousands of times to prepare for real-world financial storms.

Summary in a Nutshell

The paper proves that while AI isn't "human" (it has no heart or pride), it is an incredibly powerful mathematical puppet. If you pull the right strings, you can make it dance exactly like a human investor, making it a perfect tool for testing how the world's money might behave in the future.

Technical Summary

Technical Summary: Calibrating Behavioral Parameters with Large Language Models

1. Problem Statement

In behavioral finance, key parameters—such as loss aversion ($\lambda$), extrapolation ($\theta$), and herding ($w$)—are essential for explaining market anomalies like momentum, reversals, and excess volatility. However, measuring these parameters reliably is notoriously difficult due to:

• Measurement Error: Self-reported data (e.g., risk aversion) correlates poorly with actual incentivized behavior.
• Identification Problems: Observed choices are often a confounding mix of preferences, beliefs, and constraints.
• Scalability & Causality: Human laboratory experiments are expensive and cannot easily manipulate psychological "latent" parameters to establish causality.

The authors address whether Large Language Models (LLMs) can move beyond being mere "simulated subjects" to become calibrated measurement instruments capable of inducing and quantifying these behavioral primitives.

2. Methodology

The researchers propose a framework that treats behavioral profiles in prompts as experimental treatments designed to shift latent parameters in predictable directions.

• Experimental Design: They tested four LLM variants (GPT-4o, GPT-4o-mini, Claude-3.5-Haiku, and Gemini-2.5-Pro) across eight canonical behavioral biases using 19,200 agent–scenario pairs.
• Synthetic Environments: To prevent training data contamination, they constructed fully synthetic financial scenarios (assets, prices, and narratives) that are statistically indistinguishable from real market data but use abstract identifiers.
• Calibration Framework: They established four criteria for Calibration Validity:
  1. Monotonicity: Increasing profile strength must induce monotonic parameter changes.
  2. Range Coverage: The induced range must include or approach human benchmarks.
  3. Stability: Parameters must remain stable across repeated elicitations.
  4. Coherence: Recovered parameters must maintain theoretically predicted relationships (e.g., overconfidence should negatively correlate with herding).
• Validation Tiers: They categorized results into Strong (within 20% of human benchmarks), Moderate (within 50%), or Weak (correct direction only) validation.

3. Key Contributions

• Methodological Shift: The paper shifts the use of LLMs from replication (trying to mimic humans) to measurement (using LLMs as a controllable laboratory to study parameter ranges).
• Validation of LLMs as Instruments: They provide a systematic framework for using LLMs to populate Agent-Based Models (ABMs) with empirically grounded, rather than ad-hoc, behavioral rules.
• Boundary Mapping: The study explicitly defines the "cognitive vs. affective" boundary, identifying which biases are amenable to LLM calibration (cognitive/computational) and which are not (emotional/visceral).

4. Key Results

• Systematic Rationality Bias: In their baseline state (using "rational" profiles), LLMs exhibit significantly less bias than humans. For example, baseline loss aversion ($\lambda \approx 1.12$) is much lower than the human benchmark ($\lambda \approx 2.25$).
• Successful Calibration: Profile-based prompting successfully induced large, stable shifts. Four parameters achieved Strong Validation:
  • Loss Aversion ($\lambda$): Induced up to 3.00 (exceeding human benchmarks).
  • Herding Rate: Induced up to 90% (exceeding human benchmarks).
  • Extrapolation ($\theta$): Induced up to 0.88 (exceeding human benchmarks).
  • Anchoring ($\rho$): Induced up to 0.67.
• Calibration Failures: Parameters relying on emotional attachment or narrative salience—specifically the Disposition Effect and Representativeness Bias—failed to reach human-relevant magnitudes, suggesting LLMs lack the "visceral" component of these biases.
• External Validity (Asset Pricing): When the calibrated extrapolation parameter ($\theta = 0.88$) was embedded in an agent-based model, it successfully generated short-horizon momentum and long-horizon reversal patterns that closely matched empirical market stylized facts.

5. Significance

This research establishes a new frontier for Computational Economics. It demonstrates that while LLMs are not perfect "digital twins" of humans (due to a lack of real emotion and stakes), they are highly effective calibrated instruments for studying the cognitive mechanics of financial decision-making.

By providing a way to bridge individual behavioral biases with macro-level market dynamics (via ABMs), the paper offers a scalable, high-precision toolkit for researchers to stress-test asset pricing models and explore the causal impact of specific behavioral primitives on market stability.