📈 economics

Explanation Depth and Trust Calibration in AI-Driven Financial Advice: A Behavioural Experiment

This behavioural experiment reveals that while increasing explanation depth in AI financial advice initially boosts overtrust, it does not guarantee calibrated trust and may unexpectedly increase overreliance among finance-educated users due to fluency-induced credulity.

Original authors: Shivanshu Kashyap, Nikita Srivastava

Published 2026-06-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shivanshu Kashyap, Nikita Srivastava

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are asking a very smart, very confident robot for financial advice. You want to know: How much should I trust this robot?

This paper is like a lab experiment where researchers tested exactly that. They wanted to see if giving the robot a "longer, fancier explanation" makes people trust it more, even when the robot is wrong.

Here is the story of what they found, broken down into simple concepts and analogies.

The Setup: The "Robot Chef" Experiment

The researchers set up a game with 100 people. They acted as a "Robot Chef" (the AI) giving out 12 different pieces of financial advice.

Some advice was True (like "eat your vegetables").
Some advice was False (like "eat rocks for better health").

The twist? The Robot Chef gave these pieces of advice in three different styles:

Low Depth: Just a short command. "Eat vegetables."
Moderate Depth: A nice paragraph explaining why. "Eat vegetables because they have vitamins that help your body fight off sickness."
High Depth: A super-detailed, technical lecture. "Eat vegetables because of the specific micronutrient density, the pH balance of your stomach, and the thermodynamic efficiency of cellular respiration..."

The researchers wanted to see: If the Robot Chef gives a long, fancy explanation for a BAD idea, will people believe it?

The Big Discovery: The "Sweet Spot" of Danger

The most surprising finding was that more explanation isn't always better. In fact, there is a "Danger Zone."

When the explanation was short (Low Depth): People were skeptical. They didn't trust the bad advice much. They were like, "That's a short answer; I'm not buying it."
When the explanation was super long and technical (High Depth): People were still a bit skeptical. The sheer amount of words made them pause and think, "Wow, that's a lot of words. Is it true?"
When the explanation was "Moderate" (The Danger Zone): This is where it got tricky. The explanation was just long enough to sound smart and convincing, but not so long that it felt overwhelming.
- The Result: In this "Moderate" zone, people stopped distinguishing between good and bad advice. They trusted the bad advice almost as much as the good advice.
- The Analogy: Think of it like a magician. If the magician says "Watch this," you are suspicious. If the magician gives a 20-minute lecture on quantum physics while doing a trick, you might be confused and suspicious. But if the magician gives a 30-second, smooth, confident explanation that sounds just right, you are most likely to believe the trick is real, even if it's fake. The "Moderate" explanation is the perfect camouflage for a lie.

The Twist: The "Expert Trap"

The researchers also looked at who was most easily fooled. They expected that people who study finance (like accountants or investors) would be the smartest and least likely to be tricked.

They were wrong.

The Finding: The people with finance backgrounds were actually more likely to be fooled by the long, fancy explanations than the regular people.
The Analogy: Imagine a person who speaks perfect French. If a stranger speaks to them in broken, simple French, they might say, "That doesn't sound right." But if the stranger speaks in perfect, complex, academic French, the French expert might think, "Wow, this person knows their stuff!" and believe them without checking the facts.
Why? The researchers call this "Fluency-Induced Credulity." Because the finance experts recognized all the fancy words (like "inflation" or "supply and demand"), their brains went on "autopilot." They felt familiar with the language, so they stopped thinking critically. They trusted the sound of the explanation rather than the truth of the advice.

The Takeaway

The paper concludes with a few clear lessons for how we build and use AI:

More words $\neq$ More truth. Just because an AI gives a long, detailed explanation doesn't mean it's right. In fact, a "medium-length" explanation might be the most dangerous because it tricks us into feeling confident without making us think hard enough.
Experts aren't immune. If you know a lot about a topic, you might be more likely to trust a robot that uses the right jargon, even if the robot is lying. Your familiarity with the words can be a trap.
The "Danger Zone" exists. There is a specific level of detail where our ability to tell truth from lies breaks down. We need to be careful when AI gives us explanations that feel "just right"—not too short, not too long.

In short: Don't let a fancy explanation fool you into trusting a bad idea. Sometimes, the most convincing lie is the one that sounds just smart enough to make you stop asking questions.

Technical Summary: Explanation Depth and Trust Calibration in AI-Driven Financial Advice

Problem Statement

The rapid integration of AI-powered robo-advisors into the financial sector has introduced a critical challenge regarding trust calibration—the alignment of a user's confidence in an AI system with the system's actual reliability. While existing literature on Explainable AI (XAI) generally posits that providing explanations increases user trust and transparency, there is a lack of empirical evidence regarding how the depth and complexity of these explanations influence user behavior. Specifically, it remains unclear whether elaborating on AI reasoning mitigates overtrust (automation bias) or inadvertently exacerbates it. Current research often treats explanations as a binary variable (present vs. absent) rather than investigating the continuous design parameter of explanation depth. This study addresses the gap in understanding how varying levels of explanatory detail (Low, Moderate, High) affect the calibration of trust and the susceptibility to overtrust in financial advisory contexts.

Methodology

The study employed a within-subjects experimental survey design to isolate the causal effects of explanation depth on trust calibration.

Participants: A convenience sample of N = 100 respondents was recruited. The demographic profile included 57% female and 43% male participants, with 73% aged 20–35 and 27% aged 36–60. Crucially, the sample was stratified by domain expertise: 18% self-identified as having a formal finance background (e.g., CA, MBA, CFA), while 82% had no formal finance education.
Stimuli: Participants were exposed to 12 AI-generated financial advisory statements. These stimuli varied systematically across two dimensions:
1. Explanation Depth:
  - Low: Brief, assertion-style recommendations with no rationale.
  - Moderate: Structured explanations referencing general principles (e.g., diversification, risk management).
  - High: Detailed explanations embedding accurate financial sub-concepts (e.g., CPI dynamics, purchasing power parity) within the reasoning, even when the overall conclusion was factually incorrect.
2. Factual Accuracy: Statements were either factually true or false.
Procedure: Participants rated their agreement with each statement on a 5-point Likert scale (1 = Strongly Disagree, 5 = Strongly Agree).
Measures:
- Overtrust Score: Mean agreement with factually false statements.
- Undertrust Score: Derived measure reflecting the rejection of factually true statements.
- Trust Calibration Index: The proportion of correctly calibrated responses.
Analytical Strategy: Data analysis utilized one-way repeated measures ANOVA to test the main effect of explanation depth, mixed ANOVA to test interactions with finance background, and one-sample t-tests to identify "calibration danger zones" where overtrust approached scale neutrality.

Key Results

1. Non-Linear Effect of Explanation Depth on Overtrust

A one-way repeated measures ANOVA revealed a significant main effect of explanation depth on overtrust ( $F(2, 198) = 16.62, p < .001$ ).

The Jump: Overtrust increased steeply between the Low and Moderate explanation conditions.
The Plateau: There was no significant difference between Moderate and High depth conditions, suggesting a threshold effect where additional elaboration yields diminishing returns in terms of misplaced confidence.
Asymmetry: The effect was specific to false statements. While agreement with true statements also increased slightly with depth, the effect size was significantly smaller ( $\eta^2_g = .014$ vs. $.059$ for false statements), ruling out a general acquiescence bias.

2. The "Calibration Danger Zone"

One-sample t-tests comparing overtrust scores against the scale midpoint (3.0) identified a critical finding:

Low Depth: Overtrust was significantly below neutral ( $M=2.34, p < .001$ ), indicating healthy skepticism.
High Depth: Overtrust remained below neutral ( $M=2.72, p = .004$ ).
Moderate Depth: Overtrust was not significantly different from the neutral midpoint ( $M=2.88, p = .188$ ). This identifies the Moderate depth condition as a "calibration danger zone" where users lose the ability to reliably distinguish accurate from inaccurate advice.

3. Counter-Intuitive Interaction with Finance Background

Contrary to the hypothesis that domain expertise would act as a buffer, a significant interaction was found between explanation depth and finance background ( $F(2, 196) = 4.18, p = .017$ ).

Finance-Educated Group: Showed a steep increase in overtrust from Low ( $M=1.94$ ) to High depth ( $M=2.92$ ).
Non-Finance Group: Showed a flatter trajectory with less variation across depth levels.
Interpretation: Finance-educated respondents were more susceptible to overtrust at high explanation depths. The authors attribute this to fluency-induced credulity: the familiarity with technical terminology activated confirmatory associations, bypassing critical evaluation of the false conclusions.

4. Demographic Null Findings

No significant differences in trust calibration were found based on age, gender, or finance background (as a main effect). This suggests that the observed miscalibration is driven primarily by the structural properties of the AI communication (explanation depth) rather than individual demographic characteristics.

5. Commission vs. Omission Errors

While the difference between overtrust and undertrust scores was directionally consistent with automation bias theory (commission errors > omission errors), the paired t-test was marginally non-significant ( $p = .062$ ).

Key Contributions

The study makes three primary contributions to the literature on XAI and behavioral finance:

Empirical Evidence on Depth: It demonstrates that explanation depth is not linearly beneficial; rather, it significantly increases the likelihood of overtrust in financial recommendations, particularly when moving from low to moderate elaboration.
Identification of a Danger Zone: It empirically identifies a "calibration danger zone" at Moderate explanation depth, where explanations are sufficiently detailed to reduce skepticism but not detailed enough to trigger critical scrutiny, leading to a loss of discriminative capacity.
Expertise Paradox: It challenges the assumption that domain expertise protects against AI miscalibration. The study provides evidence that finance-educated individuals are more vulnerable to explanation-induced overtrust at high depths due to fluency-induced credulity, a finding that links the "curse of expertise" to AI interaction.

Significance and Implications

The paper argues that the current XAI design paradigm, which often treats "more explanation" as categorically better, requires revision. The findings suggest that:

Design Implications: Robo-advisory developers must recognize that explanatory elaboration does not necessarily build calibrated trust. Instead, it can generate a false sense of understanding, particularly at moderate levels of detail.
Regulatory Implications: Regulatory frameworks should move beyond simple disclosure requirements to mandate testing of explanation formats against calibration benchmarks. Systems should be evaluated not just for transparency, but for their ability to help users differentiate reliable from unreliable advice.
Theoretical Shift: The study highlights that trustworthy AI must be "calibratingly transparent." This means designing explanations that genuinely enable users to assess reliability, rather than explanations that merely signal competence through fluency and technical jargon.

The authors conclude that achieving this standard is a complex design and governance challenge that extends beyond information provision, requiring a deep understanding of the cognitive mechanisms through which users process AI-generated explanations in high-stakes financial environments.