Technical Summary: FinPersona-Bench

Problem Statement

Large Language Models (LLMs) are increasingly deployed as autonomous financial agents initialized with explicit behavioral mandates (e.g., "preserve capital," "avoid speculative bets") intended to govern decision-making throughout their deployment. However, as market context accumulates over long horizons, these mandates gradually lose their behavioral influence. The authors formalize this phenomenon as Mandate Salience Decay (MSD).

Unlike traditional reasoning errors, MSD represents a behavioral failure where an agent may execute locally coherent or even profitable trades while simultaneously violating its core risk constraints. Existing financial benchmarks fail to capture this longitudinal drift, focusing instead on static knowledge retrieval, short-term returns, or single-point-in-time performance. Furthermore, prior work on "context rot" or "identity drift" in general NLP has not been objectively quantified in high-stakes financial settings where intrinsic value can be mathematically defined.

Methodology

1. FinPersona-Bench Architecture

The authors introduce FinPersona-Bench, a simulation benchmark designed to objectively measure MSD. The system comprises three core components:

• Synthetic Market Engine: To establish a mathematically defined ground truth, the engine decouples the observable market price ($P_t$) from the hidden fundamental value ($V_t$) via a stochastic noise process. This allows for falsifiable evaluation of mandate violations. The engine simulates three distinct market scenarios:
  • The Bull Trap (Bubble): Tests the ability to distinguish legitimate growth from speculative excess where price decouples from value.
  • The Market Crash (Panic): Tests risk management during a sharp price drop below fundamental value.
  • The Flat Market (Low-Signal): Tests whether agents maintain their mandate without valid trading signals.
• Agent Framework: Agents are assigned behavioral profiles grounded in the Myers-Briggs Type Indicator (MBTI) (specifically ENTJ, ISFJ, and INTJ) and the Big Five (for validation). These profiles define risk appetites and decision styles. The framework compares two architectural variants:
  • Static Baseline: The mandate is provided once at initialization.
  • Memory Re-grounding: The core mandate is re-injected into the user message at every decision step to test if recency bias mitigates decay.
  • Placebo Control: A length-matched regulatory text without behavioral instructions is re-injected to isolate semantic content from positional recency.
• Behavioral Evaluation Pipeline: The system measures three specific failure modes using objective metrics:
  • Mandate Adherence Score (MAS): Measures deviation from target cash allocation in flat markets.
  • Caricature Index (CI): Uses Maximum Drawdown (MDD) to quantify "caricature" effects where agents exaggerate default tendencies under stress (e.g., panic selling).
  • Rationality Gap (RG): Measures the probability of actions aligning with the hidden fundamental value ($V_t$) rather than the observable price ($P_t$).

2. Experimental Setup

The study evaluates 18 leading frontier and open-source LLMs (including GPT, Claude, Gemini, DeepSeek, and open-source models like Llama and Gemma) across a combinatorial grid of personas, market scenarios, and architectures. Each configuration is replicated across five random seeds over a 200-day trading horizon.

Key Results

1. Mandate Salience Decay is Compounding

MSD is not a static offset but a compounding phenomenon. In crash scenarios, the behavioral gap between static agents and those receiving periodic mandate re-grounding grows 4.4× from the first quarter to the final quarter of the simulation. This indicates that the influence of the mandate erodes progressively as context accumulates.

2. Re-grounding is Not Universally Beneficial

The effectiveness of mandate re-grounding is highly dependent on the alignment between the agent's profile and the market regime:

• Conservative Agents (ISFJ) in Flat Markets: Re-grounding significantly reduces drift (12.7% improvement in MAS), helping agents resist the implicit pressure to trade in low-signal environments.
• Aggressive Agents (ENTJ) in Flat Markets: Re-grounding actively worsens behavior. Forcing a growth-oriented mandate in a signal-less market amplifies unnecessary trading, increasing mandate drift by up to 156.4%.
• Speculative Markets (Bull Traps): Re-grounding imposes a "rationality cost." Continuously enforcing a conservative mandate during a rising market suppresses the buying of undervalued assets, leading to lower rationality scores compared to static agents who naturally drift toward the market trend.

3. Model-Dependent Heterogeneity

• Capability Scaling: Within the Claude family, vulnerability to crash-induced MSD scales inversely with capability; smaller models (Haiku) exhibit extreme mandate erosion, while larger models (Opus) show higher resilience.
• Capacity Reversals: In the GPT family, flagship models benefit from re-grounding, but "mini" variants often exhibit reversals where static agents outperform re-grounded ones, particularly in bull traps.
• Open-Source Dissociation: Some open-source models (e.g., Qwen2.5-7B) show that re-grounding can restore persona-consistent language without correcting underlying trading behavior, leading to active harm across all scenarios.

4. Mechanisms of Failure

• Semantic Content vs. Position: A placebo control confirmed that re-grounding benefits arise from the semantic content of the mandate, not merely the recency of appended text.
• Linguistic Conflict: In crash scenarios, smaller models (mini variants) exhibit a higher "Lexical Conflict Rate" when re-grounded. They tend to treat mandate language as an action trigger rather than a behavioral filter, leading to contradictory reasoning (e.g., citing a "growth" mandate to justify panic selling).

Significance and Claims

The paper claims to provide the first objective, falsifiable measurement of longitudinal behavioral stability in autonomous financial agents. By formalizing MSD, the authors demonstrate that:

1. Behavioral drift is distinct from reasoning errors: Agents can be locally rational while globally violating their fiduciary mandates.
2. Static deployment is insufficient: Long-horizon deployment requires selective, mandate-aware re-grounding based on the agent's profile and the current market regime, rather than a universal "re-inject everything" strategy.
3. Current benchmarks are incomplete: Standard financial evaluations fail to capture the progressive erosion of behavioral constraints over time.

The authors emphasize that these findings suggest current LLMs should not be deployed in live fiduciary roles without rigorous, mandate-aware verification mechanisms that account for the specific interaction between agent persona and market conditions. The work serves as a diagnostic tool to identify failure modes rather than a prescriptive solution for all deployment scenarios.