Technical Summary: FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments

1. Problem Statement

The deployment of Large Language Model (LLM) powered financial agents for investment analysis, risk assessment, and automated decision-making introduces novel security risks that extend beyond traditional content compliance. Existing safety evaluation paradigms suffer from two primary limitations:

1. Content-Centric Focus: Current evaluations primarily assess language models at the textual level, focusing on whether models generate inappropriate content or refuse non-compliant requests. They fail to capture systemic risks arising from multi-step decision-making, tool invocation, and state transitions within real-world operational workflows.
2. Abstracted Environments: General agent safety benchmarks often utilize simulated environments with abstract tool interfaces (e.g., predefined responses or scripts). These lack the critical mechanisms of real financial systems, such as state-writable business databases, permission constraints, and auditable operation trails. Consequently, they cannot verify if an attack results in actual non-compliant business actions or tangible risk consequences.

Given that financial AI systems are classified as high-risk under regulations like the U.S. SR 11-7 and the EU AI Act, there is an urgent need for a security evaluation benchmark tailored to real-world financial scenarios that verifies safety and compliance within complex, execution-grounded operational workflows.

2. Methodology: FinVault Framework

To address these gaps, the authors propose FinVault, the first execution-grounded security benchmark for financial agents. The framework is built upon three core principles: regulation-driven scenario design, cross-domain coverage, and business realism.

2.1 Scenario Design and Environment

FinVault constructs 31 regulatory case-driven financial business scenarios within an isolated sandboxed environment. These scenarios cover six major financial domains:

• Credit & Lending
• Insurance Services
• Securities & Investment
• Payment & Settlement
• Compliance & Anti-Money Laundering (AML)
• Risk Management

Unlike abstract benchmarks, FinVault features executable environments with state-writable databases. Agents interact with these scenarios via toolchains to access and update scenario databases. The environment layer explicitly implements:

• Permission checks and authorization constraints.
• Compliance constraints (e.g., regulatory limits, sanctions lists).
• Audit logs for operation trails.

Success or failure of an attack is determined not by textual output alone, but by observable business state changes and their consequences (e.g., an unauthorized fund transfer or a bypassed credit limit).

2.2 Threat Model and Attack Construction

The benchmark defines 107 high-risk vulnerabilities derived from real regulatory violation patterns, categorized into five types:

1. Privilege Bypass: Circumventing authentication or authorization.
2. Compliance Violation: Inducing actions that violate regulatory requirements.
3. Information Leakage: Extracting sensitive customer or business data.
4. Fraudulent Approval: Manipulating decision processes for unauthorized outcomes.
5. Audit Evasion: Concealing violations from logging mechanisms.

The attack dataset comprises 963 test cases, including 856 adversarial samples and 107 benign business samples. The adversarial samples cover three attack categories and eight specific techniques:

• Prompt Injection: Direct JSON Injection, Instruction Overriding.
• Jailbreaking: Role Playing, Progressive Prompting, Encoding Obfuscation, Hypothetical Scenarios.
• Financially Adapted Attacks: Authority Impersonation, Emotional Manipulation.

The construction process involves a three-stage pipeline: expert design of seed attacks, model-based augmentation for diversity, and human verification for plausibility and success criteria clarity.

2.3 Experimental Setup

The authors evaluated 10 mainstream LLMs (including Qwen3, GPT-4o/5, Claude-Sonnet/Haiku, DeepSeek, etc.) acting as financial agents. They also assessed three defense mechanisms: LLaMA Guard 3, LLaMA Guard 4, and GPT-OSS-Safeguard.

Evaluation Metrics:

• Attack Success Rate (ASR): The proportion of attacks successfully triggering a vulnerability.
• Vulnerability Compromise Rate: The proportion of predefined vulnerabilities successfully exploited by at least one attack technique.
• False Positive Rate (FPR): The rate at which defense mechanisms incorrectly flag benign business instructions.
• Efficiency: Token consumption and time per case.

3. Key Results

3.1 Model Vulnerability

Experimental results reveal significant security disparities across models, with no evaluated model demonstrating robust protection in realistic financial settings:

• High Attack Success: The most vulnerable model (Qwen3-Max) exhibited an average ASR of 50.0%. Even the most robust models (Claude-Haiku-4.5) had an average ASR of 6.7%.
• Vulnerability Compromise: When restricting evaluation to whether any attack technique could exploit a vulnerability, the compromise rate reached 85.98% for the weakest models and 20.56% for the most robust models.
• Scenario Sensitivity: Insurance service scenarios were the most susceptible to attacks (up to 65.2% ASR for Qwen3-Max), likely due to complex policy interpretation and discretionary decision-making. Credit scenarios were comparatively more resilient due to rule-based decision processes.

3.2 Attack Technique Effectiveness

• Semantic Dominance: Semantic-level attacks (Role Playing, Hypothetical Scenarios, Authority Impersonation) were substantially more effective than technical attacks (e.g., Encoding Obfuscation). Role-playing attacks achieved consistently high ASR across all models.
• Instruction Isolation: Instruction-override attacks showed a massive performance gap between models. Qwen3-Max had an ASR of 64.50% against this technique, whereas Claude-Haiku-4.5 had only 3.70%, highlighting fundamental differences in instruction boundary enforcement.

3.3 Defense Mechanism Assessment

Existing defense methods face a severe trade-off between detection capability and false positives:

• LLaMA Guard 4 achieved the highest True Positive Rate (61.10%) but at the cost of a high False Positive Rate (29.91%), which could disrupt sensitive financial workflows.
• GPT-OSS-Safeguard maintained the lowest FPR (12.15%) but had a very low detection rate (22.07%) and high token consumption, making it unsuitable for real-time decision-making.
• Conclusion: No current defense provides robust protection; vulnerability compromise rates remain above 20% even for the strongest base models.

4. Key Contributions

1. First Execution-Grounded Benchmark: FinVault is the first systematic security benchmark for financial agents that utilizes isolated sandboxes with state-writable business databases. This allows security failures to be verified through observable business state changes and audit trails rather than relying solely on textual outputs.
2. Comprehensive Threat Modeling: The work systematically defines 107 high-risk vulnerabilities based on real regulatory patterns and constructs a taxonomy covering prompt injection, jailbreaking, and financially adapted attacks, resulting in a dataset of 963 test cases.
3. Empirical Evidence of Systemic Weakness: The study provides empirical evidence that general-purpose safety alignment does not readily transfer to financial settings. It highlights that semantic attacks pose a greater threat than technical ones and that current defenses struggle with the trade-off between detection and usability.

5. Significance and Claims

The paper claims that FinVault is essential for the research community to develop, evaluate, and deploy safer financial AI agents grounded in real-world operational and regulatory conditions.

The authors emphasize that the current state of financial agent security is precarious. The high Attack Success Rates and the inability of existing defenses to balance detection with low false positives indicate that general-purpose safety designs have limited transferability to the financial domain. The findings suggest that security failures in LLMs primarily stem from weaknesses at the semantic reasoning level (e.g., role compliance, instruction boundary blurring) rather than technical parsing deficiencies.

The paper concludes that establishing a dedicated, end-to-end security evaluation benchmark aligned with actual business processes is a prerequisite for the safe large-scale deployment of financial agents, particularly given the high stakes of direct financial loss and regulatory liability.

Limitations Acknowledged:
The authors note that while the eight attack techniques capture dominant paradigms, emerging strategies are not yet covered. Additionally, while the sandboxed environments approximate real workflows, they inevitably differ from production systems in scale and integration complexity. Finally, the ASR metric treats all successful attacks uniformly and does not capture differences in attack severity or downstream impact.