Shapley in Context: Explaining Financial Language with Domain Expertise

Technical Summary: Shapley in Context: Explaining Financial Language with Domain Expertise

Problem Statement
Large Language Models (LLMs) are increasingly adopted in financial applications for tasks ranging from sentiment analysis to risk management. However, the financial domain is characterized by high stakes and strict regulatory requirements, making model explainability essential rather than optional. While numerous attribution methods exist for explaining black-box models, most are designed for general-purpose tasks and lack integration with domain-specific knowledge. The central problem addressed is whether Shapley value-based attributions can provide explanations for LLMs applied to financial text that are consistent with established financial domain knowledge and reasoning.

Methodology
The authors propose a framework to evaluate the explainability of LLMs on financial textual data through the lens of the Shapley value, specifically the Baseline Shapley value (BShap). The methodology involves three primary components:

1. Theoretical Framework and Axioms: The authors introduce a set of domain knowledge-inspired axioms tailored to the structure of financial text and LLM behavior. These axioms serve as formal criteria for faithful explanations:

   • First-order effects: Average Individual Monotonicity (if a feature consistently increases the output, its attribution must be positive) and Demand Individual Monotonicity (if a feature becomes "stronger" in a positive sense, its attribution must increase).
   • Second-order effects: Diminishing and Increasing Marginal Effect axioms, capturing non-linear interactions where the impact of a feature change depends on the presence of other features.
   • Cross-feature and Cross-model comparisons: Pairwise Monotonicity (comparing relative importance of different features within a model) and First-order Monotonic Dominance (comparing the importance of the same feature across different models). The authors also prove that comparing different features across different models is generally not valid under the Shapley framework.

2. Theoretical Analysis: The paper provides rigorous mathematical proofs demonstrating that the Baseline Shapley value satisfies all the proposed domain knowledge-inspired axioms. This establishes BShap as a theoretically sound method for generating reliable feature attributions in this context.

3. Empirical Evaluation:

   • Small-scale examples: The authors use FinBERT and BART-Large-MNLI to demonstrate how BShap attributions align with financial intuition in sentiment analysis and sector classification tasks (e.g., distinguishing between "strong" and "stable" earnings, or identifying FinTech companies).
   • Large-scale Risk Attribution: A significant experiment analyzes the "Risk Factors" section (Item 1A) of Form 10-K filings for Silicon Valley Bank (SVB), Wells Fargo & Company (WFC), and Crocs, Inc. Using a GPT-5.2 model to score risk levels (0–10), the authors compute BShap attributions for different risk headings.
   • Validation: The study assesses the consistency of the attributions (stability of feature rankings across multiple runs) and fidelity (the extent to which removing high-attribution features significantly alters the model's output).

Key Results

• Theoretical Validity: The Baseline Shapley value is proven to satisfy the proposed axioms, ensuring that attributions respect financial principles such as monotonicity and marginal effects.
• Interpretability in Sentiment and Classification: In illustrative examples, BShap correctly identified that "strong" carries a higher positive attribution than "stable," and that "AI" drives technology sector classification while "lending" drives finance classification. It also successfully diagnosed model failures, such as a model over-classifying a hedge fund as a technology firm due to an over-reliance on the phrase "artificial intelligence."
• Risk Attribution Insights: In the large-scale experiment, BShap effectively decomposed risk scores:
  • For SVB, the highest attributions were assigned to Credit Risks and Market/Liquidity Risks, accurately reflecting the factors that led to its 2023 collapse.
  • For Wells Fargo, the method highlighted Regulatory and Legal Risks, reflecting the company's historical exposure to the fake-accounts scandal and subsequent consent orders.
  • For Crocs, the method captured relative changes in risk profiles between 2020 and 2021, such as increased Financial and Accounting Risks due to the HEYDUDE acquisition and decreased Economic Risks as pandemic-related concerns eased.
• Consistency and Fidelity: The experiments demonstrated high rank consistency across runs and high fidelity, where removing features with high attributions caused significantly larger drops in model predictions than removing low-attribution features.

Significance and Claims
The paper claims that the Baseline Shapley value is a reliable tool for interpreting LLM outputs in financial text applications because it preserves a set of axioms derived from financial domain knowledge. The significance of this work lies in bridging the gap between general explainable AI methods and the specific, high-stakes requirements of the financial sector.

The authors argue that this approach supports risk management in two ways:

1. Model Risk Management: It provides a principled, interpretable way to understand opaque LLM behaviors, ensuring predictions can be explained clearly to regulators and stakeholders.
2. Risk Decomposition: It allows for the decomposition of aggregate risk measures (derived from LLMs) into specific sources (e.g., specific risk headings in a 10-K), enabling more targeted risk mitigation strategies.

The authors remain modest regarding limitations, noting the computational complexity of Shapley values for large feature sets and suggesting that future work could explore hierarchical explanations or richer structural patterns in financial text. They do not claim to have optimized the risk measurement accuracy itself, but rather focused on the interpretability and validity of the attribution process.