📈 economics

Estimating Decision Rules from Large Language Model Agents: An Application to Bank Balance-Sheet Management

This paper demonstrates that decisions generated by large language model agents in a bank balance-sheet management simulation can be effectively approximated by simple equations, which accurately reproduce key financial metrics like return on equity and liquidity dynamics across various scenarios, though the study clarifies that these findings validate the approximation methodology rather than confirming the models' representation of real-world human behavior.

Original authors: Zlatin Sarastov

Published 2026-10-20

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Zlatin Sarastov

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Banks are the plumbing of the modern economy, moving money from savers to borrowers. When interest rates rise, as they did sharply in 2022 and 2023, the value of the long-term loans and bonds banks hold can drop, while the cost of the money they borrowed from depositors can stay low for a while. This creates a delicate balancing act. If a bank loses too much value, or if depositors suddenly panic and pull their money out, the bank can fail. To understand how this happens, regulators and bankers build computer models. These models try to predict how people will react when rates change: will a saver move their money to a better offer, or will a borrower stop taking out new loans? Traditionally, these models rely on simple mathematical formulas based on past data. But the past does not always predict the future, especially when the economic environment changes rapidly.

Researchers have recently begun using a new kind of tool to fill these gaps: large language models. These are the same powerful computer programs that can write essays, answer questions, and hold conversations. In this new approach, scientists treat these programs not as chatbots, but as simulated people. They give the computer a role, such as a bank manager or a household saver, and ask it to make decisions in a virtual economy. The goal is to see if the computer's choices can be translated back into the simple mathematical rules that regulators use to test the safety of real banks. This is not about replacing human bankers with computers, but about using the computer's ability to simulate complex behavior to build better, more realistic models of how a bank might survive a crisis.

A researcher set out to test this idea using a simulated bank in the euro area. They created a digital environment where a single bank had to make twelve monthly decisions: what interest rate to pay on savings, what margin to charge on new loans, and how to reinvest money from maturing bonds. At the same time, four groups of simulated customers—savers who are very sensitive to rates, savers who are less sensitive, and borrowers with either variable or fixed loan rates—reacted to the bank's choices and to the broader economic news. The researcher first asked a large language model to act as the bank manager and as the customers, instructing them to behave according to specific goals and constraints. The computer generated thousands of decisions across different economic scenarios, from calm years to periods of rising rates and even simulated bank runs where depositors panic.

The core challenge was that the computer's decisions were complex and varied, while the models regulators use must be simple equations. The researcher asked a fundamental question: could they take the messy, intelligent-sounding choices made by the computer and distill them into a few simple rules that a human could write down? They tried to fit the computer's behavior into standard mathematical formulas. Initially, the simple formulas did not match the computer's behavior well. The computer, acting as a bank manager, seemed to react differently to stress than the old formulas predicted. It was more cautious about lending and more aggressive about raising deposit rates when trouble appeared.

To solve this, the researcher developed a new method. They used the computer's decisions to build a new set of rules, which they then tested against fresh scenarios the computer had not seen before. They found that they could indeed create a set of simple, conditional rules that closely mimicked the computer's behavior. For example, the new rules showed that in normal times, the bank manager would keep deposit rates very close to the market average. However, if a stress signal appeared, such as a warning about the bank's liquidity, the manager would immediately raise deposit rates by a significant amount to keep customers from leaving. This rule-based approximation worked remarkably well. In ten different simulated years, the new simple rules predicted the bank's final profit with an error of less than one-third of a percentage point. They also correctly predicted how the bank's liquidity would hold up during a crisis.

The study also revealed something surprising about how the computer made its decisions. The researcher noticed that the computer's behavior depended heavily on the specific instructions they gave it. When they told the computer to act like a "rate-sensitive" depositor, it behaved that way. But when they removed those specific behavioral hints and just gave the computer the raw facts about the bank and the economy, the computer still made similar choices, though with some differences. For instance, without the specific instruction to be "insensitive," the computer's simulated households became much more likely to withdraw their money when interest rates changed. This suggests that the computer was not just following a hidden script, but was genuinely weighing the information provided. However, the researcher was careful to note that these results describe how the computer behaved under specific instructions, not how real human bankers or customers actually act.

One of the most striking findings concerned the risk of a bank run. In the simulations, if the bank's liquidity ratio—a measure of how much cash it has on hand—dropped below a certain public threshold, the simulated depositors would continue to withdraw money even after the initial bad news had passed. This created a self-sustaining panic. The researcher found that this outcome depended on how the information was presented. If the public saw a monthly snapshot of the bank's health, the panic could last for months. But if the public only saw an average of the bank's health over a year, the panic stopped much sooner. This highlights how the way regulators share information can interact with depositor behavior to either calm or worsen a crisis.

The researcher also explored the trade-off between profit and safety. They found that if the bank held longer-term fixed-rate assets, it could earn more money in calm years. However, this strategy made the bank much more vulnerable if interest rates rose sharply. In their simulations, extending the life of these assets from two to six years increased the bank's annual profit by nearly two percentage points in good times. But when interest rates rose by three percentage points, the bank's economic value—the true worth of its assets minus its debts—plummeted by hundreds of millions of euros. This confirmed a classic financial truth: higher returns in good times often come with higher risks in bad times.

Ultimately, this work demonstrates a new way to build models for the financial system. By using large language models to generate realistic decisions and then translating those decisions into simple, testable rules, researchers can create simulations that are both flexible and transparent. The study does not claim that these computer agents are perfect copies of human behavior. Instead, it shows that this method provides a powerful tool for exploring "what if" questions. It allows regulators and bankers to test how a bank might react to a crisis before it happens, using a model that is grounded in the complex, adaptive logic of modern artificial intelligence, yet simple enough to be understood and checked by humans. The results suggest that while the specific numbers depend on the instructions given to the computer, the underlying patterns of behavior—such as the tendency to raise rates during a panic or the trade-off between safety and profit—are robust and worth studying further.

Technical Summary: Estimating Decision Rules from Large Language Model Agents

Problem Statement
The paper addresses the methodological challenge of integrating Large Language Model (LLM) agents into agent-based models (ABM) for bank balance-sheet management. While LLMs can generate simulated decisions, their outputs are often opaque and dependent on prompt engineering. The core problem is determining whether LLM-generated decisions can be approximated by simple, equation-based decision rules that are reproducible and evaluable in out-of-sample scenarios. This is particularly relevant for assessing interest rate risk in the banking book (IRRBB) and liquidity dynamics, where regulatory frameworks (e.g., Basel IRRBB, LCR) rely on assumptions about depositor and borrower behavior that may shift with economic conditions. The study explicitly distinguishes between approximating LLM behavior and validating human behavior, noting that LLM responses may reflect prompt instructions rather than observed market realities.

Methodology
The author constructs a stylized, monthly leader–follower agent-based model of a euro-area bank. The model features:

Agents: One bank (leader) and four customer categories (followers): rate-sensitive depositors, rate-insensitive depositors, variable-rate borrowers, and fixed-rate borrowers.
Environment: A macroeconomic component specifying policy rates, yield curve slopes, unemployment, and stress news.
Mechanism: The bank sets deposit rates, lending margins, and bond reinvestment allocation. Customers respond based on price differences, stress indicators, and regulatory ratios (LCR, CET1). The balance sheet is updated via deterministic accounting equations.
Estimation Procedure:
1. LLM Elicitation: LLM agents (using Claude and GPT-5.5 families) are prompted to make decisions across various hypothetical states and simulated trajectories. Instructions include behavioral guidance (e.g., withdrawal limits, objective rankings) and neutral versions.
2. Equation Derivation: The LLM-generated choices are used to estimate parameters for equation-based approximations (denoted EQ0 through EQ4). This involves least-squares estimation for customer response coefficients and grid searches for bank preference parameters.
3. Conditional Rules: For complex decisions like bond reinvestment, conditional rules are derived directly from LLM choices rather than a global optimization function.
4. Evaluation: The estimated equations are tested against separate "out-of-sample" LLM simulations (held-out scenarios and different model families). Success is measured against seven criteria (V1–V7), including accounting identity, calibration targets, directional responses, and fit to LLM decisions (ROE gap, balance gap, and decision discrepancy $G$ ).

Key Contributions
The primary contribution is methodological: the paper establishes a procedure for estimating, approximating, and evaluating LLM-generated decision rules within a defined simulation environment.

Procedure: It outlines a loop of elicitation, estimation, and out-of-sample validation to determine when an equation-based specification adequately captures LLM behavior.
Instruction Sensitivity: It demonstrates that LLM responses are highly sensitive to prompt instructions. Removing behavioral guidance (e.g., specific withdrawal limits or objective rankings) significantly alters estimated coefficients, particularly for fixed-rate borrower sensitivity and withdrawal magnitudes.
Conditional Approximation: The study shows that simple, conditional rules (e.g., pricing near market in normal times, applying specific premiums during liquidity stress) often approximate LLM decisions better than a global optimization benchmark or simple regression models.

Results

Fit Performance: The final specification (EQ4), which incorporates conditional pricing and reinvestment rules, reproduces annual Return on Equity (ROE) within 0.31 percentage points in 7 out of 10 out-of-sample Claude bank-year simulations. It meets all fit criteria in these 7 cases. In 18 simulations using a second LLM family (GPT-5.5), the ROE difference remains within 0.71 percentage points, though only 6 meet all criteria.
Decision Discrepancy: EQ4 outperforms simpler specifications (market-pricing heuristics, direct regression, nearest-neighbor imitation) in terms of decision-level discrepancy ( $G$ ) and balance fit, particularly in stress scenarios involving deposit runs.
Instruction Dependence: Under neutral instructions, withdrawal magnitudes and fixed-rate borrower price sensitivity change materially. Specifically, a two-month withdrawal episode that becomes self-sustaining under original instructions (due to high withdrawal magnitudes) ceases to be self-sustaining when customer equations are estimated under neutral instructions.
Economic Trade-offs: The model illustrates a trade-off between earnings and economic value. Extending fixed-rate asset duration from two to six years increases calm-year ROE by ~1.9 percentage points but reduces economic net worth by EUR 516 million following a +300 basis point rate shock.
Liquidity Dynamics: The persistence of deposit runs in the simulation is conditional on the disclosure frequency of the Liquidity Coverage Ratio (LCR). Monthly disclosure of LCR below 100% triggers sustained withdrawals, whereas quarterly or averaged disclosures allow the stress response to subside once news ends.

Significance and Claims
The paper modestly claims that its results demonstrate a procedure for approximating LLM decisions, not that these decisions represent actual human behavior.

Not Human Validation: The author explicitly states that the study does not establish that LLM responses represent the behavior of bank managers or customers. The findings are conditional on the specific instructions and model architecture used.
Complementary Tool: The LLM-based estimation approach is presented as a complement to, not a replacement for, empirical calibration on observed transactions or incentivized human experiments. It allows for the generation of decision-level data to test model sensitivity and specification.
Policy Relevance: The study suggests that regulatory assessments of liquidity and interest rate risk should consider the interaction between disclosure rules (e.g., frequency of LCR publication) and depositor response functions, as these interactions can determine whether a stress event becomes self-sustaining. However, the specific thresholds identified are properties of the model and the specific LLM instructions, not empirically validated facts about the banking sector.

In summary, the paper provides a rigorous framework for translating LLM agent behavior into tractable mathematical rules for simulation, highlighting the critical role of prompt engineering in determining the resulting economic dynamics.

More like this