Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents

This paper introduces the Determinism-Faithfulness Assurance Harness (DFAH), a framework and set of financial benchmarks demonstrating that decision determinism and task accuracy in LLM agents are uncorrelated, thereby necessitating independent measurement to ensure reliable regulatory audit replay in financial services.

The Gist

Imagine you work at a bank, and you've hired a team of super-smart AI assistants to help you decide which transactions are safe and which ones look suspicious. These AI agents are like junior analysts: they look at data, ask questions, and make a final call.

But there's a huge problem. In the world of banking, if a regulator (the "boss" who checks your work) asks, "Show me exactly how you decided to flag this transaction," you have to be able to hit "replay" and get the exact same answer every single time.

This paper, written by Raffi Khatchadourian from IBM, introduces a new tool called DFAH (Determinism-Faithfulness Assurance Harness). Think of DFAH as a rigorous "Replay Test" lab for these AI agents.

Here is the breakdown of what they found, using simple analogies:

1. The Two Big Rules of the Game

The paper says that for an AI to be safe for banking, it needs to pass two very different tests:

• The "Replay" Test (Determinism): If you ask the AI the same question 10 times, does it give you the exact same answer 10 times?
• The "Truth" Test (Faithfulness/Accuracy): Is the answer actually correct? Did it base its decision on real facts, or did it just make things up?

The Big Surprise: The researchers found that these two things do not go hand-in-hand.

• You can have an AI that is perfectly consistent (it gives the same answer every time) but is completely wrong (it's consistently wrong).
• You can have an AI that is very smart (it gets the right answer often) but is unpredictable (sometimes it says "Yes," sometimes "No," even when the facts are the same).

The Analogy: Imagine a broken clock.

• High Determinism, Low Accuracy: A clock that is stuck at 4:00 PM. It is 100% consistent (it always says 4:00), but it is wrong 23 hours a day.
• Low Determinism, High Accuracy: A clock that is running perfectly but has a loose second hand that jumps around. It tells the right time on average, but if you check it twice in a row, the seconds might look different.

2. The "Small vs. Big" AI Showdown

The researchers tested different sizes of AI models (from small, 7-billion-parameter models to massive, "frontier" models).

• The Small, Rigid Models (Tier 1): Think of these as robotic assembly line workers. They are very strict. If you give them a task, they follow a rigid script.
  • Result: They are incredibly consistent (94–100% replayable). But because they are so rigid, they often miss the nuance and get the answer wrong (only 20–40% accuracy). They are like a robot that always says "Investigate" no matter what the transaction is, just because that's the script.
• The Big, Smart Models (Frontier Models): Think of these as creative consultants. They think deeply, try different approaches, and use many tools to solve a problem.
  • Result: They are much smarter and get the right answer more often. But because they are creative, they take different "paths" to get there. Sometimes they check Tool A first, then Tool B; other times it's Tool B then Tool A. This makes them unpredictable for a strict audit. They might give the right answer, but they can't prove exactly how they got there every single time.

The Catch: No model was found that was both a "perfect robot" (100% consistent) AND a "perfect genius" (100% accurate). You have to choose your trade-off.

3. Why This Matters for Banks

In the movie The Big Short, a bank might get in trouble not because they made a mistake, but because they couldn't explain why they made a decision.

• The Regulatory Nightmare: If a regulator asks, "Why did you reject this loan?" and the AI says, "Well, last time I ran this, I said yes, but today I said no because I felt like it," the bank is in trouble.
• The Solution: The paper suggests that for high-stakes, rule-based tasks (like checking for money laundering), banks should use the small, rigid models. Even if they aren't perfect geniuses, their "stuck clock" consistency means you can prove to the regulator exactly what happened.
• The Human Safety Net: For the "creative" big models, you can't trust them to work alone. You need a human to sit between the AI and the final decision to ensure the AI didn't just "hallucinate" a reason.

4. The "Pass/Fail" Test

The paper introduces a new way of thinking about success:

• Pass@k (The Optimist's View): "If I try 10 times, will I get at least one right answer?" (Good for coding, bad for banking).
• Passk (The Regulator's View): "If I try 10 times, will I get the exact same right answer 10 times?" (This is what banks need).

The Bottom Line

This paper is a warning label for the financial industry. It says: "Don't just look at how smart your AI is. Look at how consistent it is."

If you are building an AI for a bank, you can't just pick the "smartest" model available. You have to pick the one that can be "replayed" perfectly, even if it means accepting a model that is a bit less "creative." The DFAH tool is the new ruler they built to measure this consistency, ensuring that when the regulators come knocking, the bank can hit "replay" and show them the exact same story every time.

Technical Summary

Here is a detailed technical summary of the paper "Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents."

1. Problem Statement

Large Language Model (LLM) agents are increasingly deployed in financial services for tasks like compliance triage, portfolio rebalancing, and regulatory reporting. However, a critical gap exists in regulatory auditability. When regulators demand that a system reproduce a specific decision (e.g., "Why was this transaction flagged?") using identical inputs, most current agent deployments fail to return consistent results.

The core problem is twofold:

1. Lack of Determinism: Agents often produce different tool-call sequences (trajectories) or even different final decisions when re-run with the same input, making audit replay impossible.
2. Misconception of Correlation: There is a prevailing assumption that higher model capability (accuracy) correlates with better reproducibility. The paper challenges this, positing that determinism and accuracy are distinct dimensions that must be measured independently.

2. Methodology: The DFAH Framework

The authors introduce the Determinism-Faithfulness Assurance Harness (DFAH), a framework designed to measure and guarantee reproducibility in tool-using agents.

A. Formal Definitions

The framework defines three key metrics based on $N$ independent runs of an agent on input $q$:

• Action Determinism: The fraction of runs with identical tool sequences.
• Signature Determinism: The fraction of runs with identical tool sequences and arguments (stricter).
• Decision Determinism: The fraction of runs with identical final decisions.
• Evidence-Conditioned Faithfulness: A metric measuring whether decisions are grounded in retrieved evidence rather than hallucinated reasoning. This uses a lexical/semantic heuristic (Jaccard similarity on noun phrases/numbers) rather than an LLM-as-a-judge to avoid recursive non-determinism.

B. Evaluation Protocol

• Pass@k vs. Pass$_k$: The paper distinguishes between Pass@k (at least one success in $k$ trials, common in coding) and Pass$_k$ (all $k$ trials succeed). For financial compliance, Pass$_k$ is the required metric because every single historical decision must be reproducible.
• Stress Testing: The harness evaluates agents under four stress scenarios:
  1. Redeploy Perturbation: Fresh agent instance after restart.
  2. Data Quality Fault Injection: 10% NULL/NaN/outlier values.
  3. Temporal Shift: Using stale data (6 months prior).
  4. Market Shock: Simulating extreme volatility ($\pm3\sigma$).
• Benchmarks: Three financial benchmarks with ground-truth labels were used (50 cases each):
  1. Compliance Triage: Escalate/Dismiss/Investigate decisions.
  2. Portfolio Constraint: Approve/Reject/Modify trades based on limits.
  3. DataOps Exception: Auto-fix/Escalate/Quarantine data pipeline errors.

C. Experimental Setup

• Scale: 4,700+ agentic runs across 7 models (4 providers: Anthropic, Google, OpenAI, local Ollama).
• Models: Ranged from small local models (7–20B parameters, e.g., Qwen2.5, Granite) to frontier API models (Claude Opus/Sonnet, Gemini 2.0/2.5).
• Conditions: Runs were conducted at Temperature $T=0.0$ to isolate model non-determinism from sampling noise.

3. Key Contributions

1. DFAH Framework: The first open-source infrastructure to formally measure trajectory determinism, decision determinism, and evidence-grounded faithfulness specifically for financial agents.
2. Null Correlation Discovery: Empirical evidence showing no detectable correlation between decision determinism and task accuracy ($r = -0.11, p = 0.63$). Models can be deterministic but inaccurate, or accurate but non-deterministic.
3. Model Tier Characterization: Identification of distinct operational profiles:
   • Small Models (7–20B): Achieve near-perfect determinism (94–100%) via rigid pattern matching but suffer from low accuracy (20–42%).
   • Frontier Models: Show moderate determinism (50–96%) with variable accuracy. They explore diverse tool paths to reach decisions, leading to the "Same Conclusion, Different Reasoning" phenomenon.
4. Architectural Insight: Validation that Schema-First architectures (structured JSON outputs) significantly improve determinism compared to unconstrained ReAct-style agents without sacrificing task performance.

4. Key Results

• Determinism vs. Accuracy Trade-off: No model achieved both perfect determinism and high accuracy.
  • Example: Qwen2.5-7B achieved 100% decision determinism on Compliance Triage but only 38% accuracy (often defaulting to "investigate").
  • Example: Claude Opus-4.5 achieved 68.7% accuracy but only 72% decision determinism, with tool paths varying significantly between runs.
• The "Same Conclusion, Different Reasoning" Gap: Frontier models often converged on the correct final decision but explored vastly different tool-call sequences (Signature Determinism was often 20–58% lower than Decision Determinism). This poses a risk for audits requiring full trajectory replay.
• Stress Test Findings: Tier 1 models (small, local) maintained near-perfect determinism under data faults and redeployment, whereas frontier models showed higher variance.
• Validation Scaling: To achieve equivalent statistical reliability, Tier 3 models (120B+) require 3.7x more validation samples than Tier 1 models, making them economically impractical for strict compliance validation.

5. Significance and Implications

• Regulatory Compliance: The paper argues that in regulated environments, determinism is a gating property. An agent with 75% accuracy but 50% determinism is unusable for compliance because its behavior cannot be audited or explained consistently.
• Deployment Strategy:
  • Tier 1 (7–20B) + Schema-First: Recommended for audit-critical tasks (AML, regulatory reporting) where reproducibility is paramount.
  • Frontier Models: Recommended only for human-in-the-loop advisory roles where superior reasoning justifies the variance, provided a human acts as the consistency guarantee.
• Industry Standard Alignment: The framework aligns with emerging industry terminology (Trial, Transcript, Grader) and addresses the specific needs of the EU AI Act and US banking supervisory guidance regarding high-risk system auditability.

Conclusion

The paper concludes that single-metric evaluation is insufficient for financial AI. Institutions must measure determinism and accuracy independently. The "Determinism-Faithfulness Assurance Harness" provides the necessary infrastructure to ensure that financial agents are not only capable but also replayable, a prerequisite for regulatory approval. The findings suggest a shift toward using smaller, task-optimized models with structured outputs for compliance-critical workflows, reserving larger, more variable models for non-audited advisory tasks.