🤖 AI

Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflows

This paper introduces FinWorkBench (Finch), a comprehensive benchmark derived from authentic enterprise data that evaluates frontier AI agents on complex, multimodal finance and accounting workflows, revealing significant performance gaps where even advanced models like GPT 5.1 Pro achieve only a 38.4% success rate.

Original authors: Haoyu Dong, Pengkun Zhang, Yan Gao, Xuanyu Dong, Yilin Cheng, Mingzhe Lu, Zikun Zhu, Adina Yakefu, Shuxin Zheng

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoyu Dong, Pengkun Zhang, Yan Gao, Xuanyu Dong, Yilin Cheng, Mingzhe Lu, Zikun Zhu, Adina Yakefu, Shuxin Zheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to do your job as a financial analyst. You might think, "No problem! Just tell it to 'add up the numbers' or 'make a chart.'"

But in the real world of finance, it's not that simple. It's less like solving a math problem on a clean whiteboard and more like trying to fix a leaky roof while standing on a wobbly ladder, in the rain, holding a map that's been crumpled and taped together, and the map is written in three different languages.

That is exactly what the paper FINCH is about.

Here is the breakdown of the paper in plain English, using some everyday analogies.

1. The Problem: The "Messy Kitchen" of Finance

Most AI tests today are like asking a robot to bake a cake using a perfect, pre-measured recipe in a pristine kitchen. The ingredients are in neat bowls, the instructions are clear, and the oven is set to the right temperature.

But real finance work is a messy kitchen.

The Ingredients are scattered: Data isn't in one neat list. It's in 50 different Excel files, some PDFs, some emails, and some charts.
The Recipe is broken: The instructions are hidden inside old emails ("Hey, can you update the 2002 budget?") or buried in the history of how a spreadsheet changed over time.
The tools are weird: The "tables" aren't perfect grids. They have merged cells, weird fonts, charts embedded in the middle of text, and formulas that look like secret codes.
The job is long: You don't just "add numbers." You have to find the data, clean it, check if it makes sense, calculate new projections, draw a graph, and write a report. It's a long chain of steps where one mistake ruins the whole thing.

2. The Solution: FINCH (The "Real-World Gym")

The authors created a new benchmark called FINCH (Finance & Accounting Benchmark). Think of this as a gym for AI robots, but instead of lifting weights, they have to navigate a messy office.

Where did they get the data? They didn't make up fake problems. They dug into real, historical archives (like the famous Enron emails and real bank documents) from 2000 to 2025. They found 172 real-life tasks that actual humans had to do.
How did they build it? They used a mix of AI and human experts. Imagine a team of detectives looking at an old email thread and a stack of spreadsheets, trying to figure out: "What was the human trying to do here? What steps did they take? What was the final result?" Then, they wrote a "test" based on that story.
The Scale: This isn't a small test. It involves 1,710 spreadsheets and 27 million cells. It's a massive, chaotic library of financial data.

3. The Test: Can the Robots Do It?

The authors took the smartest AI robots available today (like GPT-5, Claude, and Gemini) and threw them into this messy kitchen.

The Results were shocking:
Even the "super-brains" failed most of the time.

The Pass Rate: The best AI only got about 38% of the tasks right. That means they failed more than 6 out of 10 times.
The Time: It took the AI an average of 17 minutes to try to solve one task, and they still got it wrong.
The "Long Chain" Problem: If a task had just one or two steps, the AI did okay. But if the task had three or more steps (like: Find data -> Clean it -> Calculate -> Draw Chart), the AI's success rate plummeted. It's like a robot that can tie one shoe but forgets how to tie the second one, or gets confused by the third.

4. Why Do They Fail? (The "Gotchas")

The paper found five main reasons why the robots are struggling:

The "Where is it?" Problem: The data is spread across 10 different files. The AI often looks in the wrong file or grabs the wrong row of numbers. It's like looking for a specific ingredient in a pantry with 1,000 unlabeled jars.
The "Secret Code" Problem: In finance, the formula in a cell is often more important than the number you see. The AI sees the number "100" but misses the hidden logic that says, "This 100 is actually a 55-day payment schedule." It ignores the instructions behind the curtain.
The "Messy Table" Problem: Real spreadsheets are ugly. They have merged cells, blank rows, and weird headers. The AI gets confused by the layout and thinks a header is a data point, or misses a column entirely.
The "Multimodal" Problem: Sometimes the data is in a PDF, sometimes in a picture of a chart, and sometimes in an email. The AI struggles to combine these different formats into one coherent picture.
The "Memory" Problem: Because the tasks are so long, the AI forgets what it did in step 1 by the time it gets to step 10. It loses the thread of the story.

5. The Big Takeaway

The paper concludes that while AI is amazing at writing poems or coding simple scripts, it is not yet ready to replace a human financial analyst.

Real finance work is too messy, too long, and too full of hidden context for current robots to handle reliably. The AI is like a brilliant student who can ace a multiple-choice quiz but freezes when asked to fix a real-world plumbing leak.

The Future:
The authors hope FINCH will become the standard "driving test" for financial AI. Until AI can pass this test, we shouldn't trust it to manage our money, our budgets, or our taxes without a human looking over its shoulder.

In short: FINCH is a reality check. It shows us that the "messy middle" of real-world work is still very hard for AI to navigate.

1. Problem Statement

Current frontier AI agents (LLMs) are increasingly integrated into enterprise workflows but face significant challenges when applied to real-world Finance & Accounting (F&A) tasks. Existing benchmarks often rely on clean, isolated tables or simple QA pairs, failing to capture the intrinsic complexity of professional work.

Real-world F&A workflows are characterized by:

Messiness: Artifacts are heterogeneous (spreadsheets, PDFs, images, emails), structurally irregular (merged cells, nested headers, inconsistent formatting), and contain cryptic business logic.
Long-Horizon Dependencies: Tasks require multi-step reasoning spanning data entry, structuring, calculation, modeling, validation, and reporting.
Multimodality: Workflows interleave text, tables, formulas, charts, and code.
Collaboration & Evolution: Work evolves through version histories and email threads, requiring agents to infer intent from partial or changing contexts.

The core question addressed is: Can current AI agents handle the messy, long-horizon, and knowledge-intensive workflows found in actual enterprise environments?

2. Methodology

A. Dataset Construction (FINCH)

The authors introduce FINCH (FinWorkBench), a benchmark derived from authentic enterprise environments rather than synthetic data.

Sources:
- Enron Email Corpus: ~15,000 spreadsheet files and 500,000 emails from 150 executives/employees.
- EUSES Corpus: ~450 financial spreadsheet files.
- External Institutions: Reports from the World Bank, Canadian/British governments, Adidas, and various investment firms (2000–2025).
Workflow Derivation Pipeline:
1. Email Mining: LLMs identify collaborative threads with business goals and attached artifacts. Human experts normalize these into task instructions.
2. Version History Analysis: LLMs analyze diffs between spreadsheet versions to infer underlying workflows (e.g., error correction, assumption updates). Experts validate and refine these into precise task instructions.
3. Deliverable Curation: High-quality reports and models are reverse-engineered into input/output pairs.
Annotation: Over 700 hours of expert effort by a team of finance/CS professionals.
- Scale: 172 composite workflows, 384 tasks, 1,710 spreadsheets (27 million cells), plus PDFs and images.
- Complexity: 78.5% of workflows involve multiple interleaved tasks (e.g., data entry $\to$ cross-sheet retrieval $\to$ modeling $\to$ visualization).

B. Evaluation Framework

FINCH employs a dual-evaluation strategy to handle the difficulty of assessing complex spreadsheet outputs:

Human Evaluation (Gold Standard): Experts verify if the model satisfactorily fulfills the instruction, checking for structural correctness, semantic fidelity, and absence of critical errors.
LLM-as-Judge (Automated): A scalable pipeline using a multimodal LLM (GPT-5-mini) to judge outputs.
- Technique: Uses structured diffs (diff_ref vs. diff_model) and compact snapshots (retaining headers and edited cells) to reduce token usage while preserving context.
- Rubrics: Evaluates Completeness, Correctness (numerical/logical), Over-edit Avoidance, and Readability.

C. Agents Evaluated

Product-side Agents: ChatGPT (GPT-5.1 Pro) and Claude (Sonnet/Opus 4.5) in interactive modes with web browsing.
API-based Models: GPT-5.1, Claude Sonnet 4.5, Grok 4, Qwen 3 Max, and Gemini 3 Pro. These were adapted using a code-generation paradigm (generating Python scripts via openpyxl, pandas) to manipulate spreadsheets directly, extending the SpreadsheetBench framework.

3. Key Contributions

First Real-World F&A Benchmark: FINCH is the first benchmark sourced entirely from authentic enterprise artifacts (emails, versioned spreadsheets, reports), capturing the "in-the-wild" messiness of finance work.
Novel Workflow Construction: A pipeline combining LLM-assisted mining of email/version histories with rigorous expert annotation to derive composite, multi-step workflows.
Multimodal & Long-Horizon Focus: Unlike prior benchmarks focusing on single-table reasoning, FINCH requires agents to navigate 1,710+ interconnected files, handle PDFs/images, and execute long chains of reasoning (median 14 tool calls per task).
Comprehensive Evaluation Protocol: A hybrid human/automated evaluation system capable of assessing structural integrity and semantic logic in complex, multi-sheet environments.

4. Experimental Results

Overall Performance: Even the strongest frontier agents fail to master real-world workflows.
- GPT-5.1 Pro: Achieves only 38.4% pass rate (Human Eval) despite averaging 16.8 minutes per workflow.
- Claude Sonnet 4.5: Achieves 25.0% pass rate.
- API-based Models: Generally underperform product agents due to the lack of iterative feedback loops (single-shot generation).
Key Failure Modes:
- Long-Horizon Composition: Performance drops sharply as task count increases. GPT-5.1 Pro drops from 44.3% (≤2 tasks) to 23.5% (>2 tasks) due to error accumulation.
- Messy Structures: Data entry and structuring tasks are particularly difficult due to irregular layouts and merged cells.
- Multimodal Challenges: Workflows involving PDFs/Images see a pass rate drop to 35.0% for GPT-5.1 Pro.
- Formula Reasoning: Models struggle to reconstruct latent business logic encoded in formulas, often misinterpreting column semantics based on headers alone.
Error Analysis:
- 35% of failures stem from formula reasoning errors (misinterpreting logic).
- 25% are data retrieval errors (wrong rows/columns/sheets).
- 25% are code generation errors (syntax/layout mismatches).
- 10% are task misunderstandings (ignoring implicit context).

5. Significance and Conclusion

Gap Identification: The results reveal a substantial gap between current AI capabilities and the demands of professional F&A work. The combination of messy inputs, long-horizon dependencies, and heterogeneous formats creates a "perfect storm" that current models cannot reliably navigate.
Beyond Single-Task Benchmarks: FINCH demonstrates that success on isolated tasks does not translate to success in composite, real-world workflows. The "messiness" of enterprise data is a primary driver of failure.
Future Directions: The paper argues that progress requires training and evaluation on long-running, computation-intensive workflows over large, multimodal artifacts. It highlights the need for agents capable of iterative self-correction, robust cross-file navigation, and deep formula-level reasoning.

Conclusion: FINCH establishes a rigorous, realistic standard for evaluating AI in finance, showing that while frontier models are powerful, they are not yet ready to autonomously handle the complex, messy, and collaborative nature of enterprise financial workflows.