📈 economics

Human-in-the-Loop Tax Auditing: Optimal Triage Between AI and Human Inspectors

This paper presents a formal economic model and large-scale synthetic experiments demonstrating that optimal tax auditing triage allocates scarce human inspection capacity based on the combined incremental value of evidence generation and superior recovery procedures, yielding welfare gains significantly higher than risk-only or uncertainty-first allocation strategies while accounting for complex factors like error dependence and behavioral deterrence.

Original authors: Kun Huang

Published 2026-10-06

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Kun Huang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Tax authorities face a problem that is as old as the concept of taxation itself: there are far more tax returns to check than there are people to check them. Governments have always had to decide which returns deserve a closer look and which can be left alone. In the past, this decision relied on human intuition or simple rules about who was most likely to be cheating. Today, however, tax agencies have powerful artificial intelligence systems that can scan millions of returns in seconds, flagging those that look suspicious. But these machines cannot do everything. They can spot patterns, but they often cannot gather the physical evidence needed to prove a crime, nor can they navigate the complex legal steps required to collect money from a stubborn taxpayer. The critical question for modern governments is no longer just how to find the bad actors, but how to decide when to stop trusting the machine and start trusting a human inspector.

This is the specific puzzle tackled by Kun Huang, an economist at Wuhan University. The research does not ask whether artificial intelligence is better than humans, a debate that often dominates headlines. Instead, it asks a more practical question: once a computer has already done its screening, what is the most valuable way to use the limited number of human inspectors who remain? The study builds a formal economic model to simulate how a tax authority should triage cases. It compares three paths: letting a case go without further action, running it through a low-cost automated check, or sending it to a human investigator. The model treats human investigation not as a magic wand that fixes everything, but as a specific tool that offers two distinct advantages. First, a human can find new evidence that the computer missed. Second, even if the computer already knows a taxpayer owes money, a human investigator might be more effective at actually collecting that money due to better negotiation skills or legal procedures.

To test these ideas, the researcher did not look at real tax records, which would be impossible to use for this kind of controlled experiment. Instead, the study created a massive synthetic world of 100,000 fake tax cases. In this digital laboratory, every detail was known: which taxpayers were actually non-compliant, how much they owed, how complex their returns were, and how likely the computer and the human inspector were to make mistakes. The computer was programmed to be fairly accurate, while the human inspector was programmed to be slightly better at finding evidence but also more expensive to employ. The simulation then ran thousands of times, testing different strategies for assigning the limited human slots. Some strategies sent the most suspicious cases to humans, while others sent the most confusing cases, or simply picked cases at random.

The results revealed that the most common intuition is often wrong. Many people assume that the best strategy is to send the cases with the highest probability of non-compliance to human inspectors. The simulation showed that this "risk-only" approach is inefficient. A case might be highly likely to be a lie, but if the computer can already resolve it easily by matching documents, sending a human inspector there is a waste of time. Conversely, a case that looks only moderately suspicious might be worth a human's attention if the computer is likely to fail at collecting the money, even if it correctly identifies the problem. The study found that the optimal strategy is to send a human to a case only when the extra value they bring—either new evidence or better collection—exceeds the value of the best alternative the computer could provide. This creates a complex map of decision-making where the best cases for humans are not always the riskiest ones, nor the most uncertain ones.

The research also uncovered surprising behaviors when the relationship between the computer and the human changes. If the human inspector makes the exact same mistakes as the computer, the human adds no value in terms of finding new information. However, the study showed that even in this scenario, the human might still be valuable if they are better at the final step of collecting the tax. Furthermore, the study demonstrated that as artificial intelligence gets smarter, the number of cases sent to humans does not necessarily go down in a straight line. Sometimes, a better computer actually helps identify a specific subset of difficult cases where a human's unique skills become even more critical, creating a situation where better technology and human labor work together rather than replacing each other.

One of the most significant findings concerns the difference between what is good for the government's budget and what is good for society as a whole. A strategy that maximizes the amount of tax collected might also impose heavy burdens on honest taxpayers, such as forcing them to spend hours preparing documents for a false alarm. The study showed that when these hidden costs are counted, the "best" strategy changes. A policy that looks like a financial win for the treasury can actually be a net loss for society if it harasses too many compliant citizens. The research also explored how the threat of an audit changes taxpayer behavior. When taxpayers know that a human might be investigating them, they are more likely to pay up voluntarily before the audit even happens. This "deterrence" effect can sometimes reverse the ranking of which cases are most important to investigate, suggesting that the value of a human inspector includes the fear they instill, not just the evidence they find.

The study concludes that there is no universal rule for when to call a human. The best decision depends entirely on the specific details of the situation: how good the computer is, how much it costs to hire an inspector, how difficult it is to collect the money, and how much burden the process places on the taxpayer. The research provides a clear framework for thinking about these trade-offs, showing that the value of a human inspector is not a fixed number but a dynamic calculation. It suggests that tax authorities should stop trying to rank cases by a single score of "risk" and instead calculate the specific economic gain of sending a human to that particular case. By doing so, they can ensure that their scarce human resources are used where they truly make a difference, rather than simply filling slots with the most obvious targets.

This work remains a simulation, a carefully constructed experiment in a digital world rather than a report from a real tax office. The numbers generated are not estimates of what will happen in any specific country, but rather a demonstration of how different economic forces interact. The study does not claim to have solved the problem of tax enforcement, nor does it offer a ready-made software package for governments to buy. Instead, it offers a rigorous way to think about the problem, stripping away assumptions to reveal the underlying mechanics of how humans and machines can work together. It shows that the future of tax auditing is not about choosing between artificial intelligence and human judgment, but about finding the precise, often counterintuitive, point where the two should meet to create the most value for everyone.

Technical Summary: Human-in-the-Loop Tax Auditing: Optimal Triage Between AI and Human Inspectors

Problem Statement
Tax administrations face a fundamental resource allocation problem: the volume of potentially auditable tax returns exceeds the capacity of human inspectors. While Artificial Intelligence (AI) can screen returns and generate risk scores, it does not resolve the question of how to allocate scarce human inspection capacity. The core challenge is that a high-risk case may be resolvable through low-cost automated document matching, while a less obvious case may require human intervention to obtain evidence or execute a recovery procedure unavailable to machines. The paper argues that allocating human resources based solely on noncompliance probabilities (risk) or uncertainty is insufficient. Instead, allocation must be an economic comparison of the incremental value of human intervention relative to the best available nonhuman alternative (release or automated verification).

Methodology
The paper develops a formal economic model and conducts large-scale synthetic computational experiments to analyze the "Human-in-the-Loop" (HITL) triage problem.

Economic Environment: The model defines three routes for a case $i$ $i$ :
1. Release (N): No further action.
2. Automated Verification (A): Low-cost processing with a specific recovery technology ( $r_A$ ) and administrative cost ( $c_A$ ).
3. Human Investigation (H): High-cost processing involving an additional binary signal ( $Z$ ) and a potentially superior recovery technology ( $r_H \ge r_A$ ).
Objectives: The model distinguishes between Fiscal Value (net tax recovery minus administrative costs) and Welfare Value (incorporating a fiscal premium $\mu$ , private compliance burdens, and the real resource costs of investigation).
Decomposition of Human Value: The value of human review ( $v$ $v$ ) is decomposed into:
- Information Value: The value of the additional evidence ( $Z$ ) provided by the human, conditional on existing machine signals.
- Procedural Value: The value derived from the human's ability to recover more tax ( $r_H > r_A$ ) or execute a more effective recovery procedure, even if the evidence is redundant.
Optimization: The authority maximizes total expected value subject to a capacity constraint ( $H$ slots). The optimal policy is a sorting rule that selects cases with the largest positive increments ( $v_i = V_{H,i} - B_{A,i}$ ), where $B_A$ is the best nonhuman route value.
Synthetic Data: The study uses a data-generating process (DGP) with 100,000 synthetic cases across 50 independent replications. It varies AI quality, human accuracy, error dependence, recovery fractions, and capacity. It explicitly avoids using real administrative data, focusing instead on "known-primitives" optimization to serve as an oracle benchmark.
Extensions: A separate commitment model is introduced to analyze how announced review allocations affect taxpayer concealment behavior and voluntary payments, distinguishing static recovery from dynamic deterrence effects.

Key Contributions and Analytical Results
The paper provides analytical characterizations and numerical results that challenge standard intuitions in tax auditing and algorithmic triage:

Optimal Triage Geometry: The optimal allocation does not necessarily follow a simple threshold on risk (posterior probability) or uncertainty (entropy).
- Intermediate-Posterior Review: In equal-recovery benchmarks, human review may be optimal for an intermediate range of posterior probabilities, not just the highest or lowest.
- Disconnected Review Regions: When procedural advantages differ, the set of cases warranting human review can be disconnected (e.g., low-risk and high-risk cases are reviewed, while medium-risk cases are automated).
Information vs. Procedure:
- Error Dependence: If human and AI errors are perfectly dependent (a "clone"), information value vanishes. However, if the human has a procedural advantage (better recovery), human review remains valuable even with zero incremental information.
- Non-Monotonicity: When marginal accuracies differ, increasing error dependence does not monotonically decrease information value; under certain conditions, high dependence can make the human signal informative in the opposite direction of the machine signal.
AI Quality and Human Demand: Improved AI quality does not universally increase or decrease the demand for human review. Better AI can reduce the need for evidence gathering but may concentrate human resources on cases where procedural recovery advantages are largest, potentially increasing the value of retained reviews while reducing the volume of reviews.
Fiscal vs. Welfare Disagreement: Fiscal-optimal and welfare-optimal allocations can rank cases differently. In a static setting with no fiscal premium ( $\mu=0$ ) and positive burdens, no costly intervention improves welfare. A positive welfare rationale requires a fiscal premium or dynamic incentive effects.
Deterrence and Commitment: In the commitment extension, ranking cases by "frozen" recovery (ignoring behavioral responses) can lead to suboptimal allocation. The marginal welfare value includes saved concealment costs and induced voluntary payments, which can reverse the ranking of cases compared to a static recovery view.

Results

Performance Gaps: In the baseline synthetic environment, the welfare-optimal triage rule outperforms "risk-only" triage by 1.662 arbitrary value units per case and "uncertainty-first" triage by 2.258 units.
Capacity: The marginal value of human capacity exhibits diminishing returns. There are scenarios where increasing capacity yields zero additional value because all positive-gain cases have already been reviewed.
Robustness: The superiority of the optimal rule holds across various parameter sweeps (e.g., different recovery distributions, error dependencies, and cost structures), though the magnitude of the gap varies.
Input Sensitivity: The study demonstrates that misspecification of key inputs (e.g., assuming constant human quality or maximal error dependence) can significantly degrade performance, sometimes making a simple "recovery-complexity" hybrid rule perform better than a misspecified optimal rule.

Significance and Claims
The paper explicitly frames its contribution as a bounded analysis rather than a universal prescription for tax authorities.

No Universal Superiority: It does not claim that human-AI combinations are universally superior or that humans are always better than AI. It identifies specific mechanisms (procedural recovery, conditional evidence) where human intervention adds value.
Oracle Benchmark: The synthetic experiments serve as a reproducible test bed to quantify the consequences of allocation rules under known primitives. It distinguishes the problem of optimizing a rule from the problem of estimating the inputs required for that rule.
Mechanism Identification: The primary contribution is the separation of "information value" from "procedural value" and the demonstration that standard heuristics (risk ranking, uncertainty ranking) fail to capture the full economic value of human intervention.
Limitations: The author acknowledges that the model relies on synthetic data and does not estimate institutional treatment effects. It does not solve the full strategic equilibrium of a dynamic tax system but isolates the marginal value of review allocation. The results are intended to guide the design of empirical studies and the measurement of specific primitives (conditional inspector performance, route-specific recovery, and taxpayer burdens) rather than to provide immediate staffing recommendations.

Technical Summary: Human-in-the-Loop Tax Auditing: Optimal Triage Between AI and Human Inspectors

More like this