Forecasting AI Efficiency During Engagement Delivery: A Simulation Study of Estimate-to-Complete Methods in Accounting Services

Accounting firms are currently navigating a significant shift as they integrate artificial intelligence into their daily work. For decades, these firms have relied on detailed budgets to predict how long tasks will take and how much they will cost, setting fees based on those predictions. The arrival of AI promises to speed up routine work, but the reality is messy. The speed gains are not uniform; some tasks shrink dramatically while others barely change, and the full benefits often only appear after staff have spent time learning how to use the new tools effectively. This creates a difficult planning problem: a budget written at the start of a project might be wildly optimistic or pessimistic, and the error might not become obvious until the work is nearly finished. To manage this, firms need a way to check their progress while the work is still happening, adjusting their final cost predictions based on what is actually occurring rather than what was hoped for at the beginning.

A recent study by Ali Cetin tackles this exact challenge by simulating thousands of accounting engagements to see which method works best for updating these cost predictions. The research focuses on a specific process called "estimate to complete," which is simply the act of looking at the work done so far and the work that remains, then calculating a new, more accurate total cost for the project. The author built a computer model that mimics the behavior of a typical twelve-month accounting engagement, complete with eight different types of tasks and three levels of staff. In this virtual world, the AI's efficiency was allowed to vary, staff members were programmed to learn and get faster over time, and the volume of work was allowed to fluctuate. By running this simulation ten thousand times, the study could test six different ways of making these mid-project predictions to see which one kept the firm's financial forecasts closest to reality.

The simulations revealed that sticking to the original plan is a poor strategy. When the researchers relied solely on the initial budget without updating it, the final cost predictions were often off by a significant margin. However, when the team updated their forecasts monthly based on actual performance, the accuracy improved dramatically. By the sixth month of a project, the best updating methods cut the prediction error in half compared to just trusting the original plan. This finding underscores a critical point: because the benefits of AI are uncertain and staff need time to learn, a budget set at the start is rarely enough. Continuous monitoring is not just helpful; it is necessary to avoid missing financial targets.

The study also discovered that there is no single "best" method for every stage of a project. The most accurate approach depends heavily on whether the staff are still in the learning phase. In the early months, when employees are just getting used to the new tools and their speed is still improving, the most reliable method was a balanced approach that split the difference between the original plan and the performance of the most recent month. This blend prevented the forecast from overreacting to the slow start that often happens when learning a new system. As the engagement progressed and the staff's performance stabilized, the best method shifted. Once the learning curve flattened out, a method that looked at the average performance over the last three months became the most accurate, smoothing out random daily fluctuations while still reflecting the current reality.

Another key finding emerged when the nature of the work itself changed. In the simulation, the researchers introduced a scenario where the volume of two specific tasks suddenly increased by thirty percent. When the mix of work changed, a forecast that looked at the project as a single, big block of work failed to capture the new reality. It was only when the forecast was broken down by individual activity—calculating the time for each specific task separately—that the prediction remained accurate. This suggests that while a high-level view might work when everything is steady, firms must look at the details of their work if the scope shifts, because different tasks benefit from AI at different rates.

The research also highlighted a common trap in planning: optimism. The simulations showed that when firms set their budgets based on ideal AI performance, they frequently missed their profit targets. In the base scenario, nearly seventy-four percent of the simulated projects ended up with a lower profit margin than planned. This happened because the initial plans often underestimated the time required for staff to learn the tools and the extra review effort sometimes needed for AI-generated work. The study suggests that firms should expect these shortfalls and use their monthly reviews to catch them early, rather than assuming the initial plan will hold true.

Ultimately, the paper offers a practical, staged guide for firms navigating this new landscape. It suggests that during the first three months of an engagement, when learning is still happening, teams should use a blended forecast that weighs the original plan and recent performance equally. Once the team has settled in, usually around the fourth month, they should switch to tracking the average performance of the last three months. If the volume or type of work changes, the team must switch to a detailed, task-by-task forecast rather than a broad overview. These rules provide a simple, robust way for firms of any size to manage the uncertainty of artificial intelligence, ensuring that their financial planning remains grounded in the reality of the work as it is actually being done.