When Validation Defines Abnormality: Prediction Targets in Bayesian Accrual Measurement

In the world of accounting, numbers tell a story about how a business is running, but not every number is a direct observation. Some figures are estimates, calculated to separate the routine from the unusual. Imagine a company's total financial activity as a river. Most of the water flows predictably, driven by the standard currents of sales, costs, and investments. This predictable flow is what experts call "normal" behavior. However, sometimes the river swells or recedes in ways that don't match the current. These deviations are called "abnormal" activities. In financial research, these abnormal deviations are often treated as clues. They might suggest that a company is hiding something, manipulating its reports, or facing unique economic shocks that standard models cannot explain. To find these clues, researchers first build a model to predict what the normal flow should look like. Whatever is left over after that prediction is the "residual," the leftover piece that represents the abnormality.

The problem is that the definition of "normal" is not fixed. Different researchers use different formulas to calculate what a company's normal behavior should be. Some formulas focus on how profits relate to cash flow, while others look at how assets change over time. Because there is no single correct formula, researchers often try many different models and let the data decide which one works best at predicting the future. They use a method called validation, where they test a model on past data to see if it can accurately guess what happened next. If a model predicts well, it gets a higher score and is trusted more. But this process of choosing the best model relies on a specific rule: what exactly are we trying to predict? Are we trying to predict the next year for a company we already know, or are we trying to predict the future for a completely new company we have never seen before? This seemingly small choice about who we are predicting for turns out to change the entire definition of what is considered "abnormal."

A recent study by Quan Pham at the Industrial University of Ho Chi Minh City investigates this exact question. The researcher looked at thousands of annual financial reports from 568 companies listed on Vietnamese stock exchanges between 2015 and 2024. The goal was to see if the rule used to test the prediction models actually changes the final measurement of abnormal behavior. To do this, the study used a sophisticated technique called Bayesian stacking, which combines many different accounting models into a single, weighted average. Instead of picking just one "best" model, this method assigns a weight to each model based on how well it performed during testing. The key experiment involved running the same tests twice, but changing only the rule for what was hidden during the test. In the first scenario, the test hid just one year of data for a specific company while keeping all its other years visible. This simulates predicting the next step for a familiar friend. In the second scenario, the test hid every single year of data for a specific company, simulating a prediction for a stranger with no history.

The results showed a dramatic shift in how the models were weighted depending on which rule was used. When the test allowed the model to see a company's past history (predicting for a familiar friend), the system heavily favored models that included a specific "firm effect." This is a statistical feature that captures the unique, persistent quirks of a single company, like its specific management style or long-term business structure. In this scenario, these firm-specific models received about 59 to 62 percent of the total weight. However, when the test hid the entire company's history (predicting for a stranger), the system almost completely ignored these firm-specific models. Instead, it shifted its trust to general, population-level models that describe how the average company behaves. In this case, the weight for firm-specific models dropped to less than 4 percent. The study confirmed through computer simulations that this shift was not random noise or a flaw in the math. It was a direct response to the data structure: when a company's unique history is available, the best predictor uses it; when that history is gone, the best predictor relies on the average.

This reallocation of trust had a direct and measurable impact on the final numbers. The study found that while the overall ranking of companies remained somewhat similar—companies that were high on the list for one method were generally high on the list for the other—the specific companies identified as the most extreme outliers changed significantly. When looking at the top five percent of companies with the most "abnormal" activity, the two methods agreed on only about 65 to 69 percent of the cases. This means that roughly one out of every three companies flagged as highly abnormal by one method was not flagged by the other. The difference became even more pronounced when the researchers adjusted the numbers to account for the uncertainty in the predictions. In that case, the agreement on the top five percent dropped to between 37 and 42 percent. Essentially, the choice of validation rule acted like a filter that let different sets of companies through the door.

The study also checked whether these different sets of companies were actually different in their real-world behavior. It found that the companies identified as abnormal only by the "familiar friend" method tended to have lower future earnings and returns on assets. In contrast, the companies identified as abnormal only by the "stranger" method tended to have higher future cash flows and earnings. This suggests that the two methods are not just labeling the same companies differently; they are actually capturing different types of economic realities. One method is tuned to spot deviations from a company's own historical pattern, while the other is tuned to spot deviations from the general market average.

The conclusion is that the tool used to measure abnormality is not neutral. The rule chosen to test the prediction model becomes part of the measurement itself. If a researcher wants to know if a company is behaving unusually compared to its own past, they must use a validation rule that keeps the company's history visible. If they want to know if a company is unusual compared to the rest of the world, they must use a rule that hides that history. The study does not declare one method superior to the other. Instead, it shows that the definition of "abnormal" is flexible and depends entirely on the question being asked. For anyone using these measurements to screen for risk, select samples for research, or investigate financial reporting, the choice of validation rule is not just a technical detail. It is a fundamental part of the measurement design that determines which companies will be seen as outliers and which will be seen as normal.