Financial Audit Assistance using Misinformation Detection and Explanation

Technical Summary: Financial Audit Assistance using Misinformation Detection and Explanation

Problem Statement
Financial auditing is a complex, knowledge-intensive discipline aimed at ensuring the integrity, accuracy, and fairness of published Financial Statements (FS), including the Balance Sheet (BS), Income Statement (IS), and Cash-flow Statement (CS). However, companies often have incentives to conceal, omit, or falsify information to misrepresent their financial health (e.g., to reduce tax liabilities or boost investor confidence). Traditional auditing relies heavily on manual evidence collection and deep domain expertise, making the process time-consuming and subjective. Furthermore, ground-truth labels indicating whether a specific FS contains misinformation are rarely available, rendering standard supervised learning approaches difficult to apply. The paper addresses the need for an AI-assisted system that can automatically detect instances of misinformation in FS and generate human-understandable explanations for the detected anomalies to assist auditors.

Methodology
The proposed approach integrates unsupervised techniques with Natural Language Processing (NLP) to create a scalable audit assistance system. The methodology is structured into three primary phases:

1. Data Preparation and Silver Labeling:

   • Dataset: The system utilizes a large corpus of semi-structured FS from 2,292 Indian listed companies over five years (2010–2014), totaling 11,460 records. Each record includes 42 financial variables (extracted from BS, IS, and CS) and 22 financial ratios.
   • Silver Label Generation: Since ground-truth labels for misinformation are unavailable, the authors generate "silver labels" by analyzing associated audit reports. They employ two models—a T5-based encoder-decoder and a MISTRAL-7B-Instruct Large Language Model (LLM)—to classify sentences in audit reports as "Adverse Remarks" (indicating potential non-conformance or fraud) or "NA."
   • Labeling Logic: A company-year tuple is assigned a "MISINFO" label if its audit report contains at least two sentences classified as adverse remarks by both the T5 and MISTRAL models. This process resulted in a dataset where approximately 19.8% of the records were labeled as containing misinformation.

2. Misinformation Detection (Unsupervised Techniques):

   • Anomaly Detection (AD): The authors apply 17 unsupervised anomaly detection algorithms (including COF, LOF, iForest, and Mahalanobis distance) implemented in the PyOD package. An ensemble method selects the top 20 companies flagged by the highest number of algorithms.
   • Step-wise Regression: The system builds regression models (OLS, Lasso, Ridge, SVR, Random Forest) to predict "susceptible" financial variables (e.g., liabilities) based on other correlated variables. Companies with the highest squared prediction errors are flagged as suspicious. Support Vector Regression (SVR) models demonstrated the highest precision in this category.

3. Explanation Generation:

   • Objective: To identify the specific financial variables responsible for a company being flagged as anomalous.
   • Silver Standard Explanations: Instead of manual validation, the authors construct "silver standard" explanations by mapping the "Adverse Remark" sentences in audit reports to financial variables. This involves assigning XBRL (eXtensible Business Reporting Language) categories to the sentences and mapping these categories to the 42 financial variables used in the structured dataset.
   • Algorithms: Three explanation generation methods are evaluated against the silver standard:
     • EMD (Explanation using Mahalanobis Distance): Identifies subsets of features where the rank difference of an anomalous point exceeds a statistical threshold.
     • EiForest (Explanation using iForest): Uses path-based features from Isolation Forest trees to identify variables contributing to isolation.
     • EMI (Explanation using Maximal Isolation): Formulates the problem as an Integer Linear Programming (ILP) task to find a conjunction of conditions that maximally isolates the anomalous point from the rest of the dataset.
   • Audit Suggestions: The system maps the identified explanatory variables back to specific adverse remark sentences from past audit reports, transforming them into actionable audit suggestions (e.g., "Check if provisions for pending taxes have been made").

Key Contributions
The paper claims several novel contributions over previous research (specifically Shinde et al., 2022; Vaishampayan et al., 2022; and Pawar et al., 2023):

1. Scale and Scope: The dataset is significantly larger (11,460 records over 5 years vs. single-year data in prior work) and integrates multiple FS types (BS, IS, CS) rather than just Balance Sheets.
2. Expanded Variables: The inclusion of additional financial variables from Income and Cash-flow statements enhances detection effectiveness.
3. Automated Silver Labeling: The proposal of two strategies (using LLMs and T5 classifiers) to automatically generate silver labels for the entire dataset, replacing the need for small, manually labeled subsets.
4. Improved Classification Model: The use of a T5 encoder-decoder model for sentence classification in audit reports, which provides both classification labels and explanations, outperforming the previously used Attn-BERT model.
5. Automated Explanation Generation: The introduction of an unsupervised method to automatically generate silver-standard explanations using XBRL mapping, eliminating the manual effort required in previous studies.
6. End-to-End Automation: The integration of these components creates a system that removes manual labor from the assistance generation process, aiming for efficiency, scalability, and objectivity.

Results

• Detection Performance: In anomaly detection, the Connectivity-Based Outlier Factor (COF) algorithm achieved the highest Precision@20 (0.60) for the year 2014, while the Local Outlier Factor (LOF) showed the highest average precision across all five years (0.56). Among regression models, SVR-based models achieved the highest average Precision@20 (up to 0.83 for specific models).
• Explanation Performance: The EiForest algorithm demonstrated the highest average Precision (0.25), Recall (0.27), and F1-score (0.24) when compared against the silver-standard ground truth explanations. The EMI algorithm achieved an average precision of 0.21. The authors note that while these metrics are modest, they represent an improvement over previous studies when accounting for the increased complexity and number of variables (42 vs. 18 in prior work).
• Model Comparison: The T5 classifier achieved an F1-score of 0.80 for adverse remark classification, outperforming the Attn-BERT model (F1=0.72).

Significance and Claims
The paper positions this work as a foundational step toward an AI-assisted Auditor Assistance system. The primary significance lies in the ability to extract insights from a massive historical corpus of FS and audit reports to provide objective, data-driven assistance. The authors claim that by automating the detection of potential misinformation and the generation of explanations (via XBRL mapping), the system can reduce the time, cost, and subjectivity involved in auditing. It aims to help auditors focus their efforts on specific, high-risk areas identified by the system. The authors explicitly state that the system is designed to assist human auditors, not replace them, by offering focused suggestions for further exploration and validation of business processes. The work serves as an integration and extension of their previous research, demonstrating efficacy on a significantly larger and more diverse dataset.