Citation-Enforced RAG for Fiscal Document Intelligence: Cited, Explainable Knowledge Retrieval in Tax Compliance

1. Problem Statement

Tax authorities and public-sector agencies face significant challenges in managing vast volumes of unstructured and semi-structured fiscal documents (e.g., tax forms, instructions, policy manuals). These documents are often multimodal (containing text, tables, scanned images, and complex layouts) and heterogeneous.

• Limitations of Current AI: While Large Language Models (LLMs) and standard Retrieval-Augmented Generation (RAG) systems show promise, they are ill-suited for high-stakes regulatory domains.
  • Hallucinations: Unconstrained generation often produces confident but factually incorrect or unsupported claims.
  • Lack of Provenance: Standard RAG pipelines often rely on LLM-generated summaries of documents for retrieval, obscuring the link between the answer and the original source.
  • Absence of Abstention: Existing systems typically force an answer even when evidence is weak, which is dangerous in tax compliance where an incorrect interpretation can lead to legal or financial penalties.
• Core Gap: There is a lack of a framework that enforces citation fidelity, preserves document provenance at the span level, and incorporates abstention mechanisms for multimodal fiscal documents.

2. Methodology

The authors propose a Source-First, Citation-Enforced Multimodal RAG Framework designed specifically for auditability and explainability.

A. Source-First Ingestion & Multimodal Extraction

Unlike approaches that summarize content before embedding, this system prioritizes raw data extraction:

• Extraction: The pipeline processes PDFs (including scanned documents) using OCR and layout parsing.
• Linearization: Complex elements like tables and flowcharts are converted into self-contained text strings that explicitly restate headers and context, rather than generating abstract summaries.
• Metadata Preservation: Every extracted chunk retains explicit metadata linking it to the Document ID and Page Range.
• No Pre-Generation Summaries: The system avoids embedding LLM-generated abstractions, ensuring the retrieval index contains only verbatim or OCR-derived content from authoritative sources.

B. Embedding and Retrieval

• Model: Uses the BAAI/bge-large-en-v1.5 dense embedding model for semantic retrieval.
• Indexing: Chunks are stored in a FAISS vector index.
• Confidence Scoring: Retrieval relies on cosine similarity scores. The system treats these scores as a proxy for evidence sufficiency. High scores indicate strong semantic alignment; low scores trigger abstention.

C. Citation-Enforced Generation

• Evidence-Only Prompting: The generative model (Meta Llama 3.2 3B Instruct) is strictly constrained to use only the retrieved evidence chunks. It is forbidden from using parametric knowledge or external assumptions.
• Hard Citation Constraint: Every generated paragraph must include explicit citations in a standardized format ([Doc ID, Page Range]).
• Validation Loop: A post-generation validator checks for citation presence and formatting. If citations are missing or malformed, the answer is regenerated. If constraints cannot be met, the system defaults to abstention.

D. Abstention Logic

• Decision Policy: If the top-ranked retrieved chunk has a similarity score below a predefined threshold, the system abstains from answering.
• Rationale: In tax compliance, acknowledging uncertainty is safer than providing a potentially misleading answer. The system explicitly communicates why it abstained (e.g., insufficient evidence).

3. Key Contributions

1. Source-First Architecture: A novel ingestion strategy that embeds raw extracted text rather than LLM summaries, enabling span-level traceability and preventing abstraction errors.
2. Citation Enforcement Mechanism: A hard constraint requiring every generated claim to be explicitly grounded in retrieved evidence with specific document and page references.
3. Abstention-Aware Policy: A decision framework that allows the system to decline answering when evidence is weak, significantly reducing hallucination risks in high-stakes scenarios.
4. Real-World Evaluation: The framework is tested on a corpus of 298 real-world fiscal documents (10,491 chunks) from the U.S. IRS, California FTB, and New York State Tax Department, without using synthetic data.
5. Comprehensive Assessment: A dual evaluation using automatic metrics and human review to assess retrieval quality, citation correctness, hallucination rates, and utility.

4. Results

The system was evaluated on 55 manually crafted queries covering federal and state tax topics.

• Retrieval Performance: The system successfully identified semantically relevant chunks for most answerable queries, with top-ranked chunks achieving cosine similarity scores > 0.55.
• Citation Fidelity:
  • Format Compliance: 100% of non-abstained answers followed the required citation format.
  • Citation Support: 94.5% of cited text fully supported the claims made in the answer.
• Hallucination Control: The hallucination rate (claims without evidence) was extremely low at 1.8%.
• Abstention Accuracy: The system correctly abstained in 88.0% of cases where no relevant information existed in the corpus.
• Usefulness: Human reviewers rated the system's usefulness at 4.2/5.0, noting that explicit citations significantly reduced the effort required to verify outputs compared to manual search.

Qualitative Comparison: Compared to standard RAG (which prioritizes fluency) and summary-based multimodal RAG (which prioritizes unified semantic representation), this framework prioritized verifiability and conservative behavior, resulting in fewer unsupported claims but slightly lower answer coverage in ambiguous cases.

5. Significance and Implications

• Trustworthy AI for Regulation: This work demonstrates that generative AI can be safely deployed in regulated domains (tax, legal, healthcare) by shifting the optimization objective from "answer completeness" to "evidence grounding."
• Auditability: By preserving document provenance and enforcing citations, the system creates an audit-ready workflow where every claim can be traced back to the original source document.
• Risk Mitigation: The integration of abstention logic prevents the system from confidently stating incorrect interpretations, a critical feature for public-sector analytics.
• Scalability: The modular design allows agencies to integrate citation enforcement and abstention policies into existing retrieval systems without wholesale replacement, facilitating responsible adoption of AI in government workflows.

In conclusion, the paper presents a practical pathway for transforming multimodal RAG from a "black box" generator into a transparent, verifiable, and conservative decision-support tool for fiscal intelligence.