💬 NLP

MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A

MM-BizRAG introduces a multimodal retrieval-augmented generation framework that dynamically routes enterprise documents through orientation-specific parsing pipelines to explicitly capture structural information, significantly outperforming existing vision-centric baselines in Q&A accuracy while offering a cost-effective evaluation metric called FastRAGEval.

Original authors: Hanoz Bhathena, Parin Rajesh Jhaveri, Rohan Mittal, Prateek Singh, Aymen Kallala, Rachneet Kaur, Yiqiao Jin, Zhen Zeng, Adwait Ratnaparkhi, Denis Kochedykov

Published 2026-06-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hanoz Bhathena, Parin Rajesh Jhaveri, Rohan Mittal, Prateek Singh, Aymen Kallala, Rachneet Kaur, Yiqiao Jin, Zhen Zeng, Adwait Ratnaparkhi, Denis Kochedykov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you work in a giant, chaotic library where the books come in two very different shapes: tall, narrow reports (like financial statements) and wide, flat slide decks (like PowerPoint presentations).

For a long time, the "robots" (AI systems) trying to answer questions from these books have taken a lazy approach. They've been told to just take a photo of every single page and feed that photo into the robot's brain. They hope the robot can "guess" the structure of the page just by looking at the picture.

The Problem:
This "photo-only" method works okay for simple slides, but it fails miserably with complex reports. It's like trying to understand a detailed blueprint by squinting at a blurry photograph of it. The robot misses the specific numbers in the tables, the flow of the text, and the relationship between a chart and the paragraph next to it. It's too busy looking at the "whole picture" to see the important details.

The Solution: MM-BizRAG
The authors of this paper built a new system called MM-BizRAG. Instead of treating every document the same, this system acts like a smart librarian who knows exactly how to handle different types of books.

Here is how it works, using simple analogies:

1. The "Smart Sorter" (Document Structure-Aware Split)

When a document arrives, the system first asks: "Is this a tall report or a wide slide deck?"

If it's a Report (Vertical): The system doesn't just take a photo. It carefully cuts the document into pieces, reading the text, extracting the tables, and describing the images. It keeps track of where everything was on the page, like a puzzle master keeping the pieces in order.
If it's a Slide Deck (Horizontal): The system realizes that on a slide, the text, image, and chart are all mixed together to tell one story. So, it treats the whole slide as a single unit, describing the entire scene at once.

2. The "Two-Backpack" Strategy (Decoupling Retrieval and Generation)

This is the system's secret sauce. Most other systems try to stuff everything into one backpack to find the answer. MM-BizRAG uses two different backpacks:

Backpack A (For Searching): This contains simplified, text-heavy summaries and descriptions. It's lightweight and fast, making it easy to search through thousands of documents to find the right ones.
Backpack B (For Answering): Once the right documents are found, the system pulls out the full, rich versions (including the actual images and tables) and assembles them perfectly for the final answer.

Why this matters: It's like using a cheap, fast map to find a city (Backpack A), and then once you arrive, using a high-definition, 3D model of the city to give your tour guide the best directions (Backpack B). You get the speed of a search engine with the depth of a human expert.

3. The "One-Call" Judge (FastRAGEval)

To test if their system is actually good, they needed a way to grade the answers. Existing grading tools were like a teacher who had to read the answer, break it down into tiny sentences, check each one, and then write a report. This took a long time and cost a lot of money.

The authors invented FastRAGEval, which is like a super-fast teacher who can read the whole answer, check the facts, and give a grade in a single glance. It's twice as fast and agrees better with human judges than the old methods.

The Results

When they tested this new system against the "photo-only" robots:

For Reports: It was a massive win, improving accuracy by up to 32%. The system finally understood the complex tables and charts that the others missed.
For Slides: It did just as well as the best photo-based systems, proving that you don't need to ignore the text to handle slides.
Speed: They found a "sweet spot" configuration that was almost as accurate as the most complex version but ran about twice as fast.

In a Nutshell

The paper argues that we shouldn't just rely on AI to "guess" the structure of a document by looking at a picture. Instead, we should actively parse (break down) the document based on its shape, use a fast method to find the right info, and then assemble the rich details only when we need to write the final answer. This approach makes the AI much smarter, especially for the complex documents businesses use every day.

Technical Summary: MM-BizRAG

Problem Statement

Recent advancements in Multimodal Retrieval-Augmented Generation (MM-RAG) have increasingly favored "minimal parsing" strategies. These approaches rely on page-level images to generate retriever embeddings and serve as direct inputs to Vision-Language Models (VLMs) for answer generation. While efficient, the authors argue that this trend neglects the explicit handling of rich, structured information inherent in complex enterprise documents (e.g., financial reports, legal filings, slide decks).

Current state-of-the-art methods often offload the understanding of document structure to pre-trained embeddings or VLMs, assuming these models can implicitly capture layout and semantic relationships. However, the paper posits that pre-trained models struggle to implicitly capture the structured information in complex enterprise documents, which often differ significantly from their training data. Furthermore, existing pipelines lack standardization in document representation and ingestion design, leading to suboptimal performance on heterogeneous enterprise data.

Methodology: MM-BizRAG

MM-BizRAG is a multimodal RAG framework designed to proactively extract and represent document structure rather than relying on implicit understanding. The system operates on a core premise: document structure should dictate the ingestion strategy.

1. Document Structure-Aware Split

The pipeline begins by classifying documents into two categories using an LLM-based classifier (or file metadata):

Vertical Documents ( $D_V$ ): Reports, filings, and text-heavy documents with a linear reading order.
Horizontal Documents ( $D_H$ ): Slide decks and presentations where content is spatially arranged and meaning is often holistic at the page level.

2. Orientation-Specific Ingestion Pipelines

Based on the classification, documents are routed through distinct pipelines:

For Vertical Documents ( $D_V$ ):
- Layout-Aware Parsing: Uses tools like Docling to extract aligned text blocks, tables, and intra-page pictures.
- Artifact Transformation:
  - Text: Linearized with unique placeholders for tables and images to preserve reading order.
  - Tables: Converted to Markdown and passed to an LLM to generate row-by-row descriptions.
  - Pictures: Processed by a VLM to generate descriptions, filtering out uninformative elements (e.g., logos).
- Positional Alignment: A unified LLM-driven pipeline uses placeholders to ensure tables and images are aligned with their corresponding text positions.
For Horizontal Documents ( $D_H$ ):
- Holistic Representation: Explicit layout parsing is deemed ineffective for slides. Instead, the system treats each slide as a holistic semantic unit.
- Page-Level Processing: A VLM generates a comprehensive textual description ( $\delta$ ) capturing all semantic content (text, visuals, relationships) for the entire page.
- Representation: The unit is a pair of the page image and its generated description.

3. Adaptive Retrieval and Generation

The framework introduces three variants (TCTE, PCMHE, TCMIE) that differ in how they construct retrievable chunks and embeddings, but share a common inference stack:

Decoupled Representations: A key architectural distinction is the decoupling of representations used for retrieval from those used for generation.
- Retrieval: Chunks are embedded (using text or multimodal models) and indexed.
- Generation (Inference-Time Assembly): Retrieved chunks are dynamically assembled into a multimodal context. For vertical documents, this involves injecting table markdown and image artifacts back into text chunks at their original placeholder positions. For horizontal documents, page images are concatenated with their descriptions.
No Finetuning: The system is designed to work with out-of-the-box LLMs and encoder models, requiring no model finetuning.

4. Evaluation Metric: FastRAGEval

The authors introduce FastRAGEval (FRE), a single-call LLM Judge metric for fine-grained generative recall. Unlike RAGChecker, which decomposes answers into atomic claims across two sequential LLM calls, FRE computes precision, recall, and F1 in a single pass. This reduces cost and latency by half while achieving stronger alignment with human judgments.

Key Contributions

MM-BizRAG Framework: A novel system that dynamically routes documents through orientation-specific ingestion pipelines (layout-aware for vertical, holistic for horizontal) to preserve cross-modal alignment.
Inference-Time Multimodal Assembly: An architectural innovation that decouples retrieval indexing from generation context, allowing for richer, grounded answers without inflating the retrieval index.
Controlled Variant Study: A systematic evaluation of three MM-BizRAG variants (TCTE, PCMHE, TCMIE) on a large internal enterprise dataset and public benchmarks (SLIDEVQA, FINRAGBENCH-V), isolating the impact of ingestion and embedding strategies.
FastRAGEval (FRE): A cost-efficient, single-call evaluation metric that outperforms existing multi-call metrics in alignment with human judgment.

Results

The paper evaluates MM-BizRAG against state-of-the-art vision-centric baselines (ColPali, VisRAG) and a text-only RAG baseline across three datasets:

SLIDEVQA: MM-BizRAG variants outperformed ColPali and VisRAG by 3.6–11.1% on FRE-Recall.
FINRAGBENCH-V: The framework showed substantial gains, outperforming vision-centric baselines by up to 32 percentage points. The authors note that vision-only pipelines degrade significantly on dense, vertically structured reports, whereas MM-BizRAG's layout-aware parsing excels.
Internal Enterprise Dataset: MM-BizRAG consistently outperformed the text-only baseline, particularly on questions requiring table or picture analysis, where the text-only baseline showed clear degradation.
Efficiency: The TCTE (Token-level chunking, Text embedding) variant is highlighted as the recommended production configuration, achieving recall within 1–3% of the best configuration while incurring roughly half the latency of the multimodal embedding variants (PCMHE).

Significance and Claims

The paper claims to re-establish the value of direct, structure-aware document understanding in multimodal RAG. It argues that for enterprise applications, relying solely on page-level images and implicit structural understanding by VLMs is insufficient for complex, heterogeneous documents.

The authors assert that MM-BizRAG provides a practical blueprint for harmonizing parsing-based and embedding-based approaches. By explicitly handling document structure and decoupling retrieval from generation, the system achieves higher quality end-to-end performance without the need for finetuning. The introduction of FastRAGEval is also claimed to offer a more efficient and human-aligned method for evaluating generative recall in RAG systems.

The paper remains modest regarding its limitations, acknowledging that public benchmarks (like SLIDEVQA) may not fully reflect the complexity of industry-grade presentations and that the evaluation on FINRAGBENCH-V was limited to a subset of English PDFs due to computational constraints. Additionally, the proprietary enterprise dataset used for primary evaluation cannot be released at this time.