Technical Summary: ChunkNorris

Problem Statement

In Retrieval-Augmented Generation (RAG) systems, the quality of Information Retrieval (IR) is fundamentally dependent on the preprocessing of source documents. Specifically, the parsing and chunking of PDF documents present significant challenges. PDFs are designed for human readability rather than machine processing, often containing complex layouts, multi-column text, tables, and figures.

Current state-of-the-art approaches to PDF parsing and chunking frequently rely on Machine Learning (ML) and Computer Vision techniques (e.g., Unstructured, Docling, Marker). While these methods can achieve high accuracy in extracting structured data, they incur substantial computational overhead, high energy consumption, and long execution times. These resource-intensive requirements often render them impractical for production environments with limited computational resources or strict latency constraints. Furthermore, existing heuristic-based methods often lack robustness in handling document hierarchy and semantic coherence without ML assistance.

Methodology

The authors introduce ChunkNorris, a novel, heuristic-based technique designed to optimize PDF parsing and chunking without relying on Machine Learning or GPU acceleration. The approach prioritizes efficiency, low energy consumption, and the preservation of document structure.

1. The Parser

ChunkNorris utilizes the PyMuPDF library for low-level document processing. The parsing pipeline converts raw PDF content into a unified, clean Markdown format (MarkdownDoc object) through several specific heuristics:

• Header/Footer Removal: Spans appearing in the same position on >33% of pages are identified and removed.
• Link Preservation: Hyperlinks, often invisible boxes in PDFs, are extracted and bound to their corresponding text spans.
• Table Extraction: The system employs a vectorized line recombination method to parse tables, handling both visible cell boundaries and inferred alignments, including merged cells.
• Structural Grouping: Text spans are grouped into lines and then blocks (paragraphs or titles) based on vertical spacing.
• Hierarchy Inference:
  • Main Title: Inferred from blocks on the first page with font sizes larger than the body.
  • Section Headers: Detected via metadata Table of Contents (ToC), regex patterns, indentation levels, or font size analysis.
• Output: The result is a structured Markdown document ready for chunking.

2. The Chunker

The MarkdownChunker operates on the structured Markdown output using three guiding principles:

• Homogeneity: Chunks are bounded by document section headers.
• Context Preservation: Parent section headers are prepended to every chunk to maintain context when sections are read in isolation.
• Uniformity: Chunk sizes are regulated to ensure consistency, as embedding models are sensitive to length variations.

Algorithm:

1. A ToC tree is constructed from detected headers.
2. Chunks are recursively generated based on this tree, containing the current section's content and all parent headers.
3. Refinement:
   • Sections below a minimum word count are discarded.
   • Chunks exceeding a hard limit are split at newline characters to preserve tables and code blocks.
   • Titles are preserved at the start of sub-chunks.

Key Contributions

1. ChunkNorris Algorithm: A fully unsupervised, ML-free parsing and chunking pipeline that achieves high performance with minimal computational overhead.
2. PIRE Dataset: The authors constructed and released the PDF dataset for Information Retrieval Evaluation (PIRE) on Hugging Face. This dataset contains 100 diverse PDFs (including financial reports, legal documents, arXiv papers, and manuals) and includes:
   • A "single-chunk" subset with 300 annotated question-document pairs.
   • A "multi-chunk" subset with 32 complex questions requiring information from multiple pages.
3. Comprehensive Benchmarking: A rigorous evaluation framework comparing ChunkNorris against existing tools (Marker, Open-Parse, Docling, PyPDF) across execution time, energy consumption, and retrieval accuracy (Recall and NDCG).

Results

The benchmark was conducted on a dataset of 100 PDFs (5,286 pages) using an Intel Xeon Gold CPU and Tesla V100S GPU.

• Execution Time: ChunkNorris demonstrated superior speed, averaging 105 ms per page, significantly outperforming Docling (533 ms), Marker (717 ms), and Open-Parse-U (2538 ms).
• Energy Consumption:
  • CPU: ChunkNorris consumed 0.47 Wh for the entire dataset, compared to 5.11 Wh for Docling and 34.32 Wh for Open-Parse-U.
  • GPU: ChunkNorris required 0.0 Wh of GPU energy, whereas other ML-based tools (e.g., Marker, Open-Parse-U) required significant GPU power (up to 777.93 Wh).
• Retrieval Performance: When evaluated using various embedding models (ranging from 23M to 335M parameters), ChunkNorris achieved competitive or superior Recall and NDCG scores compared to baseline and advanced techniques. The results suggest that high-quality heuristic structuring can compensate for the lack of ML-based semantic analysis in IR tasks.

Significance and Claims

The paper posits that heuristic-based methods remain highly viable for real-world RAG applications, particularly in resource-constrained environments. The authors claim that ChunkNorris offers a practical and efficient alternative to heavy ML-based pipelines, demonstrating that:

• High-quality parsing and chunking can be achieved without the computational cost of training or inference on large models.
• Simple, effective heuristics can preserve semantic coherence and document hierarchy sufficiently for effective Information Retrieval.
• There is a significant opportunity to reduce the ecological footprint of RAG systems by adopting lightweight, CPU-efficient parsing strategies.

The work emphasizes that for many production use cases, the trade-off between the marginal gains of complex ML parsing and the substantial costs in time and energy favors the efficient, unsupervised approach proposed by ChunkNorris. The code and dataset are made open-access to facilitate further research and adoption.