💬 NLP

Adaptive Chunking: Optimizing Chunking-Method Selection for RAG

This paper introduces "Adaptive Chunking," a framework that optimizes Retrieval-Augmented Generation (RAG) performance by dynamically selecting the best chunking strategy for each document based on five novel intrinsic metrics, achieving significant improvements in answer correctness and success rates across diverse domains without altering underlying models or prompts.

Original authors: Paulo Roberto de Moura Júnior, Jean Lelong, Annabelle Blangero

Published 2026-03-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Paulo Roberto de Moura Júnior, Jean Lelong, Annabelle Blangero

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a brilliant but slightly forgetful student (the AI) how to answer questions about a massive library of books. The problem isn't that the student is dumb; it's that the books are too big to read all at once. So, you have to cut the books into smaller "chunks" to hand to the student.

The paper "Adaptive Chunking" argues that for years, librarians have been using a one-size-fits-all pair of scissors. They just cut every book into 500-word pieces, regardless of whether the book is a legal contract, a technical manual, or a novel.

Sometimes this works. But often, it's a disaster. You might cut a sentence in half, separating a "he" from the person it refers to. Or you might cut a table in half, leaving the top half in one pile and the bottom half in another. The student gets confused, the answers are wrong, and the whole system fails.

Here is the paper's solution, broken down simply:

1. The Problem: The "Blind Scissors" Approach

Current AI systems usually chop documents up using rigid rules (like "cut every 500 words").

The Analogy: Imagine trying to cut a pizza. If you just cut it into perfect squares without looking at where the pepperoni is, you might end up with a slice that has no cheese and another that has three pepperonis.
The Result: The AI loses the "context." It sees a piece of text but doesn't know what it's talking about because the surrounding clues were chopped off.

2. The Solution: The "Smart Tailor" (Adaptive Chunking)

The authors propose a new system called Adaptive Chunking. Instead of using one pair of scissors for the whole library, this system acts like a smart tailor.

Before cutting a document, the tailor looks at the fabric (the document) and asks:

"Is this a legal contract with strict articles?" (Cut at the article breaks).
"Is this a technical manual with big tables?" (Keep the tables whole).
"Is this a story with long paragraphs?" (Cut at the paragraph ends).

The system picks the best cutting strategy for each specific document automatically.

3. The Five "Quality Checks" (The Metrics)

How does the tailor know if a cut is good? They use five specific "rulers" to measure the quality of the chunks. Think of these as a checklist for a perfect puzzle piece:

References Completeness (RC): Did we cut the pronouns?
- Analogy: If the text says "The King," and the next chunk starts with "He," that's bad. The "He" needs to stay with "The King." This metric checks that we didn't separate a person from their name.
Intrachunk Cohesion (ICC): Is the chunk about one thing?
- Analogy: A chunk shouldn't be a salad where you mixed apples, rocks, and tires. It should be a bowl of just apples. This checks if the chunk stays on one topic.
Document Contextual Coherence (DCC): Does it fit with its neighbors?
- Analogy: Even if a chunk is about apples, does it make sense if you read the chunk right before it? This ensures the flow of the story isn't broken.
Block Integrity (BI): Did we break the furniture?
- Analogy: If you have a picture or a table, you can't cut it in half. This metric ensures that tables, images, and headers stay whole.
Size Compliance (SC): Is the chunk the right size?
- Analogy: If the chunk is too small, it's useless noise. If it's too big, the student gets overwhelmed. This ensures every piece fits in the student's "backpack."

4. The New Tools

To make this work, the authors built two new "cutting tools":

The LLM-Regex Splitter: They ask an AI to look at the first few pages of a document and say, "Hey, what's the best way to cut this specific book?" The AI then writes a custom set of instructions (a regex pattern) to cut the rest of the book perfectly.
The Split-Then-Merge Splitter: This tool cuts the text into tiny pieces first, then glues them back together if they are too small, ensuring nothing is lost or too tiny to be useful.

5. The Results: Why It Matters

When they tested this on real-world documents (legal, technical, and social science), the results were impressive:

More Correct Answers: The AI got the right answer 72% of the time, compared to 62-64% with the old methods.
More Questions Answered: The system stopped saying "I don't know" and actually answered 30% more questions.
No Magic Needed: They didn't change the AI model or the prompts. They just fixed how the information was fed to it.

The Bottom Line

The paper teaches us that how you feed information to an AI is just as important as the AI itself.

By stopping the "one-size-fits-all" approach and using a smart, adaptive system that respects the unique structure of every document, we can build AI systems that are much smarter, more reliable, and actually understand what they are reading. It's the difference between handing someone a shredded newspaper and handing them a neatly organized, easy-to-read magazine.

1. Problem Statement

Retrieval-Augmented Generation (RAG) systems rely heavily on how documents are segmented (chunked) for indexing and retrieval. Current approaches suffer from two primary issues:

The "One-Size-Fits-All" Limitation: Most systems apply a single, fixed chunking strategy (e.g., fixed character count, sentence-based, or recursive splitting) to an entire corpus. This fails to account for the diverse structural and semantic nuances of different document types (e.g., legal contracts vs. technical manuals vs. social science papers).
Lack of Intrinsic Evaluation: Existing evaluation frameworks rely on extrinsic metrics (downstream RAG performance like Hits@K or Answer Correctness). This makes it difficult to isolate the specific impact of the chunking strategy itself, as performance is confounded by the retrieval model, generator, and prompt engineering. There is a critical gap in intrinsic metrics that directly assess chunk quality independent of the downstream task.

2. Methodology

The authors propose Adaptive Chunking, a framework that dynamically selects the optimal chunking strategy for each individual document based on a set of intrinsic metrics.

A. Document Pre-processing

Corpus: A diverse dataset of 33 PDFs across Legal, Technical, and Social Science domains (approx. 1.18M tokens).
Parsing: Documents are converted to Markdown using Microsoft Azure AI Document Intelligence.
Enhancements: The pipeline includes custom handling for oversized tables (splitting them), grouping headers/footers, and preserving inseparable spans (tables, figures, footnotes) to maintain structural integrity.

B. Novel Chunking Techniques

The framework introduces two new chunking methods alongside standard baselines:

LLM-Regex Splitter: An LLM (GPT-5) is prompted with a document sample to generate a specific regular expression (regex) pattern tailored to that document's structure (e.g., article delimiters in legal texts). This pattern is then applied deterministically to the full document.
Split-then-Merge Recursive Splitter: A two-pass strategy inspired by LangChain's recursive splitter but optimized to avoid tiny, context-poor chunks.
- Pass 1: Recursively split using a priority list of separators (Titles → Sections → Sentences → Characters) until segments are $\le$ target size.
- Pass 2: Greedily merge adjacent segments without exceeding the size limit, backtracking if necessary to maintain overlap.

C. Post-Processing Regularization

To ensure size constraints are met without breaking structure, two passes are applied:

Oversized Splitting: Chunks > 1,100 tokens are re-split.
Tiny-Chunk Merging: Chunks < 100 tokens are merged with neighbors (up to a max of 1,150 tokens) to prevent retrieval slot waste.

D. Five Intrinsic Evaluation Metrics

The core innovation is a suite of five metrics computed at the document level to guide strategy selection:

References Completeness (RC): Measures the fraction of entity-pronoun pairs that remain intact within a chunk (using coreference resolution). Breaking these leads to retrieval failure.
Intrachunk Cohesion (ICC): Quantifies semantic consistency within a chunk by calculating the mean cosine similarity between sentence embeddings and the chunk embedding.
Document Contextual Coherence (DCC): Assesses how well a chunk aligns with its broader local context (sliding windows of ~3,000 tokens) to ensure the chunk is understandable in isolation but contextually relevant.
Block Integrity (BI): Evaluates whether structural units (paragraphs, tables, figures) remain unbroken.
Size Compliance (SC): The proportion of chunks falling within the target token range (100–1,100 tokens).

E. Adaptive Selection Logic

For each document, the system computes the mean score of these five metrics for multiple candidate chunking strategies. The strategy with the highest aggregate score is selected for that specific document.

3. Key Contributions

Adaptive Framework: A paradigm shift from static, corpus-wide chunking to document-aware, dynamic strategy selection.
Intrinsic Metric Suite: The introduction of RC, ICC, DCC, BI, and SC, providing a robust, task-agnostic way to evaluate chunk quality.
Novel Algorithms: The LLM-Regex splitter and the Split-then-Merge recursive splitter, which better handle structural boundaries and size constraints.
Comprehensive Evaluation: A pipeline that isolates chunking impact by keeping models and prompts fixed, demonstrating that intrinsic metric improvements translate directly to downstream gains.

4. Results

Experiments were conducted on a hybrid search pipeline (BM25 + Dense Retrieval + Reranking) using GPT-4.1 for generation.

Intrinsic Metric Performance

Adaptive Chunking achieved the highest mean intrinsic score (91.07%), outperforming the best single static method (Recursive Splitter s=1100 at 90.68%).
Trade-offs: The metrics are complementary but often conflicting (e.g., smaller chunks have higher ICC but lower DCC; preserving blocks (BI) may lower ICC). This confirms the necessity of an adaptive approach rather than a single "best" method.
Post-Processing: Significantly improved Size Compliance (SC) and overall mean scores (up to +16 percentage points) by eliminating tiny fragments and oversized chunks.

Downstream RAG Performance

Without changing the LLM, prompts, or retrieval models, Adaptive Chunking yielded significant improvements:

Retrieval Completeness: Increased from 58.08% (LangChain default) to 67.68% (+9.60 pp).
Answer Correctness: Increased from 70.11% to 78.01% (+7.90 pp).
Answer Rate: The number of successfully answered questions increased by 32.7% (from 49/99 to 65/99).
Amplification Effect: Small improvements in intrinsic metrics (0.4–2.4%) resulted in disproportionately large gains in downstream performance (8–10%), suggesting that better chunks compound through the retrieval and generation pipeline.

5. Significance and Conclusion

The paper demonstrates that document-aware chunking is a critical, yet often overlooked, lever for optimizing RAG systems.

Practical Impact: It offers a practical path to more robust RAG systems without requiring expensive model retraining or complex prompt engineering.
Scalability: While the adaptive selection adds computational overhead at indexing time (approx. 210s per document for the full pipeline), this cost is amortized during the indexing phase, making it viable for production.
Future Direction: The work highlights that "one-size-fits-all" is inadequate for heterogeneous corpora. The proposed metric suite provides a standardized language for evaluating and improving chunking strategies across different domains.

Code Availability: The authors have released the implementation at https://github.com/ekimetrics/adaptive-chunking.