💬 NLP

Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation: A Dynamic Strategy with Contextual Window Expansion

This paper introduces Query-Adaptive Semantic Chunking (QASC), a dynamic strategy that integrates user queries into the document segmentation process to achieve significantly higher retrieval relevance and coherence compared to fixed, semantic, and agentic chunking methods.

Original authors: Mudit Rastogi

Published 2026-05-25

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Mudit Rastogi

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a specific recipe in a massive, unorganized library of 100 cookbooks.

The Old Way (Traditional Chunking)
Right now, most computer systems that answer questions (called RAG systems) handle these cookbooks by cutting them into equal-sized pieces, like slicing a loaf of bread into uniform slices, regardless of what's inside.

If you ask, "How do I make sourdough?" the system might hand you a slice that contains the recipe, but it also includes three pages of a story about the baker's childhood and a list of unrelated spices.
If you make the slices smaller to avoid the extra fluff, you might cut the recipe in half, leaving out the crucial step about kneading the dough.
The system has to guess the perfect slice size, but no single size works for every question. It's like trying to fit a square peg in a round hole every time you ask a question.

The New Way (QASC)
This paper introduces a new strategy called Query-Adaptive Semantic Chunking (QASC). Instead of blindly slicing the bread, QASC acts like a smart, attentive reader who highlights the exact parts of the book you need before cutting anything out.

Here is how it works, using a simple three-step process:

Finding the "Seed" (The Highlighter):
When you ask a question, QASC reads the whole document and looks for sentences that are most similar to your question. It's like a reader scanning a page and saying, "Aha! This sentence is exactly what I'm looking for!" These are called seed sentences.
- Analogy: If you ask, "What is the capital of France?", the system highlights the sentence "Paris is the capital of France."
Expanding the Window (The Context Bubble):
A single highlighted sentence often isn't enough on its own. It might need the sentence before it to explain why or the sentence after it to explain how. QASC creates a "bubble" of context around that seed sentence. It grabs a few sentences before and after the highlight to ensure the answer makes sense on its own.
- Analogy: If the highlighted sentence is "He won the race," the system grabs the surrounding sentences to tell you who "he" is and what race it was, so the answer isn't confusing.
Merging and Polishing (The Final Cut):
Sometimes, two highlighted areas are close together. QASC merges them into one smooth, coherent chunk. It also checks to make sure the chunk doesn't start or end in the middle of a thought.
- Analogy: If you highlighted two paragraphs that are right next to each other, QASC glues them together into one perfect paragraph rather than giving you two disjointed scraps of paper.

Why is this better?
The paper tested this method against the old "uniform slice" method and some other fancy methods. Here is what they found:

Better Answers: Because the chunks are built specifically around your question, the computer doesn't have to guess what is important. It gets the right information immediately.
Less Confusion: The answers generated by the computer were more accurate and made fewer mistakes (hallucinations) because the context provided was tightly focused on the question.
Speed vs. Quality: While this method takes a tiny bit more time to set up the chunks (because it has to read the question first), it is still fast enough for real-time use. It's a small wait for a much better result.

The Bottom Line
Think of traditional chunking as a factory machine that chops everything into the same size. QASC is like a personal librarian who listens to your specific request, finds the exact pages you need, and hands you a perfectly bound, relevant booklet.

The paper proves that by letting the user's question decide how to cut the document, rather than cutting first and asking later, we get significantly better, more accurate, and more helpful answers.

Technical Summary: Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation

1. Problem Statement

Retrieval-Augmented Generation (RAG) systems rely heavily on the quality of document chunking to retrieve relevant context for downstream generation. Traditional approaches predominantly utilize fixed-size chunking, which segments documents into uniform units regardless of semantic structure or user intent. This rigidity creates an irreconcilable trade-off: smaller chunks improve precision but risk losing broader context, while larger chunks preserve context but dilute relevance with extraneous material.

Existing advanced methods, such as semantic chunking (based on embedding similarity) and agentic chunking (based on intent), address structural coherence but share a fundamental limitation: they are query-agnostic. They perform segmentation as a preprocessing step independent of the specific user query. Consequently, a document is chunked identically whether the user seeks methodology, results, or a specific technical concept, missing a critical opportunity to optimize relevance at the source.

2. Methodology: Query-Adaptive Semantic Chunking (QASC)

The paper proposes Query-Adaptive Semantic Chunking (QASC), a dynamic strategy that integrates the user query directly into the segmentation process. QASC operates through a pipeline that mimics human reading behavior: identifying points of interest, gathering surrounding context, evaluating collective relevance, and synthesizing coherent units.

The methodology consists of four core mechanisms:

Seed Sentence Selection via Similarity Scoring:
- The document $D$ is split into sentences $S = \{s_1, ..., s_n\}$ .
- Sentence embeddings $e(s_i)$ and the query embedding $e(q)$ are computed using a pre-trained model (e.g., Sentence-BERT).
- Cosine similarity is calculated for all sentences against the query.
- Adaptive Thresholding: Instead of a fixed threshold, a percentile-based threshold $\tau$ (e.g., the 75th percentile of similarity scores) is used to select "seed sentences." This ensures the number of seeds scales naturally with document length and query specificity.
Contextual Window Expansion:
- For each seed sentence, a contextual window is constructed by including $m$ sentences before and after the seed.
- This can be a fixed radius or adaptive, expanding outward until the similarity of boundary sentences drops below a secondary threshold. This ensures seeds are not retrieved in isolation but are accompanied by sufficient context for interpretability.
Chunk-Level Score Aggregation:
- To avoid retrieving isolated high-scoring sentences surrounded by irrelevant noise, QASC computes an aggregate relevance score for each candidate window.
- This score is a weighted mean of the similarity scores of all sentences in the window, where weights decay exponentially with distance from the seed ( $\alpha_i = \exp(-\lambda \cdot |i - r|)$ ).
- Chunks are retained only if their aggregate score exceeds a threshold (e.g., $0.6 \times \tau$ ), ensuring holistic relevance.
Boundary Resolution and Merging:
- Overlapping or adjacent candidate windows are merged iteratively to minimize redundancy.
- Final chunk boundaries are adjusted to align with natural linguistic breaks (paragraphs, sections) to ensure logical flow and self-containment.

The algorithm operates at query time, meaning chunking is performed dynamically for each specific query rather than pre-computed during indexing.

3. Key Contributions

The paper outlines four primary contributions:

Algorithm Design: A query-adaptive algorithm that uses cosine similarity to identify seed sentences and expands contextual windows to form relevance-optimized chunks.
Scoring Mechanism: A chunk-level aggregation mechanism that evaluates the holistic relevance of a passage rather than relying on individual sentence scores, reducing the risk of context-less retrieval.
Coherence Protocol: A merging and boundary resolution protocol that handles overlapping windows and ensures smooth transitions, preserving document-level coherence.
Comprehensive Evaluation: A rigorous empirical evaluation against seven baselines (including five fixed granularities, recursive splitting, semantic chunking, and agentic chunking) across 100 technical documents and 200 diverse queries.

4. Experimental Results

The evaluation was conducted on a corpus of 100 technical documents (academic, software, and domain reports) using 200 queries across four types: Factoid, Topical, Comparative, and Multi-hop.

Performance Metrics: QASC achieved an F1-score of 0.85, representing:
- An 18–27% relative improvement over fixed chunking baselines (best fixed: 0.72).
- An 8–12% relative improvement over semantic and agentic chunking baselines (semantic: 0.76; agentic: 0.74).
Faithfulness: QASC achieved the highest faithfulness score (0.87), indicating a significant reduction in hallucinated or unsupported claims compared to baselines. This is attributed to the higher relevance density of the retrieved chunks.
Latency: QASC introduced additional latency (380 ms) compared to fixed chunking (110–120 ms) due to query-time processing, but remained comparable to embedding-based semantic chunking (340 ms) and well within sub-second interactive thresholds.
Human Evaluation: Three independent annotators (Cohen's $\kappa = 0.82$ ) rated QASC significantly higher than all baselines in Relevance (4.4/5), Coherence (4.2/5), Completeness (4.3/5), and Answer Quality (4.3/5).
Robustness: QASC demonstrated consistent relative improvements (+9 F1 points) across documents of varying complexity (low, medium, high) and query types.

5. Significance and Claims

The paper claims that QASC represents a paradigm shift from "chunk then retrieve" to "query then chunk then retrieve."

Resolution of Trade-offs: By shifting the relevance-filtering function from the retrieval stage to the chunking stage, QASC resolves the inherent tension between precision and recall in fixed chunking. It pre-filters candidates so the retriever ranks "pre-selected needles" rather than searching for needles in a haystack of irrelevant material.
Contextual Alignment: The method aligns document processing with the fundamental purpose of information retrieval: delivering the right information, in the right context, for the right question.
Practicality: While introducing query-time latency, the paper argues this is a necessary trade-off for high-quality generation. It suggests hybrid architectures (pre-chunking for initial retrieval, followed by QASC re-chunking on top candidates) as a viable path for production scalability.

The authors conclude that chunking should not be treated as a preprocessing afterthought but as a critical, first-class design decision in RAG systems. By integrating user intent directly into segmentation, QASC significantly improves retrieval quality, generation accuracy, and system faithfulness.