Disco-RAG: Discourse-Aware Retrieval-Augmented Generation

1. Problem Statement

Retrieval-Augmented Generation (RAG) has become a standard approach for enhancing Large Language Models (LLMs) in knowledge-intensive tasks by integrating external evidence. However, existing RAG systems suffer from a fundamental limitation: they treat retrieved documents as flat, unstructured sequences of chunks.

This "flat" approach leads to two critical issues:

• Intra-chunk Structural Blindness: The model fails to capture internal logical hierarchies within a single text chunk (e.g., distinguishing between a main claim and supporting evidence).
• Inter-chunk Coherence Gaps: The model struggles to identify rhetorical connections between different chunks. Consequently, when evidence is dispersed or conflicting (e.g., one chunk says "Vitamin D helps," while another says "No overall effect"), standard RAG tends to overgeneralize or hallucinate because it cannot synthesize the evidence into a coherent causal flow.

The paper argues that without modeling discourse structure (the logical and rhetorical organization of text), RAG systems cannot effectively resolve conflicting claims or synthesize complex information.

2. Methodology: Disco-RAG

The authors propose Disco-RAG, a framework that explicitly injects discourse knowledge into the RAG pipeline. Instead of simply concatenating retrieved chunks, Disco-RAG constructs a structured representation of the evidence to guide the generation process. The pipeline consists of three main stages:

A. Intra-Chunk RST Tree Construction

For every retrieved chunk, the system constructs a Rhetorical Structure Theory (RST) tree.

• Process: An LLM-based parser segments the text into Elementary Discourse Units (EDUs) and assigns rhetorical relations (e.g., Elaboration, Contrast, Cause, Evidence) between them.
• Structure: It identifies Nucleus (core) and Satellite (supplementary) units, creating a hierarchical tree that captures the local logical flow of the document.

B. Inter-Chunk Rhetorical Graph Construction

The system builds a global directed graph connecting all retrieved chunks.

• Process: Using a listwise inference strategy, the model analyzes all chunks simultaneously to predict rhetorical relations between pairs (e.g., Chunk A Supports Chunk B, or Chunk C Contradicts Chunk D).
• Structure: This graph serves as a global discourse scaffold, allowing the generator to reason across document boundaries and understand how different pieces of evidence relate to one another.

C. Discourse-Driven Planning

Before generating the final answer, the system creates a discourse-aware plan (blueprint).

• Input: The query, the retrieved chunks, the intra-chunk trees, and the inter-chunk graph.
• Output: A natural language outline that dictates the logical flow of the answer (e.g., "Start with the background, present the conflicting evidence, explain the conditions under which the claim holds, and conclude").
• Role: This plan acts as a high-level reasoning guide, ensuring the final generation follows a coherent argumentative structure rather than a random aggregation of facts.

D. Discourse-Guided Generation

The final generation stage conditions the LLM on four inputs: the raw chunks, the RST trees, the rhetorical graph, and the generated plan. This ensures the output is factually grounded and rhetorically coherent.

3. Key Contributions

1. Novel Framework: Introduction of Disco-RAG, the first RAG framework to explicitly model both local (intra-chunk) and global (inter-chunk) discourse structures to guide generation.
2. Structural Modeling: A method combining RST trees, rhetorical graphs, and discourse-driven planning to capture local hierarchies, cross-passage coherence, and argumentative flow.
3. Training-Free Approach: The method achieves state-of-the-art results without fine-tuning the underlying LLM, relying instead on inference-time structural injection.
4. Comprehensive Evaluation: Extensive experiments on three diverse benchmarks (Loong, ASQA, SciNews) demonstrating consistent improvements over standard RAG and other SOTA methods.

4. Experimental Results

The authors evaluated Disco-RAG on three benchmarks using various LLM backbones (Llama-3.1-8B, Llama-3.3-70B, Qwen2.5-72B).

• Loong Benchmark (Long-context QA):
  • Disco-RAG achieved a 71.00 LLM Score (using Llama-3.3-70B), outperforming the standard RAG baseline (62.78) by 8.22 points.
  • It significantly outperformed previous SOTA methods like StructRAG and GraphRAG, especially in long-document settings (Set 4, 200k–250k tokens).
• ASQA Benchmark (Long-form QA):
  • Disco-RAG improved Exact Match (EM) to 42.0 and ROUGE-L to 42.3, surpassing complex prompting systems like MAIN-RAG and Tree of Clarifications.
• SciNews Benchmark (Summarization):
  • The model achieved new SOTA performance across most metrics, including 21.11 ROUGE-L and 69.48 SummaC (factual consistency), beating prior SOTA systems.
• Ablation Studies:
  • Removing any single component (RST trees, rhetorical graphs, or planning) led to performance degradation.
  • The RST tree and rhetorical graph were found to be the most critical components, particularly in long-document settings.
• Robustness: Disco-RAG demonstrated superior robustness to retrieval noise and varying chunk sizes compared to standard RAG, maintaining stable performance even when 40% of retrieved chunks were irrelevant.
• Human Evaluation: On the SciNews dataset, human raters rated Disco-RAG significantly higher than standard RAG in Faithfulness (2.53 vs. 1.67) and Conciseness, confirming that the discourse structure reduces hallucinations.

5. Significance and Impact

• Paradigm Shift: The paper challenges the prevailing "retrieve-and-concatenate" paradigm in RAG, arguing that structural understanding is as crucial as retrieval accuracy.
• Solving Conflicts: By modeling rhetorical relations (like Contrast or Condition), Disco-RAG can effectively handle conflicting evidence, preventing the model from making false generalizations (e.g., distinguishing between "Vitamin D helps deficient adults" vs. "Vitamin D has no general effect").
• Efficiency vs. Performance: While Disco-RAG introduces moderate inference latency (approx. 2 seconds per query due to parsing and planning), the trade-off yields substantial gains in factual accuracy and coherence, making it highly suitable for knowledge-intensive applications where correctness is paramount.
• Generalizability: The framework is model-agnostic and can be deployed with different LLM sizes for different components (e.g., using a smaller model for parsing and a larger one for generation), offering flexibility for resource-constrained environments.

In conclusion, Disco-RAG demonstrates that explicitly modeling discourse structure transforms RAG from a simple fact-retrieval system into a sophisticated reasoning engine capable of synthesizing complex, multi-document evidence into coherent, faithful, and logically sound answers.