Technical Summary: FullCite – Explicit Evidence Grounding via Structured Inline Citation Generation

Problem Statement

As Large Language Models (LLMs) increasingly replace traditional search engines by providing direct answers, the demand for factual and faithful generation has grown, particularly in high-stakes domains like medicine, law, and science. While Retrieval-Augmented Generation (RAG) allows models to ground outputs in external documents, the mere presence of retrieved context does not guarantee faithful usage. Models often rely on parametric knowledge, incorrectly attribute claims to irrelevant documents, or produce outputs only partially supported by evidence.

Existing solutions generally fall into two categories:

1. Document-level citations: Coarse-grained attributions that reference a source document but fail to identify specific supporting evidence, potentially citing documents that contain related but non-supporting information.
2. Fine-grained citations: Approaches that reference specific evidence spans, but often lack simultaneous document attribution or struggle with strict structural adherence.

There is a critical gap in frameworks that simultaneously optimize for both document-level attribution and precise evidence span identification while maintaining semantic faithfulness.

Methodology: The FullCite Framework

The authors introduce FullCite, a framework designed to generate structured inline citations linking every claim in an answer to both its source document and a verbatim supporting evidence span. The citation format is strictly defined as {doc_id: <identifier>, snippet: <verbatim text>}.

FullCite evaluates three distinct strategies for generating these structured citations:

1. Prompt-based Generation: The model is instructed via natural language prompts to provide verbatim evidence citations after each claim. Compliance relies entirely on the model's instruction-following capabilities without decoding-time intervention.
2. Constrained Decoding: A finite-state automaton (FSA) enforces the citation grammar during inference. The automaton tracks the generation state (claim, document ID, or snippet) and restricts the next token to those consistent with the grammar and the verbatim content of the context. If the model fails to follow the structure, it retries up to three times with increased temperature.
3. Posthoc Span Alignment: The model generates text freely. Subsequently, the system attempts to approximate the cited evidence by finding the most similar snippet in the source document (given a correctly generated document ID) using word-level Jaccard similarity. A similarity threshold of 0.7 is used to reconstruct snippets that do not match word-for-word.

Experimental Setup

The framework was evaluated on three benchmarks covering diverse domains and question types:

• BioASQ: Biomedical domain (yes/no, factoid, summary, list).
• ASQA: General domain factoid questions.
• ExpertQA: Multi-domain (32 domains) questions with expert-annotated sources.

Since ASQA and ExpertQA originally lacked fine-grained evidence span annotations, the authors applied an additional annotation step using GPT-5.4-mini to extract atomic evidence spans, validated against BioASQ's gold annotations (achieving >90% ROUGE-L).

Experiments utilized two open-weight LLMs: Qwen3-8B and Gemma-3-12B-it. The evaluation measured performance across three dimensions:

1. Document-level correctness (Doc-F1): Accuracy of citing the correct source document.
2. Evidence span identification (Snippet-F1): Accuracy of identifying the precise supporting text.
3. Claim-citation faithfulness: Semantic similarity between the claim and the cited span, evaluated via sentence transformers, LLM-as-a-Judge (GPT-5.4), and human annotation.

Key Results

1. Document vs. Span Localization

The results reveal a significant disparity in model capabilities. LLMs are generally effective at identifying relevant documents (high Doc-F1 scores) but struggle to pinpoint the precise supporting spans within them (low Snippet-F1 scores).

• Baseline (Prompt-based): Achieved high Doc-F1 (e.g., 58.08 on BioASQ) but very low Snippet-F1 (6.18 on BioASQ).
• Posthoc Strategy: Showed the most substantial gains in span identification. For ASQA, Snippet-F1 improved from 12.80 (prompt-based) to 61.87 (Posthoc) using Qwen3-8B. However, this often involved a trade-off where document coverage decreased slightly in exchange for higher span precision.
• Constrained Decoding: Provided moderate improvements in span identification but was less effective than the Posthoc approach in some settings.

2. Faithfulness and Semantic Alignment

There is a noted trade-off between strict span localization and semantic faithfulness.

• Qwen3-8B achieved its highest semantic similarity scores under the baseline prompt-based setting, with scores declining under Posthoc and Constrained settings.
• Gemma-3-12B-it demonstrated more stability, with similarity scores often preserved or improved under Posthoc and Constrained settings.
• Human/LLM Evaluation: Both human annotators and GPT-5.4 rated citations highly for "Support" and "Relevance," with the Posthoc strategy generally achieving the strongest performance. Disagreements between human and LLM evaluators were primarily adjacent scores (e.g., 1 vs. 2).

3. Systematic Biases and Failures

The analysis uncovered two systematic patterns undermining evidence grounding:

• Primacy Bias: In BioASQ, 81.8% of citations targeted only the first two of five context documents, consistent with the "lost-in-the-middle" phenomenon.
• Citation Omission: Models frequently omitted citations entirely for binary yes/no questions, artificially inflating baseline scores. Once attribution was enforced, these scores dropped, revealing that the baseline success was due to a lack of grounding rather than superior performance.

4. Downstream Impact

In downstream QA tasks, the Posthoc FullCite setting yielded the strongest overall results, particularly for BioASQ (average gain of 19.85 points). While prompt-based baselines scored higher on yes/no questions, this was attributed to citation omission. For factoid, summary, and list questions, grounding significantly improved performance, especially for short, verifiable answers.

Significance and Claims

The paper claims that FullCite is the only framework that simultaneously optimizes for both document-level and evidence span attribution while maintaining competitive semantic similarity scores.

• Joint Coverage: Unlike previous works that focus on either document references or evidence snippets, FullCite provides a balanced approach to faithful attributed QA.
• Span Identification Gap: The work highlights that the primary bottleneck in faithful attribution is not document retrieval, but the precise identification of evidence spans.
• Methodological Insight: The study demonstrates that enforcing strict structure (via constrained decoding) or approximating spans post-generation (Posthoc) yields better evidence grounding than relying solely on prompt-based generation, though this may come at the cost of some semantic alignment depending on the model.

The authors conclude that achieving faithful attributed QA requires a shift in research emphasis from document-level retrieval to precise evidence span identification and evaluation methods that can distinguish surface similarity from genuine support.

Limitations

The authors acknowledge several limitations:

• Domain Scope: Findings are based on biomedical, general, and multi-domain datasets and may not transfer to legal or financial domains.
• Model Scope: Experiments were limited to two open-weight models; patterns may differ for larger or proprietary models.
• Evaluation Metrics: Automatic metrics (Doc-F1, Snippet-F1) are proxies and do not fully capture entailment; human evaluation was limited in scale.
• Unaddressed Biases: The framework does not explicitly mitigate the primacy bias or the tendency to omit citations for binary questions.
• Training: The study did not explore fine-tuning models specifically for joint document-and-span citation generation.