Technical Summary: ReFact – Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning

1. Problem Statement

Large Language Models (LLMs) increasingly rely on long-form reasoning to solve complex tasks involving long-context inputs. However, existing models often struggle to align their reasoning processes with provided contextual evidence, particularly when evidence is incomplete, noisy, or conflicts with the model's parametric knowledge. This misalignment leads to "unfaithful" Chains of Thought (CoT), where models overlook relevant context, rely on internal knowledge that contradicts the input, or generate hallucinated reasoning steps.

Current approaches to grounding reasoning generally fall into two categories, both of which have limitations:

1. Citation-Based Generation: These methods append citations after generation or during reasoning to improve transparency. However, they often treat citations as external references rather than integral components of the reasoning process, failing to ensure the cited information is sufficient to support intermediate inferences.
2. Fact-Reproducing Reasoning: These methods require LLMs to explicitly restate supporting facts from the context. While this improves alignment, rigid requirements for restatement often compromise the compactness and coherence of reasoning trajectories, leading to redundant, over-cited traces that do not adaptively leverage evidence.

The core challenge is enabling LLMs to determine when contextual grounding is necessary and how to restate source facts at an appropriate level of detail to ensure reliable, compact, and faithful reasoning.

2. Methodology: The ReFact Framework

ReFact (Adaptive Fact Restatement) is a framework designed to enable LLMs to selectively restate source facts at varying levels of granularity (entities, phrases, sentences, or paragraphs) only when necessary for a specific reasoning step. The framework employs a two-stage training pipeline: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL).

2.1 Training Data Construction

The authors construct a dataset ($D_{training}$) by synthesizing adaptive citation-based reasoning trajectories from existing QA datasets (HotpotQA_CARE and MuSiQue).

• Teacher Model: A stronger teacher LLM (Gemini-3.1-preview) generates high-quality reasoning trajectories. It is prompted to decide when to cite and how to restate facts (granularity and form) based on "Supporting Facts" provided in the context.
• Length-Diverse Contexts: To ensure generalization across long-context scenarios, the gold context is inserted into distractor documents of varying lengths (32k, 64k, 128k tokens), simulating real-world long-context challenges.
• Quality Filtering: Generated trajectories are filtered using four criteria:
  1. Format: Correct use of tags (<evidence N>...</evidence>) and separation of reasoning from answers.
  2. Accuracy: The final answer matches the gold standard.
  3. Traceability: All cited content must exist verbatim in the supporting facts.
  4. Answerability: A verifier model must be able to derive the correct answer using only the cited content, ensuring citations are sufficient.

2.2 Two-Stage Training Pipeline

The student model (Qwen3-4B and Qwen3-8B) is optimized through:

1. Cold-Start SFT: The model is trained on the curated $(question, long\_context, reasoning\_trajectory)$ pairs to learn the output format and the behavior of grounding reasoning in facts.
2. Group Relative Policy Optimization (GRPO): The model is further optimized using a reward function ($R$) that balances four components:
   • $R_{format}$: Structural correctness.
   • $R_{acc}$: Answer correctness.
   • $R_{trace}$: Faithfulness of cited content to the source.
   • $R_{ans}$: Sufficiency of cited content to answer the question.
     The final reward is a weighted sum: $R = \lambda_1 R_{format} + \lambda_2 R_{acc} + \lambda_3 R_{trace} + \lambda_4 R_{ans}$.

3. Key Contributions

• Adaptive Citation Mechanism: Unlike prior work that enforces fixed citation positions or post-hoc attribution, ReFact teaches models to adaptively determine the necessity and granularity of fact restatement. This results in reasoning traces that are denser and more faithful without unnecessary redundancy.
• SFT-to-RL Framework: The paper proposes a specific training pipeline that moves from learning citation formats (SFT) to optimizing the utility of citations for reasoning (RL), ensuring that cited facts are not just present but sufficient for inference.
• Compact and Faithful Reasoning: The approach demonstrates that improving faithfulness does not require longer reasoning traces; in fact, ReFact produces significantly shorter traces while maintaining or improving accuracy.

4. Experimental Results

Experiments were conducted on LongBench v1/v2, LV-Eval (16k–128k contexts), and ConFiQA (counterfactual faithfulness).

• Performance on Long-Context QA: ReFact consistently outperformed baselines (Zero-Shot, LongAlign, LongFaith, CARE) on LongBench v1 and LV-Eval. It achieved higher F1 scores across different context lengths (up to 128k) while generating substantially fewer reasoning tokens (e.g., reducing token usage by ~60-70% compared to CARE).
• Citation Quality and Efficiency: Analysis showed ReFact achieved higher citation F1 scores (better alignment with gold evidence) while restating significantly fewer facts per trace compared to Zero-Shot and CARE. This indicates the model learns to select only answer-sufficient evidence.
• Counterfactual Faithfulness: On ConFiQA, where context contradicts parametric knowledge, ReFact demonstrated higher Source Preference (PS) and lower Parametric Override (PO). It was more likely to follow the provided context over memorized facts.
• Mechanism Analysis:
  • Perplexity: ReFact traces yielded lower perplexity for gold answers, indicating stronger guidance for the final prediction.
  • FFN Activation: Analysis of Feed-Forward Network (FFN) activations revealed that ReFact exhibits lower activation ratios in mid-to-deep layers compared to Zero-Shot. Since these layers are associated with parametric memory retrieval, this suggests ReFact relies less on internal knowledge that conflicts with the context.
• Ablation Studies: Removing the "Answerability" reward or "Evidence Alignment" supervision led to performance degradation, confirming that optimizing for sufficiency of evidence and explicit alignment is crucial.

5. Significance and Claims

The paper claims that ReFact provides a significant step forward in making LLM reasoning both compact and faithful. By transforming citations from external references into adaptive intermediate reasoning states, ReFact enables models to:

1. Reduce Cognitive Overload: By restating only necessary facts, the reasoning process remains concise.
2. Enhance Trustworthiness: The model is less prone to hallucination and parametric override, as it is explicitly trained to verify that its reasoning steps are supported by the provided context.
3. Improve Efficiency: The reduction in reasoning tokens suggests that high-quality reasoning does not require verbose, redundant restatements, making long-context reasoning more computationally efficient.

The authors conclude that the key to faithful long-context reasoning lies not in forcing more citations, but in teaching models to adaptively select and integrate the right facts at the right granularity.