Factuality on Demand: Controlling the Factuality-Informativeness Trade-off in Text Generation

Technical Summary: Factuality-Controlled Generation (FCG)

1. Problem Statement

Large Language Models (LLMs) possess internal knowledge with varying degrees of reliability, creating an inherent trade-off between factuality (accuracy) and informativeness (detail). In high-stakes domains (e.g., medical, legal), users require conservative, highly factual outputs, whereas creative or brainstorming tasks often prioritize rich content despite potential factual risks.

Current frontier LLMs lack a built-in mechanism to control this trade-off. While users can attempt to guide behavior via prompts (e.g., "be more factual"), empirical findings show that these models do not reliably adjust their outputs to meet specific factuality targets. Existing evaluation benchmarks like FactScore reveal that off-the-shelf models often fail to satisfy even moderate-to-strict factuality constraints, necessitating a controllable alternative that allows users to specify a desired factuality level.

2. Methodology: Factuality-Controlled Generation (FCG)

The authors propose Factuality-Controlled Generation (FCG), a framework where an LLM takes a query and a numeric factuality constraint $c \in [0, 1]$ as input, generating a response where the proportion of supported factual statements meets or exceeds $c$, while maximizing informativeness.

A. Synthetic Data Generation Pipeline
Since no existing labeled dataset exists for (question, factuality constraint, gold response) triples, the authors introduce a confidence-guided synthetic pipeline to construct training data:

1. Initial Generation: A base model generates an unconstrained, detailed response ($r_0$) for a given question.
2. Fact Segmentation: The response is segmented into atomic factual statements.
3. Confidence Scoring: The model is prompted to evaluate the confidence of each atomic fact (e.g., "Is this True or False?"), deriving a probability score.
4. Iterative Pruning: Facts are ranked by confidence. To meet a target constraint $c$, the system progressively removes the lowest-confidence facts until the remaining subset satisfies the factuality threshold.
5. Reconstruction: The remaining high-confidence facts are merged back into a fluent response, preserving the original phrasing where possible.
   This process is repeated across a range of $c$ values (e.g., 0.1 to 1.0) to create aligned answer sets for supervised fine-tuning.

B. Training and Inference
The authors fine-tune a Mistral-7B model on the synthetic $(question, c, response)$ triples. During inference, the model is conditioned on the user-specified factuality level $c$, allowing it to dynamically adjust its output to meet the constraint without requiring iterative filtering or post-hoc re-ranking.

C. Evaluation Metrics
Performance is evaluated on two axes using FactScore:

• Factuality Adherence: The proportion of generations where the factuality score $f(o) \ge c$.
• Informativeness: The number of validated atomic statements in the output, normalized by length.

3. Key Contributions

• Task Definition: Introduction of FCG as a controllable generation task conditioning LLM outputs on a user-specified factuality threshold.
• Synthetic Training Framework: Development of a novel pipeline to generate training data by systematically pruning low-confidence facts from unconstrained generations, enabling supervised learning of the factuality-informativeness trade-off.
• Empirical Validation: Demonstration that fine-tuning on this synthetic data significantly outperforms prompting-based baselines in adhering to factuality constraints.

4. Results

Experiments were conducted on a biography generation task using 500 entities (450 training, 50 validation, 183 test).

• Factuality Adherence:

  • Prompting Baselines: Standard prompting (Factuality-Controlled Inference, FCI) failed to improve adherence; in some cases (e.g., $c=0.9$), adherence dropped compared to the unconstrained baseline (3.8% vs. 5.5%).
  • FCG Training: The fine-tuned model showed substantial improvements. Adherence increased by 17.2% at $c=0.8$ and 130.0% at $c=0.9$ relative to the baseline.
  • Strict Constraints: At the strictest level ($c=1.0$, requiring all information to be correct), the baseline and FCI methods achieved 0% adherence, whereas the FCG model achieved 23.6%.

• Factuality–Informativeness Trade-off:

  • The FCG model shifts the trade-off frontier outward. For any given factuality level, FCG produces responses with higher informativeness (more supported facts) than the baselines.
  • Notably, at $c=1.0$, the FCG model provided non-empty, fully factual responses, whereas the inference baseline effectively had to prune all content to avoid hallucination, resulting in zero informativeness.

5. Significance and Claims

The paper claims that FCG provides a reliable mechanism for users to customize the factuality-informativeness balance of LLM outputs according to specific application needs. By internalizing the trade-off through supervised training on synthetic data, the model learns to "drop the least-certain details but keep everything else" when high factuality is requested.

The authors position this work as a step toward more reliable and flexible LLMs, particularly for knowledge-intensive tasks. They emphasize that their method improves controllability without the computational overhead of post-hoc filtering or rejection sampling used in other frameworks.

Limitations Acknowledged:

• The study is currently limited to English and biography generation tasks.
• Experiments were conducted on a mid-sized model (Mistral-7B); scalability to larger state-of-the-art architectures remains untested.
• The framework's applicability to other languages and complex generative tasks (e.g., multimodal understanding) requires further investigation.