AGSC: Adaptive Granularity and Semantic Clustering for Uncertainty Quantification in Long-text Generation

1. Problem Statement

Large Language Models (LLMs) excel at long-form text generation but suffer from hallucinations (generating inaccurate information with high confidence). While Uncertainty Quantification (UQ) is crucial for assessing reliability, existing methods face three critical challenges when applied to long-text generation:

1. Efficiency-Granularity Trade-off: Current state-of-the-art methods (like LUQ) decompose responses into "atomic facts" to check consistency. While accurate, this creates massive computational overhead due to the sheer volume of NLI (Natural Language Inference) calls required for every sentence.
2. Topic Heterogeneity in Aggregation: Long responses often mix multiple semantic themes. Naive uniform aggregation of uncertainty scores across all sentences allows minor or off-topic sections to disproportionately skew the overall uncertainty estimate.
3. Underutilization of Neutrality: Existing methods often discard "neutral" NLI labels. However, neutrality can signal two distinct states: irrelevance (off-topic noise) or epistemic uncertainty (ambiguous content requiring deeper analysis). Treating them identically leads to information loss or unnecessary computation.

2. Methodology: The AGSC Framework

The authors propose AGSC (Adaptive Granularity and Semantic Clustering), a three-stage framework designed to balance accuracy and efficiency.

Stage 1: Adaptive Granularity Triggering

Instead of decomposing every sentence, AGSC uses NLI signals to dynamically decide the level of analysis:

• NLI Analysis: For each sentence in the anchor response, the model computes the average NLI distribution (Entailment, Contradiction, Neutral) against reference samples.
• Decision Logic:
  • Non-Neutral: If the dominant label is Entailment or Contradiction, the sentence is kept as a unit.
  • Neutral Handling: If the label is Neutral, AGSC calculates the Entailment–Contradiction Gap ($\Delta$): $|\bar{P}(\text{entail}) - \bar{P}(\text{contra})|$.
    • Skip (Noise): If $\Delta \leq \tau$ (threshold), the sentence is deemed irrelevant/off-topic and skipped to save computation.
    • Decompose (Uncertainty): If $\Delta > \tau$, the sentence is deemed ambiguous and is decomposed into atomic facts for fine-grained verification.
• Outcome: This filters out noise and avoids decomposing irrelevant sentences, significantly reducing the number of atomic fact checks.

Stage 2: GMM-Based Semantic Clustering

To address topic heterogeneity, AGSC groups semantic units rather than treating them uniformly:

• Embedding: Valid semantic units (sentences or atomic facts) are encoded into vector embeddings.
• Dimensionality Reduction: UMAP is applied to reduce dimensionality for robustness.
• Soft Clustering: A Gaussian Mixture Model (GMM) is fitted to the embeddings. Unlike hard clustering (e.g., K-Means), GMM assigns soft membership probabilities ($\gamma_{h,k}$) to each unit, reflecting the ambiguity of natural language boundaries.
• Theme-Aware Weights: These probabilities serve as weights, allowing the system to identify dominant themes and down-weight minor or noisy portions during aggregation.

Stage 3: Uncertainty Aggregation

The final uncertainty score is computed by aggregating unit-level uncertainties weighted by their cluster membership:

1. Cluster Mass ($M_k$): The sum of membership probabilities for all anchor units in cluster $k$.
2. Cluster Uncertainty ($U_k$): The mass-normalized average uncertainty of units within cluster $k$.
3. Global Score: A weighted sum of cluster uncertainties, where weights are proportional to the cluster mass. This ensures that dominant themes drive the final score, while minor/noisy themes have minimal impact.

3. Key Contributions

1. Adaptive Granularity Strategy: A novel mechanism using NLI neutrality and the entailment-contradiction gap to dynamically trigger fine-grained decomposition only when necessary, effectively filtering noise.
2. GMM-Based Semantic Clustering: Introduction of soft clustering to model latent semantic themes, enabling topic-aware aggregation that mitigates the influence of off-topic or minor content in long texts.
3. State-of-the-Art Performance: AGSC achieves the highest correlation with factuality benchmarks while reducing inference time by approximately 60% compared to full atomic decomposition methods.

4. Experimental Results

The method was evaluated on two benchmarks: BIO (biography generation) and LongFact (diverse long-context topics), using models like GPT-4.1-mini, Qwen2.5-32B, and Llama3-70B.

• Correlation with Factuality: AGSC achieved the highest Pearson (PCC) and Spearman (SCC) correlation coefficients with FActScore (ground truth factuality) across all models and datasets.
  • Example: On BIO with GPT-4.1-mini, AGSC achieved a PCC of -0.708, outperforming the next best baseline (LUQ-Atomic at -0.572).
• Robustness: AGSC maintained superior performance on the challenging LongFact dataset, where baseline methods struggled due to topic heterogeneity.
• Efficiency: By skipping irrelevant neutral sentences, AGSC reduced total inference time by ~60% compared to LUQ-Atomic, which indiscriminately decomposes all sentences.
• Ablation Studies:
  • Removing the adaptive mechanism caused significant performance drops, proving that heuristic handling of neutral labels is insufficient.
  • Replacing GMM with K-Means (hard clustering) degraded performance, confirming that soft, probabilistic membership is better suited for ambiguous semantic boundaries.

5. Significance and Limitations

Significance:
AGSC provides a scalable solution for UQ in long-form generation. It solves the "efficiency vs. accuracy" dilemma by intelligently selecting what to decompose and how to aggregate, making high-fidelity uncertainty estimation feasible for complex, multi-topic generation tasks.

Limitations:

• NLI Dependency: The adaptive trigger relies on the NLI model's ability to correctly classify "Neutral." Miscalibration in technical domains (e.g., medical/legal) could lead to skipping hallucinated content.
• Echo Chamber Effect: Like all consistency-based methods, AGSC cannot detect systematic hallucinations where the model consistently generates the same false information across all samples.
• Structural Constraints: The method is optimized for descriptive, fact-heavy text. It may struggle with logical reasoning (math/code) where sentence-level units break logical flow.
• Language Bias: Current implementation is English-centric; performance may degrade in low-resource languages due to embedding anisotropy.

In conclusion, AGSC represents a significant step forward in making LLMs more trustworthy for long-form generation by combining adaptive computation with semantic-aware aggregation.