Technical Summary: The Topology of Ill-Posed Questions

Problem Statement

Large Language Models (LLMs) frequently encounter ill-posed questions—queries that are ambiguous, underspecified, contradictory, or lack necessary context. Unlike well-posed problems, these queries may admit no valid answer or multiple plausible interpretations. Existing approaches to handling ill-posedness largely rely on analyzing model outputs (e.g., detecting refusal patterns) or focusing on specific subclasses of ambiguity through input-output rewriting. These methods often treat the LLM as a black box, failing to capture the unified internal representation of diverse ill-posedness sources. Consequently, current steering mechanisms often collapse into generic refusal templates rather than guiding the model toward source-aware clarification or targeted abstention.

Methodology

The paper proposes a framework that analyzes and steers LLM behavior by modeling the internal hidden states of prompt tokens as a topological structure.

1. Layer-wise Token Cloud Representation

Instead of compressing a prompt into a single pooled vector, the authors treat the contextual hidden states of prompt tokens at each transformer layer $l$ as a point cloud $X^{(l)}(q)$.

• Dimensionality Reduction: To manage noise and computational cost, token states are projected into a lower-dimensional space using Principal Component Analysis (PCA) fitted on the training set.
• Topological Analysis: The geometry of these token clouds is analyzed using finite zero-dimensional persistent homology ($H_0$). This method tracks how initially disconnected token components merge as a distance threshold (filtration radius) increases.
• Interpretation: A well-posed question induces token states that coalesce into a coherent structure early in the filtration. An ill-posed question preserves separated or unevenly connected groups (representing unresolved entities, contexts, or constraints) until larger scales, resulting in distinct merge patterns.

2. Compact Topological Descriptors

For each layer, the finite $H_0$ persistence diagram is summarized into three fixed-dimensional scalar descriptors:

1. Mean Finite Lifetime ($\mu$): The average scale at which components merge, indicating the degree of fragmentation.
2. Normalized Lifetime Entropy ($E$): Measures the distribution of merge events (diffuse vs. concentrated).
3. Largest-Lifetime Concentration ($R$): The ratio of the sum of the five largest lifetimes to the total, capturing the dominance of major separations.

These descriptors are concatenated across all $L$ layers to form a topology vector $z(q) \in \mathbb{R}^{3L}$ representing the question.

3. Topology-Conditioned Activation Steering

The framework utilizes these topology vectors for two purposes:

• Classification: A supervised classifier predicts binary ill-posedness or fine-grained subtypes (e.g., polysemy, missing temporal context) directly from the topology vector.
• Local Steering: Rather than applying a single global "abstention" direction, the method retrieves topologically similar positive (clarification/abstention) and negative (direct answer) examples from the training set.
  • A local steering vector $v_{local}$ is constructed as the difference between the mean activations of these retrieved neighbors.
  • This vector is added to the hidden states during the prefill and decoding stages, guiding the model toward a response that addresses the specific source of ill-posedness while preserving the original question's semantic content.

Key Contributions

1. Unified Topological Representation: Introduced a layer-wise token-cloud representation that quantifies the multiscale connectivity of ill-posed questions using finite $H_0$ persistent homology.
2. Compact Feature Extraction: Demonstrated that three compact statistics (mean lifetime, entropy, concentration) derived from $H_0$ lifetimes form a robust, low-dimensional representation capable of supporting both binary detection and fine-grained 9-way classification of ill-posedness.
3. Topology-Conditioned Steering: Developed a steering mechanism that uses topological similarity to select query-specific contrast pairs, enabling the model to shift from direct answering to source-aware abstention or clarification without collapsing into generic refusals.

Experimental Results

The framework was evaluated on three open-weight LLMs (Gemma-7B-it, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3) across three benchmarks: AmbigQA and SituatedQA (binary detection) and CLAMBER (9-way fine-grained classification).

• Classification Performance:

  • AmbigQA: Accuracy improved from 67.4% (AEN baseline) to 78.9%.
  • SituatedQA: Accuracy improved from 79.9% to 88.5%.
  • CLAMBER (9-way): Accuracy improved from 57.6% to 69.6%.
  • The topology features consistently outperformed prompt-based baselines (zero-shot, few-shot, CoT) and neuron-probing baselines (AEN).

• Steering Performance:

  • Topology-conditioned steering increased the Grounded Acceptable response rate (responses that correctly identify and address the specific missing/ambiguous element) from 11.9% to 16.4% on average.
  • The Total Acceptable rate (grounded + generic) increased from 61.4% to 70.6%.
  • Crucially, the method reduced "generic" refusals in favor of "grounded" responses, indicating that the steering successfully preserved question-specific information while altering the response behavior.

Significance and Claims

The paper claims that persistent homology provides an interpretable representation of ill-posedness that captures how the model's internal connectivity structure transforms across layers. The significance lies in demonstrating that:

1. Ill-posedness is not merely an output-level phenomenon but is encoded in the geometric arrangement of token states during the prefill stage.
2. Topological descriptors can serve as effective retrieval keys for local activation steering, allowing for targeted interventions that are sensitive to the specific subtype of ill-posedness (e.g., distinguishing between lexical ambiguity and missing temporal context).
3. This approach moves beyond fixed refusal templates, enabling LLMs to produce responses that are both abstinent (when necessary) and informative regarding the specific nature of the query's ambiguity.

The authors note limitations, including the scope restricted to three instruction-tuned models and three QA benchmarks, and suggest future work could explore higher-dimensional homology ($H_1$) or learn layer-aggregation weights.