Technical Summary: SeDT – Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability

1. Problem Statement: The "Lost in Conversation" Phenomenon

Large Language Models (LLMs) exhibit a significant performance collapse when tasks are revealed incrementally across multiple turns compared to being fully specified in a single turn. This phenomenon, documented as "Lost in Conversation" (Laban et al., 2025), results in an average performance drop of 39% across state-of-the-art models.

Crucially, the paper argues that this collapse is primarily a reliability failure rather than a capability failure. While the best-case aptitude (90th percentile) drops by only 16%, unreliability (the gap between the 90th and 10th percentiles) more than doubles (+112%).

The root cause is identified as structural:

• Flat Context Weighting: Multi-turn conversations are concatenated into a flat context window where every prior turn carries equal implicit weight.
• Attention Bias: Transformer models systematically neglect middle-context information (the "U-shaped" attention bias), causing critical constraints revealed in the middle of a conversation to be treated as noise.
• Failure Modes: This leads to premature commitment to answers, over-reliance on incorrect intermediate responses, loss of middle-turn constraints, and verbose drift.

Existing solutions, such as fine-tuning, are impractical at inference time due to the need for curated data and weight modification. Other context management strategies like "recapitulation" (repeating history) often fail to distinguish critical constraints from incidental dialogue.

2. Methodology: SeDT

The authors propose SeDT (Sentence-transformer Decision-Transformer conditioning), a training-free, inference-time method that resolves flat-context weighting by importing Return-to-Go (RTG) conditioning from offline reinforcement learning.

2.1 The Formal Parallel

SeDT draws a direct analogy between the Decision Transformer (Chen et al., 2021a) and multi-turn LLM inference:

• Trajectory: The conversation history.
• Step: Each information shard revealed in a single turn.
• Reward: The semantic relevance of that shard to the final output goal.
• Return-to-Go (RTG): Instead of leaving the model to infer importance, SeDT annotates each shard with a cumulative relevance score ($\hat{R}_t$), representing the total goal-relevant information still to come from that turn onward.

The input to the model at the final turn is transformed from a flat list of turns into a sequence of annotated tuples: $[(\hat{R}_1, \text{shard}_1), \dots, (\hat{R}_T, \text{shard}_T)]$.

2.2 Three-Signal Relevance Scoring

To calculate the relevance of each shard ($rel(t)$), SeDT combines three complementary signals:
$$rel(t) = \alpha \cdot sem(t) + \beta \cdot lex(t) + \gamma \cdot pos(t)$$
Where weights are empirically set to $\alpha=0.6, \beta=0.2, \gamma=0.2$.

1. Semantic Signal ($sem$): Deep alignment between the shard and a task-typed "goal anchor" (e.g., "Write a complete Python function"). Computed via cosine similarity of dense vector embeddings (using all-mpnet-base-v2).
2. Lexical Signal ($lex$): Surface keyword overlap (Jaccard similarity) between the goal anchor and the shard, capturing domain-specific keywords that embeddings might miss.
3. Positional Signal ($pos$): An inverted-U boost designed to counteract the transformer's natural tendency to neglect the middle of the context. It assigns the highest score to the middle turn and decreases symmetrically toward the start and end.

2.3 RTG Annotation and Prompt Construction

• Cumulative Scoring: The RTG for a turn is the suffix sum of relevance scores from that turn to the end of the conversation.
• Last-Shard Guarantee: A minimum guarantee is applied to the final shard to ensure the last constraint is not discarded, even if its raw relevance is low.
• Confidence Labels: Shards with relevance above the mean are marked CONFIRMED; others are marked UNCERTAIN. These labels are appended to the history presented to the model.

2.4 RTG-Guided Self-Correction

SeDT includes a two-guard verification mechanism:

1. Trigger: If the conversation is deemed "weak" (based on the normalized relevance of the final shard and the fraction of uncertain turns), a verification step is triggered.
2. Acceptance Guard: A verifier model generates a new answer. This answer replaces the original only if it covers more "CONFIRMED" shards (content words) than the original answer. This prevents stochastic overwriting of correct answers.

3. Key Contributions

1. Problem Identification: Identifying flat-context turn weighting as the structural root cause of multi-turn reliability collapse, establishing a formal parallel to Decision Transformer RTG conditioning.
2. SeDT Method: A training-free inference-time method requiring zero model modification and zero training data.
3. Three-Signal Relevance: A formulation combining semantic, lexical, and positional signals to directly counteract documented failure modes (premature commitment, middle-turn loss, etc.).
4. RTG-Grounded Self-Correction: A conservative verification pathway that introduces zero "hurt cases" (errors introduced by the correction mechanism).

4. Experimental Results

The method was evaluated on the Lost-in-Conversation benchmark (Laban et al., 2025) across three LLMs (GPT-4o-mini, Gemini 2.5 Flash, Llama 3.3-70B) and three tasks (Actions, Math/GSM8K, Code).

• Performance Gains: SeDT outperformed the sharded baseline in all nine model-task combinations.
  • Mean Performance ($\bar{P}$): Gains ranged from +5.6% to +37.7%. Gemini 2.5 Flash showed the largest single gain (+37.7 in Actions), recovering significantly toward its single-turn performance ceiling.
  • Reliability: Unreliability ($U$) was reduced in seven of the nine combinations. The method successfully tightened score distributions, particularly in Math and Actions tasks.
• Aptitude: The best-case capability (Aptitude $A_{90}$) improved in all cases, confirming the gains reflect genuine capability recovery rather than noise reduction.
• Ablation Studies:
  • No single signal (semantic, lexical, or positional alone) was sufficient; the combination was necessary.
  • The specific weighting ($\alpha=0.6$) was validated as meaningful, outperforming uniform weighting.
  • "Random RTG" (using random numbers instead of calculated relevance) performed better than the baseline but significantly worse than SeDT, proving that the calculated relevance signal, not just the numerical prefix format, drives performance.
• Self-Correction: The mechanism triggered in ~28% of runs but resulted in zero hurt cases, only replacing answers when the new answer demonstrably covered more confirmed constraints.

5. Significance and Claims

The paper claims that the "Lost in Conversation" problem is at least partially a problem of context weighting, which can be addressed at inference time without retraining.

• Structural Insight: The work shifts the blame from model "unreliability" or "intent misalignment" to the structural representation of conversation history.
• Practicality: By requiring no fine-tuning and at most one additional LLM call (for verification), SeDT offers a scalable solution for real-world multi-turn interactions where users reveal constraints incrementally by necessity.
• Mechanism: The success of RTG conditioning suggests that instruction-tuned LLMs have learned to respond to explicit importance signals (like numerical prefixes found in code comments or outlines) during pretraining, allowing them to leverage this "implicit in-context demonstration" to re-weight their attention toward critical constraints.

The authors conclude that "telling the model which past turns matter is sufficient to substantially recover the performance lost in conversation."