Technical Summary: Context Dependence and Reliability in Autoregressive Language Models

Problem Statement

Autoregressive Large Language Models (LLMs) generate outputs by conditioning on extensive, structured contexts that often contain redundant instructions, repeated facts, retrieved passages, and interaction history. In critical applications (e.g., medical or legal domains), it is essential to identify which specific context elements are truly responsible for an output. However, standard explanation methods—such as attention visualization, gradient saliency, and perturbation-based ablation—struggle with this task in redundant settings.

The core issue is context redundancy. When multiple context elements convey overlapping or identical information, existing methods often misassign credit. They may distribute importance equally among duplicates, suggesting that multiple independent sources were necessary when only one was sufficient. Furthermore, these methods exhibit instability: minor changes in input structure (e.g., reordering retrieved chunks or paraphrasing instructions) can cause attribution scores to shift unpredictably, even if the model's output remains identical. This sensitivity undermines interpretability and creates security risks, such as in prompt injection scenarios where malicious instructions repeated in memory or context might be erroneously credited as influential.

The paper argues that the fundamental failure mode is a lack of dependence awareness. Current methods often measure marginal importance or surface correlations rather than the unique, conditional contribution of a context element given the rest of the prompt.

Methodology: RISE

To address these challenges, the authors propose RISE (Redundancy-Insensitive Scoring of Explanation), a method designed to quantify the unique influence of each input element conditioned on the rest of the context.

Core Concept: Conditional Unique Dependence (CUD)

RISE is grounded in information theory. Instead of measuring how much a context unit $C_i$ correlates with the output $\hat{T}$, it measures the Conditional Unique Dependence (CUD):
$$\Delta_i \triangleq I(C_i; \hat{T} \mid C_{\setminus i})$$
where $I(\cdot)$ denotes conditional mutual information, $C_i$ is a specific context unit (e.g., a system instruction, a retrieved chunk, or a dialogue turn), and $C_{\setminus i}$ represents all other context units. This metric quantifies the information $C_i$ provides about the generation that cannot be recovered from the remaining context.

The RISE Score

To enable comparison across units, the CUD values are normalized into a RISE score:
$$\text{RISE}(C_i) = \frac{\text{CUD}(C_i; \hat{T} \mid C_{\setminus i})}{\sum_{j=1}^m \text{CUD}(C_j; \hat{T} \mid C_{\setminus j}) + \epsilon}$$
This formulation ensures that:

1. Redundancy Suppression: If a unit is a duplicate of another (or fully recoverable from the rest), its conditional mutual information is zero, and it receives zero attribution.
2. Stability: Adding redundant context does not alter the attribution scores of the original, informative units.
3. Context-Level Granularity: RISE operates on structured context units (e.g., prompt segments, RAG chunks) rather than individual tokens, making it suitable for auditing complex pipelines.

Implementation Variants

The paper outlines two implementations of this principle:

1. RISE-Lite: A lightweight, model-agnostic scoring module that computes dependence-aware scores for auditing and monitoring without modifying the model.
2. Integrated Learned Selector: A trainable gating network that uses the dependence-aware principle to select a sparse subset of context units (retaining a mandatory recent window and top-$K$ unique anchors) during inference. This reduces input dimensionality while preserving essential information.

Key Contributions

1. Identification of Redundancy as a Failure Mode: The authors identify that standard explanation methods fail to distinguish between essential context and mere repetition, leading to "over-crediting" of distractors and instability under paraphrasing or reordering.
2. Dependence-Aware Attribution: The paper proposes conditional information as a principled foundation for reliable attribution, introducing RISE to evaluate the importance of structured context units based on their unique contribution.
3. Empirical Validation: Through controlled experiments on redundant prompting and Retrieval-Augmented Generation (RAG), the authors demonstrate that RISE produces significantly more stable and robust explanations than traditional perturbation-based methods, specifically by reducing redundancy-driven inflation of importance.

Experimental Results

The authors evaluated RISE against attention-based heuristics and perturbation-based baselines (e.g., LOUO) using GPT-2 and open-weight autoregressive models on the SQuAD v1.1 dataset and synthetic redundancy tests.

• Redundancy Sensitivity: Baseline methods showed high "Dup-Split" (fragmentation of credit across duplicates) and sensitivity to surface-form changes. In contrast, RISE maintained low Dup-Split, effectively suppressing credit for redundant units.
• Stability vs. Faithfulness: While attention methods showed high rank stability (due to diffusion of credit across duplicates) and perturbation methods showed high stability (often collapsing onto structurally necessary units like the question), RISE prioritized redundancy invariance. It intentionally reallocated credit away from redundant copies toward uniquely informative units.
• Structural Dominance: Perturbation-based methods were found to collapse onto the "Question" unit because removing it disrupts generation regardless of semantic evidence. RISE-Lite avoided this by conditioning on the remaining context, correctly identifying informative evidence even when the question was present.
• Faithfulness: In removal tests, RISE demonstrated a positive faithfulness gap, indicating that removing top-ranked units significantly reduced the likelihood of the correct answer, whereas attention-based methods showed negative gaps.

Significance and Claims

The paper claims that dependence awareness is critical for building trustworthy AI systems. By shifting attribution from surface associations to true conditional reliance, RISE addresses three specific failure modes:

1. Over-crediting Distractors: Preventing the illusion that multiple independent sources agreed when information was merely duplicated.
2. Instability: Ensuring explanations remain consistent despite minor paraphrasing or reordering of context.
3. Susceptibility to Manipulation: Reducing the risk that injected or repeated malicious instructions are erroneously credited as influential.

The authors emphasize that RISE does not require access to internal model mechanisms (like neurons or attention heads) and relies solely on observable input-output behavior, making it suitable for real-time monitoring, auditing, and safety interventions in agentic and RAG systems. The work concludes that suppressing redundancy does not remove essential information but prevents importance from scaling with repetition, yielding explanations that reflect unique reliance rather than frequency.