For What Reason? Interpreting Models' Encoding of Causation and Antithesis

Technical Summary: Interpreting Models' Encoding of Causation and Antithesis

Problem Statement
Discourse relations are fundamental to document structure, language understanding, and the ethical alignment of Large Language Models (LLMs). Despite their importance, limited research has investigated how Transformer models encode these relations internally. While recent work has begun to identify "discursive circuits," there remains a gap in understanding the specific mechanisms by which instruction-tuned models distinguish between opposing discourse relations, specifically causation (e.g., "so") and antithesis (e.g., "yet"). This paper addresses this gap by investigating how LLaMA and Mistral models encode these relations, focusing on token-level features, layer-specific roles, key concepts, and information propagation.

Methodology
The authors frame the investigation as a next-token prediction task. The models are prompted to complete sentences where the missing token is constrained to be either "able" or "unable." The sentence structure is designed such that the conjunction ("so" vs. "yet") and the sentiment of the verb in the satellite clause determine the correct answer, creating a controlled environment to test causal reasoning.

• Data: A dataset of 130 training pairs and 30 test pairs was generated using ChatGPT 4o. The pairs include variations with positive and negative verbs, as well as explicit negation, to prevent the model from relying solely on conjunction cues.
• Models: The study utilizes instruction-tuned variants of LLaMA-3-8B and Mistral-7B, both featuring 32 layers.
• Interpretability Techniques: The authors employ a suite of mechanistic interpretability methods:
  • Activation Patching: Replacing internal activations from a base input with those from a counterexample (e.g., swapping "so" for "yet") to measure causal effects on output logits.
  • Logits Attribution: Calculating the dot product between layer hidden states and the logit difference direction to determine if a layer pushes the model toward the correct answer.
  • Probing: Training logistic regression classifiers on hidden states to detect the presence of specific concepts (e.g., "able" vs. "unable," verb sentiment, clause sentiment).
  • Attribution Patching (EAP) & Integrated Gradients (EAP-IG): Used to identify and quantify the importance of specific edges (connections) within the computation graph.

Key Results
The models achieved high performance on the task (LLaMA: 0.96 F1, Mistral: 0.80 F1), enabling robust internal analysis.

1. Token-Level Features (RQ1): SHAP analysis reveals that the influence of discourse markers ("so" vs. "yet") is modulated by verb sentiment. For positive verbs, "so" drives the "able" prediction, while "yet" drives "unable." However, with negative verbs, these associations can reverse or shift, indicating that the model integrates verb semantics rather than relying on simple cue words.
2. Layer Roles (RQ2):
   • Early-to-Mid Layers: These layers are sensitive to interventions at the discourse marker token ("so"/"yet"). Patching activations here transiently alters output probabilities, suggesting these layers encode local relational meanings.
   • Mid-to-High Layers: Interventions at the final token position affect probabilities in mid-level layers, indicating these layers integrate sentence-level meaning. Higher layers primarily propagate or refine the decision rather than re-evaluating it; once the decision is consolidated in mid-layers, swapping activations in higher layers fails to flip the output.
   • Bias: Certain layers exhibit inherent biases, consistently pushing predictions toward "able" or "unable" regardless of the correct answer (e.g., layers 2, 3, 8, 10, 12 favoring "unable").
3. Conceptual Encoding (RQ3): Probing experiments show that the model actively encodes verb sentiment and satellite clause sentiment. High probe accuracy for "able" vs. "unable" is found in heads from layers 8–16 at the discourse marker position, and in layers 13–14 at the final token position.
4. Information Propagation (RQ4): Edge attribution patching reveals a stable core of connections (51% overlap between 0-shot and few-shot settings) that consistently drive decision-making. However, processing negative verbs shows lower edge overlap across shot settings (33.9%), suggesting greater instability or different pathway reliance for negative sentiment compared to positive sentiment.

Significance and Contributions
The paper claims to provide insight into the internal mechanics of how critical discourse relations are encoded in Transformer models. The primary contributions are:

• Mechanistic Insight: Demonstrating a division of labor where early/mid layers encode local relational cues and mid/high layers integrate and finalize decisions.
• Circuit Stability: Identifying a core set of stable connections that support information flow, while noting that specific edges can vary based on context (verb type, prompt configuration).
• Asymmetric Representation: Observing that certain layers exhibit asymmetric biases toward specific outcomes, suggesting discourse-based reasoning is not uniformly represented across all layers.

The authors conclude that these findings enable heightened model performance and alignment with human preferences. They assert that their methodology is extensible to additional discourse relations and model architectures, though they acknowledge limitations regarding the specific focus on English and the two selected RST relations.