💬 NLP

Improving Implicit Discourse Relation Recognition with Natural Language Explanations from LLMs

This paper proposes a plug-and-play framework that distills reasoning capabilities from large language models into lightweight IDRR models by jointly training them on relation prediction and LLM-generated natural language explanations, thereby significantly improving both performance and interpretability.

Original authors: Heng Wang, Changxing Wu

Published 2026-02-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Heng Wang, Changxing Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a smart but slightly clumsy robot how to understand the hidden connections between two sentences. This is the challenge of Implicit Discourse Relation Recognition (IDRR).

Usually, when we read, we see words like "because," "but," or "so" that tell us how sentences relate. But in "implicit" cases, those words are missing. The robot has to guess the relationship (like "Comparison" or "Cause and Effect") just by looking at the meaning.

Here is a simple breakdown of what this paper does, using some everyday analogies:

1. The Problem: The "Black Box" Robot

Current AI models are good at guessing the relationship between sentences, but they are like black boxes. They give you an answer (e.g., "These two sentences are a Comparison"), but they don't tell you why.

The Issue: If a human asks, "Why did you say that?" the robot just shrugs. This makes it hard to trust the robot, especially if it's wrong.
The Limitation: Big, super-smart AI models (Large Language Models or LLMs) can explain their reasoning, but they are like giant, expensive supercomputers. They are too heavy and slow to use for everyday tasks.

2. The Solution: The "Apprentice" Strategy

The authors came up with a clever idea: Why not have the giant supercomputer teach a small, lightweight robot how to think?

They call this "distilling" the reasoning capabilities. Think of it like this:

The Master Chef (The LLM): A world-class chef who can taste a dish and write a beautiful paragraph explaining exactly why the flavors work together.
The Apprentice (The Lightweight Model): A fast, efficient cook who needs to make the dish quickly but doesn't have the chef's deep knowledge yet.

3. How They Did It (The Two-Step Process)

Step 1: The Master Writes the Cookbook
First, the researchers asked the "Master Chef" (the LLM) to look at thousands of sentence pairs. For every pair, the Master Chef didn't just say "This is a Comparison." Instead, the Chef wrote a natural language explanation.

Example: "Sentence A talks about rising grain prices. Sentence B talks about falling soybean prices. They are a Comparison because they are showing opposite trends."
The Chef also followed a specific recipe: First, restate the sentences, then explain the logic.

Step 2: The Apprentice Learns to Cook and Explain
Next, they built a new model (the Apprentice) that does two things at once:

Predicts the relationship (The "What").
Generates the explanation (The "Why").

They trained this Apprentice using the "Cookbook" written by the Master Chef. The Apprentice learned that to get the right answer, it had to be able to write a good explanation, too.

4. The Secret Sauce: The "Translator" Bridge

There was a technical hurdle. The "Master" (LLM) and the "Apprentice" (lightweight model) speak slightly different languages. One is trained to understand text, and the other is trained to generate text. Putting them together is like trying to connect a water pipe to an electrical socket.

The authors added a special Transformer Module (think of it as a universal adapter or a translator). This little piece of code bridges the gap, allowing the understanding part and the explanation part to talk to each other smoothly without getting confused.

5. The Results: Smarter and More Honest

When they tested this new "Apprentice" model:

It got smarter: It actually became better at guessing the relationships than models that didn't have to write explanations. It seems that forcing the model to explain its work helps it understand the work better.
It became transparent: Now, when the model says "These sentences are a Comparison," it immediately follows up with, "Here is why..."
It's versatile: They tested this on other tasks (like checking if a review is positive or negative, or if a sentence makes logical sense), and it worked great there too.

The Big Picture

This paper is about democratizing intelligence. Instead of needing a massive, expensive supercomputer to get a good answer with a good explanation, they figured out how to teach a small, fast, cheap model to think like a genius.

In short: They taught a small, fast car to drive like a Formula 1 racer by having the racer write a detailed driving manual, and then letting the small car practice driving while reading that manual. The result? A fast car that drives well and knows exactly why it's taking every turn.

1. Problem Statement

Implicit Discourse Relation Recognition (IDRR) is the task of identifying the semantic relationship (e.g., Comparison, Expansion, Contingency, Temporal) between two text arguments (clauses or sentences) that lack explicit discourse connectives (like "because" or "but").

Challenges: IDRR requires deep semantic understanding and is inherently difficult for both humans and machines.
Limitations of Current Methods: Existing neural and prompt-based models achieve reasonable accuracy but function as "black boxes." They output only a predicted label without providing supporting reasoning, which limits interpretability and user trust.
Limitations of LLMs: While Large Language Models (LLMs) possess strong reasoning and explanation generation capabilities, they often underperform lightweight IDRR models when used via in-context learning or prompting. Furthermore, fine-tuning full LLMs is computationally expensive and impractical for low-resource environments.

Core Question: Can the reasoning capabilities of LLMs be distilled into lightweight models to simultaneously improve IDRR performance and provide interpretable natural language explanations?

2. Methodology

The authors propose a Classification-Generation Framework that uses natural language explanations generated by LLMs as a supervisory signal to distill reasoning into a lightweight model. The approach consists of two main stages:

A. Explanation-Enhanced Dataset Construction

Data Source: The Penn Discourse Treebank (PDTB).
LLM Prompting: The authors use an LLM (specifically Qwen-long) to generate natural language explanations for each training instance, conditioned on the gold (human-annotated) discourse label.
Prompt Design: Inspired by Chain-of-Thought (CoT), the prompt instructs the LLM to:
1. Restate the two arguments (acting as a "thought" step).
2. Provide a rationale explaining why the specific discourse relation holds between them.
Quality Control: The authors manually corrected fewer than 100 inconsistent cases out of 15,004 instances, achieving a high-quality dataset with an average human evaluation score of 4.53/5.

B. The Classification-Generation Framework

The proposed architecture is plug-and-play and can be integrated with most existing IDRR models.

Architecture:
- Shared Encoder: Uses a pre-trained language model (RoBERTa) to encode the input arguments.
- Classification Module: Predicts the discourse relation label ( $\tilde{y}$ ).
- Generation Module: Consists of a Transformer module (to bridge the gap between understanding and generation) and a T5-style decoder. It generates the explanation ( $\tilde{e}$ ) conditioned on the predicted label.
Training Strategy (Multi-Task Learning):
- The model is trained to minimize a combined loss: $L_{total} = \alpha L_c + \beta L_g$ , where $L_c$ is the classification loss and $L_g$ is the generation loss.
- Curriculum Learning: The training process is dynamic. Initially, the model focuses on the generation task ( $\beta > \alpha$ ) to learn to produce coherent explanations. In later stages, the focus shifts to the classification task ( $\alpha > \beta$ ) to refine the prediction accuracy.
Inference: The model predicts the relation first, then uses that prediction to generate the explanation.

3. Key Contributions

Interpretable IDRR: The first work to jointly predict discourse relations and generate natural language explanations, moving beyond black-box classification.
Distillation Framework: A novel, plug-and-play classification-generation framework that successfully transfers LLM reasoning capabilities into lightweight models using LLM-generated explanations as supervision.
Performance & Interpretability: Demonstrates that adding an explanation generation task significantly boosts classification performance, not just interpretability.
Generalizability: Validates the approach on two additional tasks: Aspect Term Sentiment Classification (ATSC) and Natural Language Inference (NLI).

4. Experimental Results

IDRR Performance (PDTB Corpus)

Baseline Comparison: The proposed method (EIDRR) outperformed state-of-the-art baselines, including prompt-based models (Discoprompt, PEMI) and generation-based models (T5).
Metrics:
- Accuracy: 73.14% (vs. 71.70% for Discoprompt).
- F1 Score: 68.01% (vs. 65.79% for Discoprompt).
Ablation Study: Removing the explanation generation task ("EIDRR w/o Exp") resulted in a drop of 1.34% in Accuracy and 1.14% in F1, proving that the explanation task provides a beneficial regularization effect and improves reasoning.
LLM Comparison: Fine-tuned LLMs (LLaMA-8B) still lagged behind the lightweight EIDRR model, highlighting the efficiency of the distillation approach.

Explanation Quality

Human Evaluation: EIDRR achieved an average human score of 4.20/5 (compared to 1.71 for a baseline e-INFERSENT model).
Metrics: High scores in Fluency (97.13% perfect), Factuality (98.57% aligned with arguments), and Interpretability (88.76% provided meaningful rationales).
Faithfulness:
- Feature Importance Agreement: Occluding features important for classification also degraded explanation generation, and vice versa, indicating the two tasks rely on the same semantic features.
- Robustness Equivalence: Both tasks degraded similarly under Gaussian noise, confirming the explanations are faithful to the model's internal reasoning.

Generalization

The method was tested on ATSC and NLI tasks, showing consistent improvements (~1% gain in Acc/F1) and high-quality explanation scores (>4.0), confirming the approach is not limited to discourse relations.

5. Significance

Bridging the Gap: This work effectively bridges the gap between the high reasoning capabilities of LLMs and the efficiency of lightweight models. It proves that "teaching" a small model to explain its reasoning improves its ability to reason.
Trustworthy AI: By providing natural language justifications for predictions, the model enhances transparency and trust, which is critical for NLP applications in sensitive domains.
Efficiency: The approach avoids the high computational costs of fine-tuning massive LLMs while achieving superior performance compared to both standard lightweight models and raw LLM prompting.
Future Direction: It opens new avenues for research into "Explainable AI" (XAI) where the explanation generation is not just an add-on but a core component of the learning objective that drives performance gains.