🤖 AI

When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness

This paper presents a comparative study across six benchmarks and four models demonstrating that while natural language explanations generally enhance in-context learning performance, their effectiveness varies significantly based on the explanation source (with external LLMs often rivaling human rationales) and the selection strategy, particularly revealing that faithfulness-based filtering yields inconsistent results and that different faithfulness metrics can lead to divergent conclusions.

Original authors: Mahdi Dhaini, Adam Dejl, Juraj Vladika, Volkan Özer, Barbara Plank, Gjergji Kasneci

Published 2026-08-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mahdi Dhaini, Adam Dejl, Juraj Vladika, Volkan Özer, Barbara Plank, Gjergji Kasneci

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, there is a growing desire to understand not just what a computer says, but why it says it. When a large language model answers a question, it often produces a string of text that looks like a human reasoning through a problem. These strings of text are called natural language explanations. They serve a dual purpose: they help humans trust the machine by showing its logic, and they can actually help the machine think better. This second effect is known as in-context learning. Imagine asking a computer to solve a new puzzle. If you give it a few examples of similar puzzles along with the correct answers, it can often figure out the new one without needing to be retrained. This is in-context learning. Now, imagine if you added a few sentences to those examples explaining how the answer was reached. Does that extra explanation make the computer smarter, or does it just add noise? This is the central question researchers are trying to answer as they build more capable and trustworthy AI systems.

A team of researchers from universities in Munich, London, and across Germany set out to test exactly this. They wanted to know if adding explanations to the examples given to an AI actually improves its performance, and if so, where those explanations should come from. They tested four different AI models of varying sizes on six different types of reasoning tasks, ranging from common-sense questions and detecting sarcasm to solving complex math word problems. The researchers compared three main sources for these explanations: explanations written by humans, explanations generated by the AI itself as it tried to solve the problem, and explanations generated by a separate, powerful AI model acting as a helper. They also tested whether the quality of the explanation mattered, specifically looking at "faithfulness," which is a measure of whether the explanation actually matches the steps the model took to reach its answer.

The results offered a clear picture of what works best in practice. When the task was to classify information, such as deciding if a sentence was sarcastic or determining the relationship between two statements, adding explanations to the examples usually helped the AI perform better. However, the source of the explanation mattered significantly. Explanations generated by a separate, powerful AI model often provided the biggest boost in accuracy, performing just as well as, and sometimes better than, explanations written by humans. This is a significant finding because human-written explanations are expensive and slow to produce, while AI-generated ones can be created instantly. In contrast, explanations generated by the AI itself while it was trying to solve the problem were much more hit-or-miss. They sometimes helped, but often they were less reliable than those from an external source.

The researchers also investigated whether filtering these self-generated explanations to keep only the "faithful" ones—those that truly reflected the model's reasoning—would improve results. They found that this strategy yielded only small average gains and was highly inconsistent. Sometimes, selecting the most faithful explanations helped, but other times it made the model perform worse. The outcome depended heavily on which specific mathematical test was used to measure faithfulness, the specific task the model was doing, and the size of the model itself. This suggests that there is no single, perfect way to judge the quality of an explanation that works for every situation. Furthermore, the team tested how robust the system was by swapping explanations between different examples or using explanations from completely different topics. While the models did not completely break down when given these mismatched explanations, their performance did drop, indicating that the explanation needs to be semantically aligned with the specific example to be truly helpful.

On the more complex task of mathematical reasoning, the dynamics shifted. Here, the benefit of adding explanations was less consistent and depended heavily on the specific model and the source of the explanation. For the largest models, the standard method of asking the model to think step-by-step on its own remained very strong. However, for smaller models, high-quality external explanations could provide a substantial lift in performance, acting as a form of guidance that the smaller model could not generate on its own. The study concludes that while adding explanations to prompts is a powerful tool, it is not a magic bullet. The best approach depends on the task and the resources available. For many practical applications, using explanations generated by a separate, capable AI offers a reliable and cost-effective way to improve performance, whereas relying on the model to explain its own reasoning requires careful selection and offers less predictable results.

Technical Summary: When Do Explanations Help In-Context Learning?

Problem Statement

Natural language explanations (NLEs), or free-text rationales, are increasingly utilized as inputs to influence Large Language Model (LLM) behavior in In-Context Learning (ICL). While NLEs are widely used to improve interpretability and model performance, it remains unclear how different sources of NLEs (human-annotated, self-generated, or externally generated by another LLM) compare in their predictive utility. Furthermore, the role of explanation quality—specifically faithfulness (whether an explanation accurately reflects the model's decision process)—and the robustness of ICL to semantic misalignment (irrelevant or out-of-distribution rationales) are underexplored. This lack of systematic evidence hinders the ability of practitioners to make informed decisions regarding the accuracy-cost trade-off in explanation-augmented prompting pipelines.

Methodology

The authors present a comprehensive comparative study evaluating the predictive utility of NLEs across six diverse reasoning benchmarks (ECQA, e-SNLI, SNARKS, Boolean Expressions, Causal Judgment, and GSM8K) and four instruction-tuned models of varying sizes (GPT-4o-mini, Llama-3.1-8B, Llama-3.3-70B, and Mistral-7B).

The experimental framework involves four primary steps:

Few-shot Sample Selection: The study adopts an error-based selection strategy, choosing $n=6$ samples that were misclassified by the target model in a zero-shot setting. These samples serve as corrective signals.
NLE Generation: Three sources of explanations are generated for the selected samples:
- Human-NLEs: Rationales from existing explainable datasets (where available).
- Self-NLEs: Generated by the target model itself using a post-hoc Chain-of-Thought (Ph-CoT) method. Crucially, for misclassified samples, self-explanations are regenerated conditioned on the gold label to ensure fair comparison with other sources.
- LLM-NLEs: Generated by external explainer models (GPT-4o-mini or o3-mini) prompted to provide rationales for the correct input-output pairs.
NLE Selection Strategies: The study tests how selecting examples based on explanation quality affects performance:
- Random Selection: Baseline selection without quality assessment.
- Most-Faithful Selection: Selecting self-NLEs with the highest scores based on two automated faithfulness metrics ( $f_{m1}$ from the LExT framework and $f_{m2}$ based on self-consistency).
- Lowest-Faithful Selection: Selecting self-NLEs with the lowest scores to test the inverse relationship.
- Misalignment Stress Tests: Two ablation settings where rationales are either randomly swapped within the same dataset (in-domain mismatch) or replaced with rationales from a completely different dataset (Out-of-Distribution/OOD) to test robustness.
Evaluation: Performance is measured by downstream task accuracy, comparing explanation-augmented prompts against baselines (Zero-Shot, standard Few-Shot without rationales, and Zero-Shot Chain-of-Thought).

Key Contributions

Comprehensive Comparative Study: The first large-scale evaluation comparing human, self-generated, and externally LLM-generated NLEs as in-context rationales across multiple models and reasoning tasks.
Modular Evaluation Framework: A reproducible framework integrating error-based sample selection, explanation quality scoring (faithfulness), and prompt construction to isolate the impact of NLE source and selection strategy.
Analysis of Faithfulness Metrics: A detailed investigation into how different faithfulness metrics ( $f_{m1}$ and $f_{m2}$ ) agree or disagree, and how these disagreements influence the selection of examples and subsequent model performance.
Robustness Stress Tests: Introduction of random and OOD rationale swaps to quantify the sensitivity of ICL to semantic misalignment.

Results and Analysis

1. NLE Source and Predictive Utility (RQ1)

Classification Tasks: On classification-style benchmarks (ECQA, e-SNLI, SNARKS, Boolean, Causal Judgment), adding NLEs to few-shot prompts generally improves accuracy over standard Few-Shot prompting.
Source Comparison: Externally generated LLM-NLEs (from GPT-4o-mini or o3-mini) consistently provide the strongest and most stable downstream utility, often matching or exceeding human-annotated rationales where available. Self-generated NLEs show more variability and generally lag behind external sources, though they can be competitive in specific settings.
Math Reasoning (GSM8K): The benefits of NLEs are more model- and source-dependent. While Chain-of-Thought (CoT) remains a strong baseline for math, LLM-generated NLEs generally outperform self-NLEs, though the gap varies significantly by model size.

2. Faithfulness and Selection (RQ2)

Metric Disagreement: The two faithfulness metrics ( $f_{m1}$ and $f_{m2}$ ) disagree substantially (average disagreement rate $\sim$ 50%), indicating they capture different operational aspects of faithfulness. This disagreement directly affects which self-NLEs are selected and their resulting utility.
Utility of Faithfulness-Based Selection: Selecting self-NLEs based on measured faithfulness yields small average gains overall but is highly sensitive to the specific metric, task, and model.
- On GSM8K, $f_{m1}$ -based selection significantly improved performance for Llama-70B but degraded it for Mistral-7B.
- $f_{m2}$ -based selection showed mixed results, sometimes improving and sometimes reducing performance compared to random selection.
Conclusion: Faithfulness-based selection is not a universally reliable filter; its effectiveness depends heavily on the chosen metric and the specific model-task combination.

3. Robustness to Misalignment (RQ3)

Partial Robustness: Models exhibit partial robustness to misaligned rationales. While randomly swapped in-domain rationales and OOD rationales generally reduce performance compared to faithful selection, models do not completely collapse.
Semantic Alignment Importance: The performance degradation observed with OOD rationales (e.g., pairing a math problem with a sarcasm explanation) confirms that semantic alignment between the rationale and the example is a critical factor for performance gains. In-domain random swaps show mixed effects, suggesting that even mismatched in-domain rationales may provide some task-relevant cues.

Significance and Practical Implications

The paper concludes that NLE source, selection strategy, and faithfulness metrics are critical design choices in explanation-augmented ICL.

Practical Guidance: For practitioners, externally generated LLM-NLEs offer a practical, high-utility alternative to expensive human annotation, particularly for classification tasks. They provide stable performance without requiring a complex faithfulness-scoring stage.
Self-Explanations: While self-generated explanations are cost-effective, their utility is less reliable and highly sensitive to selection strategies. Faithfulness-based filtering of self-NLEs should be approached with caution, as metric choice can lead to divergent outcomes.
Alignment: The study emphasizes that semantic alignment is crucial; using non-aligned rationales (even from the same dataset) can negate the benefits of explanation-augmented prompting.
Model Size: Smaller models (SLMs) show more pronounced performance gains from high-quality external explanations compared to larger models (LLMs), which appear to internalize more reasoning patterns and rely less on external scaffolding.

Overall, the work provides empirical evidence to guide the selection and reporting of explanations in prompting pipelines, highlighting that while explanations can shape model behavior, their utility is contingent on the source, quality, and alignment of the rationale.