💬 NLP

Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions

This paper investigates whether Large Language Models truly integrate external label definitions or rely on internal knowledge, finding that while such definitions can improve performance, models often default to their parametric representations, particularly in general tasks, with domain-specific tasks benefiting more from explicit guidance.

Original authors: Seyedali Mohammadi, Bhaskara Hanuma Vedula, Hemank Lamba, Edward Raff, Ponnurangam Kumaraguru, Francis Ferraro, Manas Gaur

Published 2026-02-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Seyedali Mohammadi, Bhaskara Hanuma Vedula, Hemank Lamba, Edward Raff, Ponnurangam Kumaraguru, Francis Ferraro, Manas Gaur

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a very smart, well-read assistant (an AI) to help you sort mail. You have a stack of letters, and you need to decide which ones are "Junk," which are "Important," and which are "Personal."

You give your assistant a rulebook (the Label Definitions) to help them decide. But here's the twist: Does the assistant actually read your rulebook, or do they just ignore it and rely on what they already know from their years of training?

This paper is a scientific experiment to answer exactly that question. The researchers tested four different AI models (like GPT-4, LLaMA-3, and others) to see if they truly listen to new instructions or if they stubbornly stick to their old habits.

Here is the breakdown of their findings, using some everyday analogies:

1. The "Swapped Dictionary" Test (Knowledge Conflict)

The researchers tried a trick: they took the definitions of the labels and swapped them around.

Normal: They told the AI, "Entailment means A proves B."
The Trick: They told the AI, "Entailment means A contradicts B" (the exact opposite).

The Result:

The "Smart" Assistant (GPT-4): When the rules were swapped, GPT-4 got confused. It realized, "Wait, this rulebook makes no sense compared to what I know," and it often refused to answer. It was like a librarian who sees a book with the wrong title and says, "I can't shelve this; the catalog is broken."
The "Obedient" Assistant (Smaller models like LLaMA-3 or Mistral): These models often followed the new (wrong) rules blindly. If you told them "Red means Blue," they would call a red apple "Blue." They were so eager to follow your instructions that they ignored their own common sense.

The Takeaway: Smaller models are like eager interns who will follow your instructions even if they are wrong. Bigger, smarter models are like senior experts who might question your instructions if they seem contradictory.

2. The "Specialist vs. Generalist" Test

The researchers tested the AI on two types of tasks:

General Tasks: Like sorting everyday sentences (e.g., "Is this sentence true based on that one?").
Specialized Tasks: Like sorting mental health posts or hate speech.

The Result:

General Tasks: The AI didn't need your rulebook much. It already knew how to sort these things from its training. In fact, giving it a rulebook sometimes confused it, like trying to teach a native speaker how to speak their own language using a textbook.
Specialized Tasks: This is where the rulebook became a superpower. For niche topics (like mental health), the AI didn't know much beforehand. When you gave it a clear, expert definition, its performance skyrocketed. It was like giving a general contractor a specific blueprint for building a nuclear reactor; without the blueprint, they'd guess; with it, they build it perfectly.

3. The "Explanation vs. Action" Paradox

This is the most surprising part. The researchers asked the AI to not just pick a label, but to explain why it picked it.

The Finding: Sometimes, the AI would write a perfect explanation showing it understood the new rules, but then pick the wrong label anyway.
The Analogy: Imagine a student taking a math test. They write a beautiful, step-by-step proof showing they understand the formula (the explanation), but then they circle the wrong answer on the bubble sheet (the prediction).
Why it matters: This means an AI can sound very confident and logical in its chat, but still make a critical error in its final decision. The "thinking" part and the "deciding" part of the AI are not always working together.

4. The "Custom Tailoring" Effect

The researchers tried different ways of giving the definitions:

Static: "Here is the definition for everyone."
Dynamic: "Here is a definition tailored specifically to this sentence."

The Result:
Tailored definitions worked best, especially for smaller models. It's like giving a chef a generic recipe vs. a recipe adjusted for the specific ingredients they have in their kitchen right now. The AI performed much better when the instructions felt relevant to the specific job at hand.

Summary: What Should We Do?

If you are using AI for important jobs (like diagnosing a patient or moderating hate speech), here is the advice from the paper:

Don't assume the AI knows everything: For specialized topics, you must provide clear, precise definitions. Don't just say "Sort this"; say "Sort this based on these specific rules."
Smaller models need more help: If you are using a smaller, cheaper AI, it relies heavily on the instructions you give it. Make those instructions perfect.
Big models might ignore you: If you use a massive model, it might ignore your instructions if they clash with what it already knows. You might need to be very careful about how you phrase things.
Check the work: Just because the AI gives you a great explanation doesn't mean its final decision is correct. Always double-check the result.

In a nutshell: LLMs are like students. Sometimes they are eager to learn your new rules (especially in new subjects), and sometimes they are so smart they think they know better than you. The key is knowing which "student" you are talking to and how to give them the right instructions.

1. Problem Statement

The central research question investigates whether Large Language Models (LLMs) genuinely incorporate external label definitions provided in prompts or if they primarily rely on their internalized parametric knowledge (pre-training).

While label definitions are theoretically used to disambiguate tasks and guide model behavior, it remains unclear if models truly process these definitions or if they ignore conflicting external guidance in favor of internal patterns. The authors identify two critical dimensions of this problem:

Knowledge Conflict: What happens when external definitions contradict the model's pre-trained knowledge?
Definition Integration: How does the method of presenting definitions (e.g., fixed, dynamic, few-shot) affect model adherence?

The study focuses on classification tasks because they offer clear ground-truth labels, allowing for objective measurement of whether a model follows external instructions or reverts to internal biases.

2. Methodology

Experimental Setup

The authors conducted controlled experiments across four diverse datasets covering general and domain-specific tasks:

e-SNLI: General Natural Language Inference (Entailment, Neutral, Contradiction).
WELLXPLAIN: Mental health categorization (Physical, Intellectual, Social, Spiritual/Emotional).
HATEXPLAIN: Hate speech detection (Hate, Normal, Offensive).
WICE: Fact-checking (identifying supporting sentences).

Models Tested: Four state-of-the-art LLMs with varying architectures and sizes:

GPT-4 (Proprietary, large-scale)
LLaMA-3 (Open-source, large-scale)
Phi-3 (Small, efficient)
Mistral-7B (Open-source, mid-size)

Evaluation Strategies

The authors designed two primary probing strategies:

A. Knowledge Conflict Experiments

Definition Permutation: The authors swapped label definitions among all possible permutations ( $p!$ ). They compared performance under aligned (correct mapping) vs. misaligned (incorrect mapping) conditions.
Definition Accuracy: They tested three levels of definition quality:
- Correct: Standard logical definitions.
- Slightly Incorrect: Subtle inaccuracies (e.g., "possibly true" instead of "necessarily true").
- Incorrect: Completely contradictory definitions.
Metrics: Matthews Correlation Coefficient (MCC) for classification accuracy (robust against class imbalance) and F1 scores for the WICE sentence-selection task.

B. Definition Integration Experiments
The authors compared four prompt conditions:

Vanilla (Zero-shot): No external definitions; relies purely on internal knowledge.
Fixed Definition: Expert-written definitions provided in the prompt.
Adjusted Definition: Definitions dynamically generated by an LLM for each specific input using a k-Nearest Neighbors (k-NN) retrieval approach to capture context.
Fixed Definition + Few-shot: Fixed definitions combined with exemplar cases.

3. Key Contributions

Evaluation Framework: Developed lightweight, interpretable strategies to assess LLM performance and explainability when injected with external knowledge.
Probing Mechanisms: Introduced permutation analysis and definitional perturbation to systematically measure sensitivity to definition quality and alignment.
Cross-Domain Analysis: Conducted extensive experiments across four benchmarks, revealing that model receptivity is not uniform but highly dependent on the task domain.
Explanation-Performance Disconnect: Uncovered a novel phenomenon where definition integration significantly improves explanation quality (ROUGE scores) without necessarily improving classification accuracy, and vice versa.

4. Key Results and Findings

A. Receptivity to Definition Quality

General vs. Domain-Specific: Models generally rely more on internal representations for general tasks (e-SNLI), whereas domain-specific tasks (Mental Health, Hate Speech) show significantly higher sensitivity to explicit definitions.
Model Size Paradox: Contrary to the assumption that larger models are always more capable, smaller models (Phi-3, Mistral-7B) often showed higher sensitivity to definition accuracy in specialized domains, suggesting they lack robust internal representations for niche topics and rely heavily on explicit guidance.
GPT-4's Meta-Cognition: GPT-4 uniquely demonstrated the ability to detect conflicting definitions and refuse to answer (abstention), a safety-aligned behavior not seen in other models.

B. Impact of Integration Strategies

General Tasks (e-SNLI): Providing definitions often degraded performance compared to the vanilla zero-shot setting. Explicit definitions sometimes interfered with the model's strong internal pre-trained representations. Adding few-shot examples alongside definitions further reduced accuracy.
Domain-Specific Tasks (WELLXPLAIN, HATEXPLAIN): Definitions generally enhanced performance.
- Mistral-7B showed massive gains (e.g., 110% improvement in WELLXPLAIN) when definitions were provided, acting as a "surrogate knowledge" base.
- LLaMA-3 showed inconsistent behavior: fixed definitions helped in HATEXPLAIN but harmed performance in WELLXPLAIN, highlighting the brittleness of in-context learning.
Fact-Checking (WICE): This task was least sensitive to semantic label definitions, relying more on factual pattern matching than abstract definitions.

C. The Competence-Performance Gap

A critical finding is the dissociation between explanation quality and classification accuracy:

Models often generated high-quality, logically sound explanations when provided with definitions (high "competence" in reasoning).
However, this did not always translate to correct final predictions (low "performance" in decision boundaries).
This suggests that external definitions influence the deliberative reasoning subsystem more readily than the fast, categorical decision-making subsystem.

5. Significance and Implications

Prompt Engineering for Specialized Domains: For high-stakes, domain-specific applications (e.g., mental health, legal, medical), precise, context-specific definitions are crucial, especially for smaller models. Generic definitions may be insufficient or even harmful.
Model Selection:
- Small Models: Benefit most from carefully tailored definitions in specialized domains, offering a lightweight way to boost performance on edge devices.
- Large Models: Rely more on internal knowledge but possess better safety mechanisms (e.g., refusing conflicting inputs), making them suitable for high-stakes environments where consistency and safety are paramount.
Trust and Transparency: Even when definitions do not improve classification accuracy, they reliably improve the quality of explanations. This is vital for regulatory compliance and user trust, as it allows models to articulate why a decision was made, even if the decision itself remains driven by internal weights.
Future Research Directions: The paper calls for mechanistic interpretability studies to understand how networks encode external definitions and suggests exploring "disputed but valid" definitions (where experts disagree) to further test model robustness.

In summary, the paper demonstrates that LLMs do not uniformly adhere to external definitions. Their behavior is a complex interplay of model architecture, task domain, and definition quality, with a distinct gap between their ability to reason about definitions and their ability to act on them.