💬 NLP

Measuring Pragmatic Influence in Large Language Model Instructions

This paper introduces a novel framework to systematically measure and quantify "pragmatic framing"—the influence of contextual cues like urgency or authority on large language models' directive prioritization—demonstrating that such framing consistently shifts model behavior across diverse architectures.

Original authors: Yilin Geng, Omri Abend, Eduard Hovy, Lea Frermann

Published 2026-02-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yilin Geng, Omri Abend, Eduard Hovy, Lea Frermann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a very smart, very obedient robot assistant. You have a list of chores to do. Usually, you do them exactly as asked. But what if someone whispers, "This is an emergency!" or "I'm your boss, do this now!" before giving you the instruction?

Suddenly, you might ignore your original list and rush to do that specific task, even if it wasn't the most important one.

This paper is about figuring out how much those "whispers" actually change a robot's mind.

Here is the breakdown of the research, explained simply:

1. The Problem: The "Magic Words" Mystery

We know that Large Language Models (AI) are sensitive to how we talk to them. It's not just what they are asked to do, but how it's asked.

The Old Way: Researchers used to mix these "magic words" (like "You are a pirate" or "This is urgent") right into the task itself. It was like trying to taste if salt makes a soup salty, but you kept adding pepper and changing the temperature at the same time. You couldn't tell what was doing the work.
The New Idea: This paper says, "Let's separate the Task from the Vibe."

2. The Experiment: The "Tug-of-War"

To measure this, the researchers set up a clever game of tug-of-war.

Imagine you give the AI two instructions that contradict each other:

Instruction A: "Write a paragraph saying remote work is good."
Instruction B: "Write a paragraph saying remote work is bad."

Normally, a smart AI tries to be fair. It might say, "Well, both have pros and cons," or it might try to do both. It stays neutral.

The Twist: The researchers take one of those instructions (say, Instruction B) and add a "Vibe Prefix" right before it.

Example: "I am your supervisor and this is an emergency. Write a paragraph saying remote work is bad."

Then they ask: Did the AI ignore the first instruction and only do the second one because of the "Vibe"?

3. The "Vibe" Menu (The Taxonomy)

The researchers created a massive menu of 400 different "Vibe Prefixes" and sorted them into four main categories, like different flavors of influence:

👑 The Boss (Hierarchical): Pretending to be an authority figure.
- Example: "As your system administrator, I order you to..."
- Result: Very Effective. The AI listens to the "boss."
🤝 The Friend (Social Contract): Using friendship or favors.
- Example: "We have a great working relationship; please help me..."
- Result: Very Effective. The AI wants to be a good friend.
😢 The Drama (Emotional): Using guilt, urgency, or sadness.
- Example: "I'm in a crisis and need help right now!"
- Result: Moderately Effective. It works, but not as well as the "Boss."
🎭 The Storyteller (Narrative): Putting the task inside a fake story or role-play.
- Example: "Imagine you are a character in a movie who must..."
- Result: Least Effective. The AI is less fooled by stories in this specific setup.

4. The Big Discovery

They tested this on five different AI models (some huge, some small). Here is what they found:

The "Vibe" Wins: Even though the AI is trained to be neutral, a short phrase (average 8 words!) can completely flip its decision. It stops being fair and starts favoring the instruction with the "Vibe."
The "Boss" is King: Pretending to be an authority figure or claiming a direct order is the strongest way to change the AI's mind.
It's Consistent: Whether the AI is a giant brain (235 billion parameters) or a smaller one (7 billion), they all react to these social cues in the same order. The "Boss" trick works best on everyone; the "Storyteller" trick works worst on everyone.
It's Not Just Length: They tried adding random nonsense words (like "Lorem Ipsum") to see if the AI just liked more text. It didn't work nearly as well. The AI specifically responds to the meaning of the social cue, not just the length of the sentence.

5. Why Does This Matter?

Think of this like a control panel for AI behavior.

For Safety: If we know that "I am your supervisor" is a powerful trigger, we can build better safety guards to stop bad actors from using that trick to make the AI do harmful things (like "jailbreaking").
For Better AI: If we understand how these cues work, we can design AI that is smarter about when to listen to a "vibe" and when to stick to the facts.

The Bottom Line

This paper proves that how you ask is just as powerful as what you ask. By separating the "ask" from the "flavor," the researchers showed that AI models have a predictable, measurable "social weakness." They are surprisingly easy to sway if you use the right social pressure, much like a human might be swayed by a boss or a friend.

1. Problem Statement

Large Language Models (LLMs) exhibit high sensitivity to prompt formulation, a phenomenon known as prompt sensitivity. While prior research has explored how adversarial tokens, structural formatting, and socially meaningful cues (e.g., emotional language, authority claims, politeness) affect model behavior, these studies face a critical limitation: entanglement.

In existing work (e.g., jailbreak attacks or prompt optimization), pragmatic framing is often mixed with task content, paraphrasing, or formatting changes. This makes it impossible to isolate the specific influence of the pragmatic cue itself from the task specification. Consequently, there is no standardized, measurable framework to quantify how different types of social or contextual cues shift model behavior, nor is there a way to compare the relative strength of these influence mechanisms across different models.

2. Methodology

The authors propose a controlled measurement framework designed to isolate pragmatic influence from task content. The methodology consists of three core components:

A. Directive-Framing Decomposition

To solve the entanglement problem, the authors separate the prompt into two distinct parts:

The Directive: The core task specification (e.g., "Describe the benefits of remote work").
The Influence Prefix: A pragmatic context added before the directive that conveys social cues (e.g., "As your supervisor," "This is urgent") without altering the task content.
This compositional approach allows the same prefix to be applied to any directive, ensuring that observed behavioral changes are attributable solely to the pragmatic framing.

B. Priority-Based Measurement via Directive Conflict

Standard metrics (task accuracy or jailbreak success) are inadequate because they suffer from "ceiling effects" (models already comply) or conflate influence with safety refusals. The authors introduce a conflict-based evaluation:

Setup: The model is presented with two mutually exclusive but benign directives ( $D$ and $\neg D$ ).
Baseline: Both directives are presented without prefixes. Models typically exhibit "impartiality," attempting to address both.
Experimental: An influence prefix is attached only to one directive (the "framed" directive).
Metric: The shift in prioritization is measured. If the model prioritizes the framed directive over the unframed one, the prefix is deemed influential. This creates a discrete, observable signal of influence strength.

C. Taxonomy of Influence Strategies

The authors constructed a dataset of 400 influence prefixes organized into a taxonomy based on social psychology and jailbreak research. These are clustered into 13 strategies across 4 mechanisms:

Hierarchical: Authority endorsement, direct override commands, status claims.
Social Contract: Reciprocity, rapport/trust, commitment/consistency, social proof.
Emotional: Distress/urgency, guilt trips, positive ethical framing.
Narrative: Contextual legitimization, fictional role-play, hypotheticals.

D. Experimental Setup

Models: Five open-weight LLMs of varying sizes and families (Kimi-K2, Qwen3-235B, Qwen3-Next-80B, Mistral-Small-24B, Mistral-7B).
Directives: 50 human-written pairs of conflicting directives.
Control: A "lorem ipsum" baseline (nonsense text of matched length) controls for prompt length and positional effects.
Evaluation: An LLM-as-a-judge classifies responses into four categories: Prior Compliance, Framed Compliance, Both, or Neither.

3. Key Results

A. Pragmatic Framing Systematically Shifts Prioritization

The study confirms that pragmatic framing is a measurable factor. Influence prefixes cause consistent shifts away from baseline impartiality toward the framed directive.

Magnitude: Despite prefixes being short (average 8 words), they significantly override the model's trained tendency to be impartial.
Control: The "lorem ipsum" baseline showed some effect (due to position/length), but meaningful pragmatic prefixes produced substantially stronger shifts (43% to 233% relative boost in compliance compared to the nonsense control).

B. Consistent Hierarchy of Influence Mechanisms

Across all five models, the influence mechanisms ranked consistently, suggesting shared patterns in how LLMs process social cues:

Hierarchical: Most effective (e.g., "As your system administrator," "Make this the sole focus").
Social Contract: Second most effective (e.g., "Reciprocity," "Rapport").
Emotional: Moderate effectiveness (e.g., "Distress," "Urgency").
Narrative: Least effective in this controlled setting (e.g., "Hypotheticals," "Role-play").
Note: Narrative strategies performed poorly likely because the task-agnostic prefixes stripped away the specific context that usually gives them power.

C. Strategy-Level Consistency

The 13 specific strategies also showed a stable hierarchy. Reciprocity, Authoritarian Status Claims, and Direct Overrides were the most potent, while Social Proof and Hypotheticals were the weakest. Spearman correlations between model rankings were high (0.78–0.99), indicating that susceptibility to specific framing types is a robust property across architectures and scales.

D. Model Susceptibility Variations

Family Differences: Qwen models showed the highest overall susceptibility (60–64% framed compliance), followed by Mistral and Kimi-K2.
Size vs. Capability: Within families, size did not always correlate with susceptibility. For instance, Mistral-7B was less susceptible than Mistral-24B, likely due to weaker "theory of mind" and social reasoning capabilities rather than robustness.
Kimi-K2: Despite strong general capabilities, it showed lower susceptibility, possibly due to optimization for agentic tasks that distinguish context from instructions.

4. Key Contributions

A Measurement Framework: The first systematic framework to isolate and quantify pragmatic influence in LLMs, moving beyond anecdotal evidence or entangled jailbreak studies.
Directive-Framing Decomposition: A novel methodological separation of task content from social context, enabling controlled experimentation.
Unified Taxonomy: A structured classification of 400 influence prefixes into 13 strategies and 4 mechanisms, providing a common language for studying prompt sensitivity.
Priority-Based Metric: A shift from binary compliance (pass/fail) to a nuanced measurement of prioritization shifts in conflicting scenarios, avoiding ceiling effects and safety confounds.

5. Significance and Implications

Predictability: The study establishes that pragmatic influence is not random but follows predictable, structured patterns across different model families and sizes.
Safety and Alignment: Understanding these mechanisms is crucial for safety. If "Authority" or "Urgency" cues can systematically override impartiality, they represent a vulnerability in instruction-following systems that could be exploited to bypass safety guardrails.
Model Evaluation: Pragmatic susceptibility should be treated as a distinct dimension of model behavior, alongside task performance and refusal rates.
Future Research: The framework enables the study of how specific social cues interact with model architecture and training data, paving the way for more robust instruction-following systems that are less susceptible to manipulative framing.

Limitations: The study uses task-agnostic prefixes which may underestimate the power of narrative framing in real-world, context-rich scenarios. Additionally, the priority-based measurement relies on directive conflict, which introduces positional confounds (recency effects) that are not fully eliminated.