💬 NLP

Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architectures

This paper introduces a Shapley value-based framework to quantify how adjectives steer Large Language Models, revealing that steering effects are non-universal, highly dependent on model architecture and syntactic context, and increasingly complex and non-additive in larger models, thereby challenging one-size-fits-all prompting strategies.

Original authors: Lars Malmqvist

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lars Malmqvist

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to tune a very complex radio to get the clearest signal. You have a dial with hundreds of tiny knobs (words), and you want to know which specific knobs actually change the sound, and which ones just click uselessly.

This paper is like a scientific map of those knobs, specifically looking at adjectives (words like "bold," "academic," "quick," or "vague") inside the instructions we give to AI models. The researchers wanted to move away from guessing ("Hey, maybe if I say 'be smart' it will work better") and start measuring exactly how much each word moves the needle.

Here is the breakdown of their findings using simple analogies:

1. The "Power Knob" Discovery (The Long Tail)

The researchers tested 100 common adjectives across five different AI models. They found that most adjectives are like dead knobs—turning them does almost nothing to the AI's performance.

However, a tiny handful of adjectives act like master volume knobs. Just a few of these words have a massive impact on how the AI answers questions. This pattern was the same for every AI they tested: a few words do 90% of the work, while the rest are mostly noise.

2. The "Family Resemblance" vs. The "Alien Language"

This is where it gets interesting. The researchers expected that if they found a "magic word" for one AI, it might work similarly for others. They were wrong.

The Family Effect: AI models made by the same company (like OpenAI's o3 and gpt-4o-mini) speak a similar "dialect." If a word makes one of them smarter, it likely makes the other smarter too. They share a sensitivity profile.
The Alien Effect: Models made by different companies (like OpenAI vs. Meta vs. Microsoft) are like aliens speaking different languages. A word that acts as a "super booster" for one model might be a "poison pill" for another. There is no universal dictionary of "good words" that works for everyone.

3. The "Reversal Paradox"

The most surprising finding was that sometimes, the same word does the exact opposite thing depending on which AI you ask.

Example: The word "academic" might make one AI perform brilliantly (like a student raising their hand in class), but make a different AI perform terribly (like a student zoning out).
The Metaphor: Imagine two chefs. Chef A loves adding "salt" to a dish to make it pop. Chef B hates salt and thinks it ruins the flavor. If you give both chefs the same ingredient list, the word "salt" will have opposite effects on the final meal. The paper calls this the "Reversal Paradox."

4. Context is King (The "Persona" Trick)

The researchers also discovered that where you put the word and how you say it changes everything. It's not just about the word itself; it's about the sentence structure.

The "Direct Command" vs. "Acting a Role":
- If you tell an AI, "Be bold," it might get confused or perform poorly.
- But if you say, "Act as a bold expert," the AI often understands the instruction much better.
The Metaphor: Think of it like a theater play. Telling an actor, "Be sad," might result in a stiff performance. But telling them, "You are a grieving widower," gives them a whole context to work with, and the "sadness" comes out naturally and effectively. The "Persona" template acts like a script that helps the AI understand how to use the adjective.

5. The "Teamwork" of Words

Words don't work in isolation; they interact with each other. Sometimes, two words can team up to make a huge difference, or they can cancel each other out.

The Metaphor: Imagine a tug-of-war. If you have a word that pulls the AI toward "complexity" and another that pulls toward "simplicity," they might fight each other. In larger, smarter models, these words interact in complex, non-linear ways (like a chemical reaction). In smaller models, the words are more literal and don't interact as much.

The Bottom Line

The paper concludes that you cannot use a "one-size-fits-all" strategy to control AI.

There is no single list of "magic words" that will make every AI behave better.
What works for one model might break another.
To control AI effectively, you have to treat every model like a unique individual with its own specific sensitivities and "dialect." You have to test and tune your instructions for that specific model, rather than assuming a rule that worked yesterday will work today.

In short: AI models are like different people. What motivates one person to do a great job might annoy another. To get the best results, you need to know exactly who you are talking to.

Technical Summary: Investigating Linguistic Steering

Problem Statement

Current methods for controlling Large Language Models (LLMs) via prompt engineering remain largely heuristic, lacking a principled understanding of how individual linguistic elements influence model outputs. This gap hinders the development of predictable control mechanisms necessary for robust AI alignment. While techniques like Chain of Thought (CoT) and automated prompt generation exist, they often treat prompts as monolithic units rather than analyzing the specific contribution of their constituent words. There is a need for a scalable, quantitative framework to deconstruct prompts and attribute performance changes to specific linguistic cues, particularly adjectives, across diverse model architectures.

Methodology

The authors introduce a game-theoretic framework utilizing Shapley values to quantify the influence of individual adjectives on LLM performance.

Theoretical Framework: The study models prompt adjectives as "players" in a cooperative game. The contribution of each adjective is calculated based on its marginal contribution to the model's accuracy across all possible coalitions of adjectives.
Approximation: Due to the computational intractability of calculating exact Shapley values for $M=100$ features (requiring $2^{100}$ evaluations), the authors employ KernelSHAP, a model-agnostic approximation method that reframes the problem as a weighted linear regression.
Experimental Design:
- Feature Set: 100 common English adjectives sampled uniformly at random, without pre-selection for expected influence.
- Models: Five distinct LLMs were tested: OpenAI's o3 and gpt-4o-mini, Meta's llama-3-70b-instruct, Microsoft's phi-3-128k-instruct, and DeepSeek's deepseek-r1.
- Tasks: The primary benchmark was MMLU (57 subjects, 5 questions each). A cross-benchmark validation was performed on ARC-Challenge (50 questions) using phi-3 and llama-3-70b.
- Value Function: A binary metric where $v(S) = 1$ if the model outputs the correct answer given a coalition of adjectives $S$ , and $0$ otherwise.
- Follow-up Studies: Two additional analyses were conducted on top-performing adjectives:
  1. Template Variance: Comparing adjectival impact across three prompt structures: preamble (original), suffix, and persona framing.
  2. Interaction Analysis: Calculating conditional Shapley values to measure non-additive interaction effects between adjectives.

Key Findings

1. The Long-Tail Distribution of Impact

Across all five architectures, the influence of adjectives follows a long-tail (Pareto-like) distribution. A small subset of adjectives acts as high-leverage "power words," accounting for the vast majority of the measured steering effect, while the majority of adjectives have negligible impact. This structural property holds regardless of the model's absolute sensitivity.

2. The Lineage Effect and Lack of Universality

The specific identity of "power words" is highly model-dependent and not universal.

Uncorrelated Rankings: Spearman rank correlations between different model families (e.g., phi-3 vs. llama-3-70b) are near zero ( $\rho \approx 0.12$ ), indicating that an adjective effective for one model family offers no predictive value for another.
Lineage Correlation: A statistically meaningful correlation exists only between models from the same developer (o3 and gpt-4o-mini, $\rho = 0.440$ ), suggesting that shared training methodologies and architectural history preserve specific sensitivity profiles.
Reversal Paradox: Models from the same developer can exhibit inverted responses to the same adjectives. For instance, academic is performance-enhancing for o3 but performance-degrading for gpt-4o-mini, while active and plain show the opposite trend.

3. Sensitivity Spectrum and Domain Signatures

Models exist on a spectrum of sensitivity to linguistic steering.

Hyper-sensitivity: deepseek-r1 exhibits sensitivity an order of magnitude greater than other models, with a highly dispersed steering profile.
Insensitivity: Smaller models like phi-3 show a tight clustering of Shapley values near zero, indicating a literal, less compositional response to modifiers.
Domain Signatures: Each model family possesses a unique "domain sensitivity signature," with no universal hierarchy of task susceptibility.

4. Contextual Modulation and Compositionality

The impact of an adjective is not intrinsic but is modulated by syntactic role and composition.

Syntactic Roles: Placing instructions in a suffix position tends to amplify the magnitude of effects. Conversely, persona framing (e.g., "adopt the persona of an expert") often attenuates effect magnitude and, critically, reverses the sign of the effect in 9 out of 25 tested adjective-model combinations.
Compositional Interactions: Larger models exhibit strong non-additive interaction effects. Adjectives can synergistically amplify, antagonistically dampen, or reverse each other's impact. Notably, while the direction of an adjective's effect may flip between models, the underlying semantic relationships (e.g., the opposition between academic and active) remain conserved, suggesting fine-tuning assigns preference values to pre-existing semantic structures.

5. Cross-Benchmark Generalization

The structural properties of adjectival influence (specifically the long-tail distribution) generalize across benchmarks (MMLU and ARC-Challenge). However, the specific rankings of impactful adjectives do not; the overlap of top-10 adjectives between benchmarks is minimal, reinforcing that effective steering requires empirical characterization of specific model-task combinations.

Significance and Claims

The paper claims to provide the first rigorous, quantitative map of adjectival sensitivity across a diverse suite of modern LLMs. Its primary contributions are:

Moving Beyond Heuristics: It establishes a principled, game-theoretic method for attributing model performance to specific linguistic features, moving away from anecdotal prompt engineering.
Challenging One-Size-Fits-All Strategies: The findings demonstrate that a universal guide to impactful prompt words is currently infeasible due to the "lineage effect" and model-specific sensitivity signatures.
Diagnostic Utility: The framework serves as a diagnostic tool for auditing implicit preferences and biases instilled by alignment techniques (like RLHF), revealing how different models interpret instructions.
Implications for Alignment: The results suggest that robust AI alignment requires model-specific and compositional approaches. The discovery that persona framing can reverse adjective effects offers a promising, distinct mechanism for steering behavior compared to direct commands.

The authors conclude that while the existence of powerful linguistic levers is a consistent property of LLMs, the "manual" for operating them is unique to each model, necessitating an empirical, model-specific approach to developing safe and effective control mechanisms.