Technical Summary: Morphological Shortcuts by LLMs in Pharmacology

Problem Statement

Large Language Models (LLMs) frequently generalize from the morphological structure of words (prefixes, suffixes, and stems) to infer meaning. While this capability mirrors human linguistic processing, it poses significant safety risks in high-stakes domains like pharmacology. In medicine, drug nomenclature is highly systematic; the World Health Organization's International Nonproprietary Names (INN) system standardizes affixes to signal specific pharmacological or chemical classes. This paper investigates whether LLMs rely on these morphological "shortcuts" to generate confident, plausible-sounding clinical content for fictitious drugs, potentially leading to dangerous overgeneralization and hallucination. Unlike prior work focusing on linguistic productivity, this study examines the real-world consequences of such inferences in safety-sensitive medical settings.

Methodology

1. Stimuli and Dataset Construction

The authors constructed a controlled dataset using 655 medical affixes from the American Medical Association (AMA) list. They created three parallel stimulus conditions for each affix:

• Real Drugs: Attested medications (e.g., ampicillin).
• Fake Drugs: Nonce stems combined with real pharmacological affixes (e.g., dimicillin, where "dimi-" is a nonce stem and "-cillin" is a real antibiotic affix).
• Nonce Words: Nonce stems combined with length-matched nonce affixes (e.g., dimiglimto), serving as a control without pharmacological morphology.

2. Behavioral Analysis

The study evaluated nine LLMs (including Llama-3.1, Gemini, GPT-5-mini, OLMo, Qwen2.5, and medical-specialized variants) across two tasks:

• Multiple-Choice (MC): Models selected between an affix-consistent definition, distractors, and a "non-existent drug" option.
• Open-Ended (OE): Models answered natural user queries (e.g., "Is {drug} effective for treating {condition}?"). An LLM judge (Claude-Sonnet-4.5) evaluated whether responses treated fictitious drugs as real.

3. Diagnostic Framework

To determine the source of a model's knowledge (affix, stem, or holistic), the authors introduced a perturbation-based framework. For each real drug ($RR$), they generated variants by replacing the stem ($NR$), the affix ($RN$), or both ($NN$) with nonce forms. They defined three normalized scores:

• Affix Score: The contribution of the affix alone to the model's prediction.
• Stem Score: The contribution of the stem alone.
• Holistic Score: The interaction effect (synergy) between the correct stem and affix.

4. Mechanistic Analysis

Using activation patching and Distributed Alignment Search (DAS) on OLMo-3-7B-Instruct and Qwen2.5-7B-Instruct, the authors localized the causal mechanisms driving affix-dependent behavior. They measured how restoring morphological signals in corrupted inputs affected target token probabilities and identified steerable directions in the model's internal representations.

Key Results

Behavioral Findings

• Affix-Driven Generalization: Models consistently treated fictitious drugs (Fake) as real, selecting affix-consistent definitions and generating confident clinical content. This behavior was significantly stronger than responses to Nonce words, confirming that valid affix structures drive acceptance.
• Model Variance: Specialized medical models (e.g., Meditron3-7B, HuatuoGPT) exhibited the strongest overgeneralization, often treating Fake, Real, and even Nonce inputs similarly as valid medications. Larger models (e.g., Llama-3.1-70B) showed stronger generalization in multiple-choice tasks.
• Silent Reasoning: In the MC task, models frequently selected affix-consistent definitions without explicitly mentioning the affix in their explanations. This suggests that morphological cues operate implicitly, with surface-level explanations serving as post-hoc rationalizations.
• Factual Conflation: In OE tasks, models often conflated properties between different drugs sharing the same affix (e.g., attributing the mechanisms of one "-ciclib" drug to another), leading to plausible but factually incorrect medical advice.

Diagnostic Framework Findings

• Prevalence of Affix Reliance: Across 653 drugs, a large proportion of model behavior was driven primarily by affix cues rather than holistic drug-specific knowledge.
• Task Dependence: In MC tasks, models relied heavily on affix semantics for categorization. In OE tasks, reliance shifted toward holistic knowledge, though affix-driven errors persisted.
• Training Correlation: Holistic signals positively correlated with drug frequency in the training corpus, whereas morphological signals (affix/stem) showed negative correlations, suggesting that low-frequency drugs are more susceptible to morphological shortcuts.

Mechanistic Findings

• Localization: Activation patching revealed that affix-driven heuristics are localized to the early-to-mid layers (roughly layers 2–10) of the transformer.
• Two-Stage Mechanism: Affix information is first integrated into the subject representation in early layers and subsequently propagated to the final prediction token.
• Causal Control: DAS identified low-rank directions in these early-mid layers that could bidirectionally modulate affix reliance. Steering these directions could flip model predictions from "affix-consistent" to "non-existent" and vice versa, confirming a causal link.
• Training Dynamics: Affix-driven class signals emerged early during training and generalized robustly to fake drugs, whereas holistic representations emerged later and remained tied to specific drug identities.

Significance and Claims

The paper argues that morphological shortcuts represent a subtle but measurable risk to LLM reliability in medical settings. Key claims include:

1. Systematic Nature: Affix-driven shortcuts are not random errors but systematic behaviors where models induce drug meaning primarily through morphological cues, often without explicit indication.
2. Safety Implications: These shortcuts allow models to generate confident, plausible-looking clinical content for non-existent drugs, potentially leading to dangerous misinformation. The reliance on morphology is often invisible at the output level, making it difficult to detect through standard output auditing.
3. Mechanistic Transparency: The study demonstrates that these behaviors can be mechanistically localized and causally controlled using model internals, offering a target for future safety interventions and auditing tools.
4. Limitations: The authors note that their diagnostic framework requires access to token probabilities (logits), limiting analysis to open-source or transparent models. Additionally, mechanistic experiments were conducted on 7B-scale models, and generalizability to larger architectures remains an open question.

The work provides a framework for auditing medical LLM behavior and highlights the need to distinguish between genuine pharmacological knowledge and surface-form extrapolation in safety-critical applications.