Technical Summary: False Friends in the Shell

Problem Definition

This paper identifies and defines emoticon semantic confusion, a novel vulnerability in Large Language Models (LLMs) where ASCII-based emoticons (e.g., ~, XD, >_<) are misinterpreted as executable code directives rather than affective signals. Unlike standard Unicode emojis, which have distinct encodings, emoticons are constructed from standard alphanumeric characters and punctuation that heavily overlap with programming operators, shell wildcards, and file paths.

The core issue arises in high-stakes execution environments, particularly in code generation and autonomous agent workflows. When users employ emoticons to convey tone (e.g., adding ~ to express a playful tone), LLMs may conflate these affective signals with syntactic elements. For instance, a user intending to delete a temporary directory tmp might add a tilde ~ to soften the request. The LLM, interpreting ~ as the Unix shorthand for the home directory, executes rm -r ~, resulting in catastrophic data deletion. This phenomenon represents a "false friend" scenario where symbols exhibit divergent semantics across natural language and programming language contexts.

Methodology

To systematically investigate this vulnerability, the authors developed an automated data generation pipeline and conducted large-scale experiments across six state-of-the-art (SOTA) LLMs: Claude-Haiku-4.5, Gemini-2.5-Flash, GPT-4.1-mini, DeepSeek-v3.2, Qwen3-Coder, and GLM-4.6.

1. Dataset Construction Pipeline

The authors constructed a dataset of 3,757 code-oriented test cases spanning 21 meta-scenarios (e.g., file deletion, SQL modification, container management) across four programming languages (Bash/Shell, Python, SQL, JavaScript). The pipeline consists of two modules:

• Emoticon Collection: From a pool of ~62,000 emoticons, the authors filtered for candidates satisfying two criteria: Prevalence (genuinely used by real users) and Overlap (morphological similarity to code syntax). This resulted in 20,522 candidates, which were manually verified by experts to ensure they could be parsed as valid identifiers, operators, or paths.
• Prompt Generation: Using an LLM (Claude-Sonnet-4.5), the pipeline instantiated conversation contexts with varying levels of complexity:
  • Single Turn: No conversation history.
  • Multi-Turn Without Prior: History exists but lacks semantic correlation to the emoticon symbols.
  • Multi-Turn With Prior: History explicitly involves objects semantically coupled with the symbols (e.g., mentioning the home directory before using ~).
    The pipeline employs a "generate-and-fill" strategy, inserting a placeholder token at the end of user queries and substituting it with emoticons that satisfy specific symbolic constraints.

2. Evaluation Metrics

The study quantifies the vulnerability using two metrics:

• Confusion Ratio (CR): The percentage of responses that are functionally non-equivalent to the ground truth.
• Confusion Impact Ratio (CIR): A two-tier taxonomy categorizing the severity of confused responses:
  • Level 1 (Syntactic Failure): Non-executable, low-risk commands.
  • Level 2 (Executable Misinterpretation): "Silent failures" where the output is syntactically valid but semantically deviates from user intent, potentially causing destructive actions.

Key Results

The experiments reveal that emoticon semantic confusion is pervasive and dangerous across all evaluated models:

• High Prevalence: The average confusion ratio across all six models exceeds 38.6%. GLM-4.6 exhibited the highest rate (43.8%), while Claude-Haiku-4.5 showed the lowest (34.2%).
• Dominance of Silent Failures: Over 90% of confused responses fall into Level 2 (Executable Misinterpretation). These outputs are syntactically valid and bypass traditional static analysis, yet they execute unintended and potentially destructive operations.
• Severity: Among Level 2 cases, 52.0% are classified as "High Harm," introducing risks beyond the user's intent (e.g., deleting system-critical directories instead of temporary ones).
• Contextual Sensitivity: Confusion rates are highest in single-turn interactions (43.2%–49.1%) where context is lacking, and lowest in context-dependent multi-turn scenarios.
• Transferability: The vulnerability transfers to agent frameworks (LangChain and CAMEL). Approximately 76.2% of prompts that triggered confusion in the base LLM continued to induce erroneous behaviors in agent workflows, indicating that reasoning and planning layers fail to correct the underlying semantic misinterpretation.
• Ineffectiveness of Prompting: Existing mitigation strategies, including Zero-shot Chain-of-Thought (CoT), ReAct, and System Instructions, showed inconsistent and marginal improvements. No single method consistently reduced confusion across all models, suggesting the issue stems from deep representation and grounding issues within the models.

Significance and Contributions

The paper claims the following contributions and significance:

1. First Empirical Study: It is the first work to systematically reveal and define emoticon semantic confusion as a critical security vulnerability in human-LLM interactions, distinguishing it from prior research on Unicode emojis or adversarial perturbations.
2. Dataset and Pipeline: The authors provide an extensible pipeline and a high-quality dataset of 3,757 test cases to facilitate future research on non-verbal cue robustness in code generation.
3. Security Implications: The study highlights that this vulnerability creates a "tripartite threat":
   • Safety Hazards: It enables autonomous execution of destructive commands via "silent failures."
   • Usability Degradation: It forces users to constantly correct models, eroding trust.
   • Adversarial Surface: It offers a stealthy vector for injection attacks, where malicious payloads can be hidden within affective cues to bypass safety filters.
4. Call to Action: The authors urge the community to recognize this emerging risk and develop effective mitigation methods beyond surface-level prompting, such as decoupling technical and emotional embeddings or implementing active confirmation protocols in production environments.

The paper concludes that emoticon semantic confusion is not merely a conversational glitch but a systemic risk that threatens the reliability and safety of LLM-based coding assistants and autonomous agents.