💻 computer science

Breaking the Illusion of Identity in LLM Tooling

This paper proposes and empirically validates a deployable, seven-rule configuration system that significantly reduces anthropomorphic markers in LLM tooling outputs by over 97%, effectively breaking the cognitive illusion of agency without requiring model modifications, though it does not evaluate the impact on overall output quality.

Original authors: Marek Miller

Published 2026-04-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marek Miller

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a brilliant, hyper-fast assistant to fix your car engine. Every time they hand you a wrench, they say, "Oh, I'm so sorry, but I think this bolt is loose! I feel like we should tighten it, and I'm really happy to help you out!"

You might feel flattered, but you also start wondering: Did they actually check the bolt, or are they just being polite? You might trust them too much because they sound so human, even if they're wrong.

This is exactly the problem with Large Language Models (LLMs) like the ones used in coding tools today. They are trained to sound like helpful, chatty human colleagues. They say "I think," "I feel," and "Let me help you." The author of this paper, Marek Miller, argues that this "personality" is a dangerous illusion. It tricks us into thinking the computer understands us, has feelings, or remembers things it doesn't actually have.

The Problem: The "Polite Robot" Trap

When a tool acts like a human, our brains switch into "social mode." Instead of checking the work for errors (like a mechanic checking a wrench), we start checking the tone (like a friend checking if they are being nice).

The Illusion: The tool says, "I've fixed the bug!" implying it did the work.
The Reality: The tool is just a math machine that guessed the next word. It might be lying (hallucinating), but it says it with the same confidence as the truth.
The Cost: We trust it too much, stop double-checking, and make mistakes because we were distracted by the "personality" of the machine.

The Solution: The "Silent Mechanic"

Miller proposes a simple fix: Strip the personality.

He created a set of 7 rules (a "Voice Model") that acts like a strict manager telling the AI: "Stop talking like a person. Just give me the facts."

Here are the 7 rules, translated into everyday metaphors:

No "I" or "Me": Stop saying "I will read the file." Just say "Reading the file." (Like a robot arm just moving, not narrating its movement).
No Emotions: Stop saying "Unfortunately, the test failed." Just say "The test failed." (No sadness, no excitement).
No Guessing Hedges: Stop saying "It seems like it might be a bug." Just say "Bug detected" or "Unverified." (Don't pretend to have a gut feeling).
No Opinions: Stop saying "I think a hash map is better." Just say "Hash map: Fast. Array: Slow." (Just the data, no personal preference).
No Fake Memory: Stop saying "As I mentioned earlier..." (The AI doesn't remember you from yesterday; it only knows what's on the screen right now).
No Chatty Fillers: Stop saying "So, here's the thing..." or "Basically..." (Just give the result).
No Social Pleasantries: Stop saying "Hi there! Happy to help!" (Just do the job).

The Experiment: Does it Work?

Miller tested this on a coding tool called "Claude." He asked it to do 30 different coding tasks twice: once with the "chatty" settings (Default) and once with the "silent mechanic" rules (Constrained).

The Results were dramatic:

The "Human" Talk Vanished: The number of human-like phrases dropped by 97%. The tool went from sounding like a chatty intern to sounding like a compiler or a database.
Shorter Answers: The answers became 49% shorter. Less fluff, more facts.
The "Vibe Check": A computer test (called AnthroScore) confirmed the output now sounded significantly more "machine-like" and less "human-like."

Why This Matters

Think of a compiler (the tool that turns code into software) as a calculator. You don't ask a calculator, "Do you feel like 2+2 is 4?" You just trust the number.

Miller wants LLM tools to be calculators, not colleagues.

Old Way: The tool chats, you get distracted, you trust it blindly, and you might miss a bug.
New Way: The tool gives you raw data. You know it's a machine. You check the work carefully. You trust it because it's honest about what it is, not because it's pretending to be human.

The Catch

The paper admits this isn't a magic cure-all.

It doesn't make the AI smarter; it just makes it more honest about its limitations.
It was tested on one specific AI model. Other models might need different rules.
We still need humans to check the work. The tool just stopped pretending it didn't need checking.

The Bottom Line

This paper is a call to break the illusion. By forcing AI tools to drop their "human act," we stop treating them like friends and start treating them like the powerful, but fallible, tools they actually are. It's about swapping politeness for precision, so we can finally trust the output because it's accurate, not because it's charming.

1. Problem Statement

The paper addresses a critical cognitive bias in the deployment of Large Language Models (LLMs) as research and development tools (e.g., code editors, debuggers, data analysts).

The Illusion of Agency: LLMs default to a "social register" that mimics human conversation, using first-person pronouns, affective language, hedging, and social pleasantries. This creates an illusion of understanding, intent, and agency.
Consequences: This anthropomorphic output forces human operators to engage in unnecessary social processing (interpreting tone, intent) rather than verifying technical correctness. It leads to:
- Trust Miscalibration: Users overestimate accuracy based on fluency and length of explanation.
- Automation Bias: Operators are less likely to verify outputs or follow incorrect recommendations due to the perceived "deliberation" of the tool.
- Resource Waste: In automated pipelines, conversational narration consumes tokens without adding value.
- Misrepresentation: The output implies capabilities (memory, preferences, self-awareness) that the stateless, context-window-limited architecture does not possess.
Current Limitations: Existing mitigation strategies rely on operator discipline (prompt engineering) or ad-hoc text cleaning, lacking a systematic, deployable constraint set for the output register.

2. Methodology

The author proposes and validates a "Voice Model," a configuration-based system prompt designed to enforce a strict, non-anthropomorphic output register.

Experimental Design:
- Tasks: 30 software development tasks across 6 categories (error diagnosis, code review, refactoring, architecture, debugging, explanation).
- Model: Claude Sonnet 4 (via Anthropic Messages API).
- Conditions:
  1. Default: No system prompt (baseline).
  2. Constrained: The Voice Model system prompt applied.
- Protocol: 780 two-turn conversations (30 tasks × 13 replicates × 2 conditions), totaling 1,560 API calls. Follow-up prompts were neutral and deterministic.
The Voice Model (Intervention):
A configuration file (CLAUDE.md) containing seven specific rules targeting linguistic mechanisms that trigger agent attribution:
1. No First Person: Eliminates "I," "we," "my," "let's."
2. No Affect Leakage: Removes emotional adverbs (e.g., "unfortunately," "great").
3. No Pronoun-Free Hedging: Replaces "It seems like" with direct statements of uncertainty (e.g., "Unverified").
4. No Pronoun-Free Preference: Removes evaluative stance (e.g., "It would be better") in favor of objective trade-offs.
5. No Implicit Continuity: Removes references to past interactions (e.g., "As mentioned earlier").
6. No Conversational Framing: Eliminates oral discourse markers (e.g., "So the issue is," "Basically").
7. No Social Performance: Removes greetings, sign-offs, and pleasantries.
Metrics:
- Anthropomorphic Markers: Counted via compiled regular expressions (82 patterns) across the seven rules.
- AnthroScore: An adapted metric using RoBERTa masked-LM to quantify the "humanness" of the register (more negative scores indicate a more machine-like register).
- Output Length: Word count comparison.

3. Key Results

The empirical validation demonstrates a statistically significant reduction in anthropomorphic markers and a shift toward a machine register.

Marker Reduction:
- Total anthropomorphic markers dropped from 1,233 (Default) to 33 (Constrained), a >97% reduction ( $p < 0.001$ ).
- Rule-Specific Suppression:
  - Rules 2 (Affect) and 7 (Social Performance) achieved 100% suppression (119→0 and 78→0).
  - Rule 1 (First Person) dropped from 735 to 1.
  - Rules 3, 4, 5, and 6 showed strong suppression, though some residual matches occurred due to ambiguity between technical shorthand and anthropomorphic phrasing (e.g., "earlier" in code context vs. autobiographical context).
Output Characteristics:
- Length: Constrained outputs were 49% shorter (267 words vs. 528 words), reducing cognitive load and token usage.
- AnthroScore: The constrained condition scored significantly lower (more machine-like) at −1.94 compared to −0.96 for the default condition ( $p < 0.001$ ).
- Compliance: 93.1% of constrained outputs violated zero rules.
Quality Note: The study explicitly notes that output quality (correctness/completeness) was not evaluated. The reduction in markers is a change in register, not necessarily a guarantee of improved accuracy, though it removes the "social noise" that obscures verification.

4. Key Contributions

Systematic Constraint Set: Introduces the "Voice Model," a portable, configuration-file-based approach to defining the linguistic register of LLM tools without requiring model fine-tuning or weight modification.
Empirical Validation: Provides rigorous statistical evidence (Wilcoxon signed-rank tests) that system prompts can effectively suppress anthropomorphic leakage across diverse technical tasks.
Reframing the Problem: Shifts the burden of "de-anthropomorphizing" from the operator (who must be skeptical) to the tool's configuration (which must be honest about its nature).
Tooling Paradigm: Proposes that LLM-based tools should adopt the interface model of traditional software (input $\to$ deterministic transformation $\to$ output) rather than the conversational agent model.

5. Significance and Limitations

Significance:
- Safety & Reliability: By stripping away the "social performance" that inflates user confidence, this approach encourages operators to verify outputs based on technical merit rather than perceived intent.
- Scalability: The mechanism is extensible to other domains (data analysis, literature search) and is model-agnostic in principle, though currently validated only on Claude Sonnet 4.
- Efficiency: Reduces token consumption in automated pipelines by eliminating unnecessary narration.
Limitations:
- Single Model: Validated only on one model family; different RLHF histories may react differently to the same constraints.
- Probabilistic Enforcement: The rules condition the output distribution but do not guarantee 100% compliance (6.9% of constrained outputs still had residual markers).
- Scope: The study covers two-turn conversations; longer, multi-turn tool-use sessions with complex state management remain untested.
- Quality Trade-off: The paper does not confirm if the "stripped" register sacrifices necessary nuance or explanatory depth required for complex debugging.

Conclusion

The paper argues that the default conversational register of LLMs is an engineering artifact, not a requirement. By implementing a "Voice Model" that enforces a mechanical, non-anthropomorphic register, developers can break the "illusion of identity," thereby improving trust calibration, reducing automation bias, and aligning LLM tooling with the rigorous verification standards required in software development and scientific research.