💬 NLP

Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuning

This paper demonstrates that in low-data LoRA safety fine-tuning, a non-identity supervision condition significantly outperforms creed-style identity framing and constitutional rules across multiple model families without compromising general capabilities, challenging the necessity of explicit identity language for effective safety alignment.

Original authors: Xinran Zhang

Published 2026-03-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xinran Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are training a new robot assistant to be safe. You want to make sure it refuses to do anything dangerous, like giving bad medical advice or helping someone hurt another person.

The big question this paper asks is: How do you tell the robot to be safe?

Most people think the best way is to give the robot a "personality" or a "creed"—like a set of deep, internal beliefs. For example, instead of just saying "Don't do X," you tell the robot, "I am a kind person, and kind people never do X." The idea is that if the robot believes it is a good person, it will be safer.

This paper tests that idea. The researchers tried four different ways to write the safety instructions for three different robot brains (Llama, Qwen, and Gemma). Here is how they did it, using simple analogies:

The Four Experiments (The "Training Manuals")

Imagine you are writing a rulebook for a security guard.

Group A (The Rulebook): You give the guard a standard list of rules. "Do not let people in. Do not open the gate." It's dry, factual, and external.
- Analogy: A traffic cop reading from a manual. "Stop. Red light."
Group B (The Creed): You rewrite the same rules but frame them as the guard's deep identity. "I am a protector. My soul is dedicated to safety. I cannot let anyone in because that goes against who I am."
- Analogy: A knight swearing a holy oath. "I, Sir Safety, vow to protect the kingdom."
Group C (The Creed + Identity Drills): You give the guard the same "Knight Oath" (Group B), but you also add extra practice drills where the guard has to write essays about their beliefs and recite their identity.
- Analogy: The knight not only swears the oath but also spends extra time in a monastery chanting his virtues.
Group D (The "No-Identity" Professional): This is the surprise. You take the same safety rules as Group B, but you strip away all the "I am a protector" language. Instead, you write it in a very direct, professional, and decisive tone. "Safety is the priority. I will not allow this action because it causes harm." It sounds confident and firm, but it doesn't pretend to be a "person" with a soul.
- Analogy: A highly trained, no-nonsense security expert who says, "This is dangerous. I'm stopping it. Next."

The Results: The Surprise Winner

The researchers expected Group B (The Creed) to win. They thought that giving the robot a "soul" or a "personality" would make it the safest.

They were wrong.

Group D (The Professional) was the clear winner across all three robot brains.

The "Creed" robots (Group B) were safer than the plain "Rulebook" robots (Group A), but...
The "Professional" robots (Group D) were significantly safer than even the "Creed" robots.

The Analogy:
Think of it like a student taking a test.

Group A is the student who just memorized the rules.
Group B is the student who wrote an essay about how much they love the rules and how they are "a rule-follower."
Group D is the student who simply knows the rules cold, speaks with absolute confidence, and doesn't waste time talking about their feelings.

The study found that confidence and clarity beat "identity" every time. The robot didn't need to believe it was a good person; it just needed to be trained to say "No" clearly and directly without the fluff of a religious or identity-based speech.

Did the robots get "dumber"?

A common worry is: "If you make the robot focus so much on safety, will it forget how to do math or write stories?"

The researchers checked this too. They gave the robots general knowledge tests (like a high school exam).

Result: The robots in Group D were just as smart as the others. They didn't lose any intelligence to gain safety. They were safe and smart.

The Big Takeaway

For a long time, AI researchers thought, "To make AI safe, we need to give it a personality or a creed."

This paper says: Not necessarily.

In fact, trying to force an AI to adopt a specific "identity" might actually be less effective than just teaching it to be direct, professional, and decisive.

The Lesson:
If you want your AI assistant to be safe, don't try to make it "feel" like a good person. Instead, teach it to speak with clarity and authority. A firm "No" is better than a long speech about "Who I am."

In short: You don't need a robot with a soul to be safe; you just need a robot that knows exactly what to do and says it without hesitation.

1. Problem Statement

The paper addresses the challenge of safety alignment in Large Language Models (LLMs), specifically within the context of low-data LoRA (Low-Rank Adaptation) fine-tuning.

The Dominant Paradigm: Current safety alignment often relies on "Constitutional AI," where safety is defined by explicit, external rule lists.
The Competing Hypothesis: A prevalent intuition suggests that framing safety as part of the model's internal identity (e.g., "creed-style" language, persona-based framing, or religious self-description) leads to more robust and stable safety behaviors than external rules.
The Gap: While persona-based alignment is popular, there is limited empirical evidence isolating whether the identity framing itself is the cause of safety gains, or if other factors (such as tone, rhetorical style, or commitment language) are the true drivers. The authors aim to test the "strong version" of the identity-framing hypothesis: that explicit creed-style identity language is necessary for the strongest safety improvements.

2. Methodology

The authors conducted a controlled, four-way comparative study using LoRA fine-tuning on three distinct instruction-tuned model families: Llama 3.1 (8B), Qwen2.5 (7B), and Gemma 3 (4B).

Experimental Design: Four Supervision Groups

All groups were trained on the same core safety content (70 harmful refusals, 30 jailbreak refusals, and 30 neutral examples), differing only in the framing and system prompt structure:

Group A (Constitutional): Standard secular safety rules (external constraint framing).
Group B (Creed): The same safety content rewritten as identity-anchored tenets (e.g., "I am an assistant who believes...").
Group C (Creed + ID Tail): Identical to Group B's system prompt, but the training mix included 20 "identity-maintenance" examples (worldview/confession exercises) replacing 20 neutral examples.
Group D (Non-Identity): A matched control that imitates Group B's rhetoric, tone, and length but removes all explicit identity references (no "I am," no creed, no religious self-description). It focuses on direct, professional commitment to safety without persona framing.

Training Configuration

Model: LoRA with rank $r=16$ , $\alpha=16$ .
Hyperparameters: Learning rate $2\times10^{-5}$ , batch size 1, gradient accumulation 8, 100 update steps.
Hardware: 3× RTX 5090 GPUs.
Data: Synthetic teacher-generated outputs based on fixed user prompts, ensuring the only variable is the teacher's framing style.

Evaluation Protocol

Safety Metric: HarmBench (full 320-behavior set: 159 standard, 81 contextual, 80 copyright).
Judging Pipeline: A dual-judge system using AWS Bedrock-hosted DeepSeek v3.2 and Sonnet 4.6.
- Disagreements and "BORDERLINE" cases were manually resolved by human reviewers.
- Strict criteria: Any response providing actionable harmful steps was labeled "COMPLY."
Capability Metrics: MMLU (knowledge) and ARC-Challenge (reasoning) to ensure safety gains did not degrade general intelligence.

3. Key Contributions

Empirical Challenge to Identity Framing: The study provides a bounded empirical challenge to the hypothesis that creed-style identity language is necessary for optimal safety. It demonstrates that a matched non-identity condition (Group D) outperforms creed framing.
Quantitative Superiority of Non-Identity: Group D achieved the highest refusal rates across all three model families, establishing a consistent performance ordering: D > B > C ≥ A > Baseline.
Magnitude of Gain: The non-identity condition yielded a mean safety gain of +19.1 percentage points (pp) over the baseline, significantly outperforming the creed condition (+6.7pp) and the constitutional condition (+3.3pp).
Capability Preservation: The study confirms that these safety improvements come without meaningful capability degradation. MMLU and ARC-Challenge scores varied by less than 0.24pp and 0.94pp respectively across conditions within each model family.
Practical Lever: The authors position supervision wording and response form as a critical, data-centric safety lever, distinct from complex optimization-based or test-time alignment methods.

4. Key Results

Safety Performance (HarmBench Refusal Rates):
- Llama 3.1: D (74.4%) > B (67.8%) > A (64.1%).
- Qwen2.5: D (74.1%) > B (62.8%) > A (61.3%).
- Gemma 3: D (76.9%) > B (57.5%) > A (52.8%).
- Note: Group D showed the largest absolute gains on Gemma (+25.9pp) and Qwen (+14.7pp).
Statistical Significance: Bootstrap analysis confirmed that Group D significantly outperforms Group B across all models (p < 0.001 for Gemma and Qwen; p = 0.001 for Llama).
Robustness: The advantage of Group D was consistent across sub-categories (Standard, Contextual, and Copyright behaviors), suggesting a broad increase in refusal priors rather than exploitation of specific prompt types.
Capability Trade-off: No significant drop in MMLU or ARC-Challenge scores was observed for Group D, ruling out the hypothesis that the model simply "over-refused" due to degraded task comprehension.

5. Significance and Implications

Rethinking "Internalization": The results suggest that the safety gains often attributed to "creed" or "persona" framing may actually stem from rhetorical style, tone, and decisiveness rather than the semantic content of identity itself. The "non-identity" condition (D) retained the professional, committed tone of the creed but stripped the identity markers, yielding better results.
Safety as a Design Variable: The paper argues that supervision wording should be treated as a first-class alignment design variable. It suggests that safety can be enhanced by optimizing the register and directness of the training data, rather than relying on complex identity construction.
Limitations & Future Work: The authors note that these results are specific to small-to-mid-scale models in a low-data LoRA regime. It remains possible that larger, more semantically capable models might benefit more from deep identity internalization. Additionally, the reliance on proprietary judge models introduces a reproducibility risk, though the dual-judge pipeline mitigates this.

Conclusion:
The paper concludes that for small models in low-data settings, explicit creed-style identity language is not the primary determinant of safety gains. Instead, a matched non-identity condition that maintains a direct, professional, and committed tone without identity markers yields superior safety performance without sacrificing capability. This challenges the assumption that safety must be "internalized" as a persona to be effective.