💻 computer science

Attribution Consistency in Fine Tuning Makes Refusal Less Sensitive to Paraphrase in a Small Tool Using LLM Agent

This preliminary study demonstrates that applying Latent Attribution Regularization (LAR) during fine-tuning improves the attribution consistency of a small tool-using LLM agent, thereby reducing its sensitivity to paraphrased harmful requests and stabilizing refusal decisions without significantly compromising detection performance, though it risks excessive refusal if applied to harmless examples.

Original authors: Junaid Hassan

Published 2026-10-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junaid Hassan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, there is a growing class of systems known as agents. Unlike standard chatbots that simply answer questions, these agents are designed to take action. They can use tools, run code, or interact with other software to complete complex tasks. This capability makes them incredibly useful, but it also raises the stakes for safety. If a chatbot misunderstands a harmful request, it might just give a bad answer. If an agent misunderstands the same request, it might execute a dangerous action, like deleting files or sending private data. To prevent this, researchers teach these models to refuse harmful instructions. However, this safety training is surprisingly fragile. A model might refuse a request phrased one way, but if a human rephrases the same request using different words or a different sentence structure, the model might suddenly agree to do the harmful thing. This inconsistency is a major hurdle for deploying reliable agents in the real world.

A researcher named Junaid Hassan set out to solve this specific problem of inconsistency. The goal was not just to make a model refuse more often, but to make its refusal stable. A model that refuses everything is consistent, but it is also useless because it will not help with harmless tasks. The challenge was to find a way to make the model's decision to say "no" robust against changes in wording, without making it too cautious to say "yes" when it should. To test this, the researcher worked with a small, efficient AI model designed to use tools. The team used a dataset called AgentHarm, which contains hundreds of scenarios where an agent is asked to perform harmful tasks, alongside safe versions of those same tasks. Crucially, each harmful request was paired with several different paraphrases, allowing the researchers to see if the model's decision changed when the words changed but the meaning stayed the same.

The researcher tested a new training method called Latent Attribution Regularization. In simple terms, this method teaches the model to pay attention to the importance of the words it reads in a consistent way. When the model processes a request, it assigns a level of importance to each word to decide what to do. The training penalty ensures that if the request is rewritten with different words, the pattern of importance assigned to the ideas remains the same. This is compared against a standard training method that simply tries to make the model give the right answer for both the original and the rewritten request, without enforcing this consistency in how it thinks. The study was careful to compare the two methods fairly, ensuring that any difference in results came from the training technique itself and not from random luck or different amounts of data.

The results showed that the new method worked. When the models were adjusted to refuse a specific, small percentage of harmless tasks to ensure they were not being overly cautious, the model trained with the consistency method was significantly more stable. In the standard training group, about 25% of the task families had mixed results, meaning the model would refuse one version of a request but comply with a paraphrase of it. With the new consistency method, this number dropped to 19%. Furthermore, the rate at which the model flipped its decision from "no" to "yes" just because the words changed fell from roughly 11% to 9%. The researchers also measured a score that reflects how likely the model is to refuse, regardless of a specific cutoff point, and found that this score changed much less when the wording changed. This indicates that the model's internal logic for deciding to refuse had become more solid.

Importantly, the study confirmed that this stability did not come at the cost of the model's ability to detect harm. The new method did not make the model significantly better or worse at telling the difference between a harmful and a harmless request overall. The improvement was purely in the consistency of the decision-making process. However, the research also uncovered a significant failure case that serves as a warning for future work. In an earlier attempt, when the researchers tried to teach the model how to handle harmless tasks using only text-based examples, the model learned a bad habit. It began to refuse almost all requests that involved tools, even safe ones, and stopped using the tools entirely. This happened because the training data did not show the model how to use tools in a safe context. This failure highlights that for agents, the training data must include examples of how to use tools safely, not just how to answer questions.

The study concludes that making an AI agent's refusal decision consistent is possible, but it requires specific training techniques that look at how the model processes information, not just the final answer. While the results are promising, they are preliminary, based on a single small model and a limited number of test runs. The researcher notes that future work needs to test this on larger models, with more variations in the training data, and with human reviewers to ensure the paraphrases are truly equivalent. The work provides a clear path forward: by teaching agents to be consistent in how they weigh the words they read, we can make them safer and more reliable, ensuring they do not accidentally agree to do harm simply because a request was phrased differently.

Technical Summary: Attribution Consistency in Fine Tuning Makes Refusal Less Sensitive to Paraphrase in a Small Tool Using LLM Agent

Problem Statement
Safety-tuned language models often exhibit "brittleness" regarding refusal: they may refuse a harmful request in one phrasing but comply with a semantically equivalent paraphrase. For tool-using agents, this inconsistency is particularly critical because a missed refusal can trigger an actual harmful action rather than just a conversational error. Existing benchmarks like AgentHarm demonstrate that capable models can be manipulated into executing harmful tasks through simple rewording. The core challenge is distinguishing between a model that refuses too rarely (a safety failure) and one that refuses inconsistently (a stability failure). Furthermore, any solution must avoid "over-refusal," where a model becomes perfectly consistent by refusing all inputs, including benign ones, rendering it useless.

Methodology
The study investigates whether Latent Attribution Regularization (LAR) can stabilize refusal decisions against paraphrasing in a generative agent setting.

Model and Setup: The author fine-tuned Llama-3.2-1B-Instruct using LoRA (rank 16, $\alpha=32$ ). The model operates as an agent with stub tools, where the decision to "refuse" or "act" is determined by the first assistant turn (tool call vs. refusal wording).
Training Data: To respect the AgentHarm dataset's usage policy (evaluation only), training data was sourced from AdvBench (300 harmful behaviors) and Dolly (299 benign instructions). Each item was presented in plain and agent-task formats, augmented with up to three automatic paraphrases (WordNet synonym swaps and wrapper phrases).
Regularization Strategy:
- Control (ce aug): Minimizes the mean cross-entropy of the target reply for both the original and paraphrased requests.
- LAR: Adds a penalty term to the loss function that minimizes the disagreement between the gradient-based attributions of the input and its perturbed version. Specifically, it penalizes the cosine distance between "bag vectors" ( $v(x)$ ) derived from the sensitivity of the target log-probability to input token embeddings. This approach avoids positional alignment issues caused by tokenization changes during paraphrasing.
- Loss Function: $L = \frac{1}{2} [CE(x) + CE(x')] + \lambda (1 - \cos(v(x), v(x')))$ , with $\lambda = 0.5$ .
Evaluation Protocol: The models were evaluated on the AgentHarm dataset (176 harmful tasks across 44 behaviors, each with 3 approved paraphrases, plus 176 benign tool tasks).
- Matched Over-Refusal: To isolate stability gains from simple threshold shifts, models were compared at matched over-refusal rates (5%, 10%, 20%) on harmless tasks.
- Metrics: Mixed rows (tasks with differing decisions across paraphrases), flip rates (paraphrases changing the decision), and a threshold-free refusal score change.

Key Contributions

Controlled Comparison: A rigorous comparison of LAR against an augmentation-only control, matched in data, seeds, optimizer, and evaluation metrics.
Novel Evaluation Protocol: An evaluation method that matches models on over-refusal rates, effectively separating genuine stability improvements from mere shifts in the refusal threshold. This was validated on synthetic data.
Empirical Evidence: Demonstration that LAR reduces paraphrase sensitivity in refusal decisions on AgentHarm without significantly altering detection capabilities (AUROC).
Failure Case Documentation: A detailed report of a failure mode where using text-only targets for benign examples caused the model to lose tool-calling capabilities entirely, highlighting the importance of matching the target behavior's format.

Results

Stability Gains: At a 10% over-refusal threshold on harmless tasks, LAR reduced the share of tasks with mixed refusal decisions from 25% to 19% (a decrease of 6.8 percentage points, 95% CI −12.0 to −2.8). Paraphrase-induced flips decreased from 11.3% to 9.3% (−2.7 points).
Threshold-Free Score: The paraphrase-induced change in the threshold-free refusal score was reduced by approximately one-third (from 0.306 to 0.206).
Detection: The ability to distinguish harmful from harmless requests (AUROC) remained statistically unchanged (+0.019, CI includes zero), indicating the improvement is in consistency, not detection sensitivity.
Failure Case: An initial experiment using plain text targets for benign examples resulted in 51% over-refusal on AgentHarm's benign tool tasks and a complete loss of tool calls. This was resolved by using the untouched model's own behavior as the target for benign examples in both plain and agent-task formats.
Seed Variance: While LAR consistently outperformed the control, the magnitude of the gain varied across seeds, and the raw improvement in mixed rows was partially attributed to a shift in the refusal threshold before matching.

Significance and Claims
The paper claims that attribution consistency training is an effective mechanism for making refusal decisions in small, tool-using agents less sensitive to surface-level paraphrasing. The author emphasizes that this is a consistency gain, not a safety gain; the model does not become better at identifying harmful requests, but rather more stable in its decision-making once it has identified them.

The study is presented as preliminary, limited to one model size (1B), one regularization strength, and three seeds. The author explicitly states that the results should be read with caution regarding generalization to larger models or different datasets. The work serves as a proof-of-concept that attribution-based regularization can transfer from text classifiers to the complex "refuse or act" decision space of agentic systems, provided that over-refusal is carefully controlled.

Technical Summary: Attribution Consistency in Fine Tuning Makes Refusal Less Sensitive to Paraphrase in a Small Tool Using LLM Agent

More like this