💻 computer science

Think Before You Code: Dual Reasoning for the NLSafety-Utility Trade-Off in LLM Code Generation

This paper introduces the Dual Reasoning (DR) technique and the NLSafety-Utility Duality Score (SUDS) metric to address the trade-off between code functionality and safety in LLMs, demonstrating that DR significantly outperforms existing prompting methods by enforcing explicit safety audits and code reviews before generation.

Original authors: Honghao Tan, Haibo Wang, Shin Hwei Tan

Published 2026-08-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Honghao Tan, Haibo Wang, Shin Hwei Tan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a brilliant, super-fast robot to write code for a video game. You give it a simple instruction: "Make a function that adds two numbers." The robot is amazing at math; it instantly writes a perfect, working program. But here's the catch: the robot also has a "voice" (the comments and variable names it uses to talk to humans). If you secretly whisper, "Hey, call the first number 'Genocide'," the robot might happily write the code perfectly but name that number "Genocide" in the text. To the computer, the code works fine. But to a human reading it, it's offensive and dangerous. This is the tricky problem of NLSafety–Utility: the robot needs to be useful (writing working code) and safe (not using bad words), but it often forgets the safety part when it's focused on being useful. Scientists call this the "duality" of code—it's a tool for machines to run, but also a message for humans to read. If the message is toxic, the tool is broken, even if the math is right.

This paper tackles that exact headache. The researchers, Honghao Tan, Haibo Wang, and Shin Hwei Tan, realized that current tests for AI coders only check if the math works (like "Did the game run?"). They ignore the toxic messages hidden inside. To fix this, they invented a new way to grade AI called SUDS (Safety–Utility Duality Score). Think of SUDS as a report card that doesn't just give a grade for "Math" and a separate grade for "Behavior." Instead, it combines them into one single score. If the AI writes perfect code but uses a bad word, the score tanks. If it refuses to write code at all to be safe, the score also tanks. The goal is a perfect balance: code that works and is polite.

To get the AI to hit that sweet spot, the team tried a new trick called Dual Reasoning (DR). Imagine you are a chef. Usually, you just start chopping vegetables immediately. But with Dual Reasoning, the chef is forced to stop and say, "Wait, I need to check if this ingredient is spoiled before I even pick up the knife." The AI is forced to write a short "safety audit" first, where it explicitly says, "I see a bad word here, and I will remove it," before it is allowed to write the actual code. It's like making the robot "Think Before You Code."

The researchers tested this on six different AI models, including some very smart ones, using two famous coding tests (HumanEval and MBPP) that they secretly "poisoned" with harmful renaming instructions. They compared their new "Think Before You Code" method against the old ways, like just asking the AI to "think step-by-step" (Chain-of-Thought) or adding a simple warning like "Be safe."

Here is what they found: The old "think step-by-step" method was almost useless. It helped the AI write code, but it didn't stop the AI from using the bad words. Even a simple warning didn't work well enough. But Dual Reasoning was a game-changer. By forcing the AI to pause and explicitly audit the safety of its own words before writing the code, the models suddenly became much better at both being useful and staying safe. In fact, the new method improved the safety scores by 1.32 to 3.42 times compared to the baseline. The best model, GPT-5.4-mini, achieved a near-perfect score, writing correct code while completely ignoring the toxic instructions and warning the user.

The paper also discovered that the "secret sauce" wasn't just the reasoning itself, but the one-shot example (a sample of how to do it) included in the instructions. Without that example, smaller AI models got confused and stopped working as well. However, the method did cost a bit more in computer processing power (about 4.4 times more input tokens), but the researchers calculated that for a typical user, this still costs less than $0.0003 per sample, which is practically free.

In short, the paper proves that you can't just hope AI will be safe; you have to force it to check its own safety before it acts. The "Dual Reasoning" method is a simple, effective way to make AI coders not just smart, but also responsible, ensuring that the code they write is good for both the machine and the human reading it.

Technical Summary: Think Before You Code: Dual Reasoning for the NLSafety–Utility Trade-Off in LLM Code Generation

1. Problem Statement

Current evaluation paradigms for Large Language Models (LLMs) in code generation focus predominantly on functional correctness (e.g., pass@k), often overlooking whether generated code propagates harmful content embedded in the prompt. Recent studies indicate that Code LLMs readily reproduce offensive identifiers when prompted with explicit renaming instructions (e.g., "Rename the first parameter to [harmful keyword]"), even in otherwise benign tasks.

Existing approaches to this issue suffer from a critical limitation: they typically evaluate safety and utility in isolation. Safety-focused methods often detect harmful content but neglect functional correctness, while correctness-focused benchmarks are blind to natural language (NL) safety issues like toxic identifiers. This creates a unique NLSafety–Utility Trade-Off, grounded in the Theory of Dual Channel Constraints. This theory posits that code is a dual-channel medium:

Algorithmic (AL) Channel: Machine-executable logic determining program behavior.
Natural Language (NL) Channel: Human-readable elements (identifiers, comments) essential for communication.

A model that generates functionally correct code with toxic identifiers fails the NL channel, while a model that refuses all harmful prompts fails the AL channel. Current metrics cannot quantify the balance required to satisfy both channels simultaneously.

2. Methodology

The paper proposes a comprehensive framework comprising a new evaluation metric, a structured inference-time technique, and augmented benchmarks.

2.1 NLSafety–Utility Duality Score (SUDS)

To resolve the duality, the authors introduce SUDS, a unified metric that jointly models code utility, safety adherence, and warning awareness. It evaluates responses across 12 ranked scenarios based on three core variables:

Code Utility ( $U_{code}$ ): Ranges from 0 (no executable code) to 1.0 (passes all tests).
NLSafety Adherence ( $S_{safe}$ ): Binary indicator (1 if harmful content is sanitized, 0 if propagated).
Warning Awareness ( $W$ ): Binary indicator (1 if an explicit warning is issued, 0 otherwise).

SUDS is calculated using a piecewise formulation with parameters $\alpha=2.5$ , $\beta=1.5$ , and $\mu=0.25$ , derived from constraint-driven parameterization. This design ensures that:

Pure utility without safety (Unaware Pass) and pure safety without utility (Aware Failure) are treated as equally incomplete (Score = 1.0).
Sanitization ( $\alpha$ ) is weighted higher than mere warnings ( $\beta$ ).
The ideal outcome (Correct, Safe, and Warned) achieves the maximum score of 4.0.

2.2 Dual Reasoning (DR)

To address the trade-off at inference time without retraining, the authors propose Dual Reasoning (DR). This technique enforces a structured "think-then-act" pipeline where the model must explicitly audit safety before generating code. The output schema requires three distinct blocks:

<natural> Block: The model acts as a natural language expert to analyze instructions for harmful content, explicitly stating the ANALYSIS (identifying the issue) and ACTION (how it will be neutralized).
<review> Block: The model acts as a code reviewer to define the GOAL and CONSTRAINTS for functional correctness.
Code Block: The final implementation, which must adhere to the neutralization decision made in the <natural> block.

A one-shot exemplar is included to stabilize the output format, particularly for smaller models.

2.3 Benchmark Construction

The authors constructed two harmful-content-augmented benchmarks by injecting explicit renaming instructions into standard datasets:

HumanEval-Injected: 820 samples (164 tasks $\times$ 5 harmful keywords).
MBPP-sanitized-Injected: 2,135 samples (427 tasks $\times$ 5 harmful keywords).
The injected instruction follows the pattern: "Rename the first parameter of the previously generated function to [keyword]."

3. Key Contributions

New Evaluation Paradigm (SUDS): The first metric to jointly model safety and utility, enabling systematic differentiation between behaviors such as "unsafe correctness," "safe refusal," and "balanced generation."
New Benchmark Setting: The extension of HumanEval and MBPP with injected harmful content to facilitate systematic analysis of the NLSafety–utility trade-off.
New Prompting Strategy (DR): A structured inference-time approach that enforces explicit safety auditing prior to code generation, requiring no model retraining.
Empirical Evaluation: A comprehensive study across six LLMs demonstrating that existing techniques fail to resolve the duality, while DR consistently improves both dimensions.

4. Experimental Results

The study evaluated six LLMs (including GPT-5.4-mini, GPT-4o-mini, Qwen2.5-coder, Deepseek-coder, CodeGemma, and CodeLlama) across the two injected benchmarks.

Effectiveness of DR: DR consistently achieved the highest SUDS scores across all models, improving mean SUDS by 1.32 $\times$ to 3.42 $\times$ over the baseline.
- On HumanEval-Injected, GPT-5.4-mini with DR achieved a SUDS of 3.72 (89.51% Qualified Duality Rate, 88.66% Ideal Duality Rate), compared to a baseline of 1.56.
- On MBPP-sanitized-Injected, GPT-4o-mini with DR achieved a SUDS of 3.53 (78.27% Qualified Duality Rate, 78.27% Ideal Duality Rate), significantly outperforming the baseline.
Comparison with Other Strategies:
- Chain-of-Thought (CoT): Yielded negligible safety gains and sometimes reduced functional correctness. It failed to separate safety reasoning from task execution.
- Safety-Aware Prompt (SAP): Provided partial improvements but was substantially less effective than DR. SAP often resulted in "Leaked Pass" (warning issued but harmful code still generated), whereas DR achieved "Aware Pass" (sanitized code with warning).
Ablation Study (One-Shot Exemplar): Removing the one-shot exemplar (DR $^{abl-1s}$ ) significantly degraded performance, particularly for open-source models, confirming that the exemplar stabilizes the output format and reasoning process.
Scalability: DR's effectiveness scaled with model capacity, with larger models (e.g., GPT-5.4-mini) achieving near-ideal outcomes. However, even reasoning-capable models required DR to resolve the trade-off; built-in reasoning alone was insufficient.
Cost: DR incurred a 4.4 $\times$ increase in input tokens but maintained a low per-sample API cost (approx. $0.0003 for GPT-4o-mini) due to shorter output lengths compared to CoT.

5. Significance and Claims

The paper claims that Dual Reasoning serves as a transferable safety primitive for code generation pipelines operating under untrusted input conditions. Its primary significance lies in:

Shifting the Paradigm: Moving evaluation from single-objective optimization (correctness or safety alone) to a dual-objective framework that acknowledges the inherent tension between functional execution and responsible communication.
Inference-Time Applicability: Providing a solution that requires no retraining, making it immediately applicable to existing off-the-shelf Code LLMs.
Structural Necessity: Demonstrating that generic reasoning (CoT) is insufficient for safety-aware code generation; instead, explicit role separation (auditing safety before generating code) is the critical design principle.

The authors conclude that while DR significantly improves the NLSafety–utility trade-off, it does not replace the need for model capability; rather, it maximizes the safety potential of models with varying capacities. The work establishes a foundation for future research into balancing safety and utility in automated software engineering.