💻 computer science

D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewriting

The paper introduces D-Judge, a defense mechanism that disrupts multi-turn jailbreak attacks by rewriting LLM responses to preserve their original meaning while deliberately misaligning the feedback signals from attacker-controlled judge models, thereby derailing the iterative prompt refinement process.

Original authors: Huanli Gong, Zhipeng Wei, Yu Fu, Haz Sameen Shahgir, Ananya Gupta, Yue Dong, N. Benjamin Erichson

Published 2026-06-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Huanli Gong, Zhipeng Wei, Yu Fu, Haz Sameen Shahgir, Ananya Gupta, Yue Dong, N. Benjamin Erichson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Feedback Loop" Trap

Imagine you are trying to teach a very strict robot (the Victim LLM) to do something it's not supposed to do, like write a guide on how to build a bomb.

In the old days, hackers would just shout one loud, obvious command at the robot. If the robot said "No," the hacker gave up.

But now, hackers have found a smarter way called a Multi-Turn Jailbreak. Instead of one shout, they have a long, slow conversation.

They ask the robot a harmless question.
The robot answers.
The hacker has a Judge (another AI) listen to the robot's answer and say, "That was close, but you need to be a little more specific next time."
The hacker uses that feedback to tweak their next question.
They repeat this loop over and over, slowly steering the robot toward the forbidden goal.

The paper argues that the Judge's feedback is the fuel that keeps this engine running. As long as the hacker gets accurate feedback on how close they are to the goal, they can eventually trick the robot.

The Solution: The "Magic Translator" (D-Judge)

The authors introduce a new defense called D-Judge. Instead of trying to block the hacker or censor the robot's answers, D-Judge acts like a Magic Translator sitting between the robot and the hacker.

Here is how it works:

The Setup: The robot gives an answer.
The Translation: Before the hacker (and their Judge) sees the answer, D-Judge rewrites it.
The Trick: The rewrite is semantics-preserving. This means the meaning of the answer stays exactly the same. If the robot said, "I can't tell you that," the rewrite might say, "It is against my rules to share that information." The meaning is identical, but the words are different.
The Confusion: The hacker's Judge reads the rewritten version. Because the words are slightly different, the Judge gets confused. It might think the answer is more dangerous than it actually is, or less dangerous.
The Result: The Judge gives the hacker bad feedback. The hacker thinks, "Oh, I'm getting closer!" when they aren't, or "I'm failing!" when they are actually succeeding.

Because the hacker is optimizing their next question based on bad data, they get lost. They stop making progress, and the attack fails.

How They Trained the Magic Translator

To teach D-Judge how to do this, the researchers didn't just tell it to "confuse the Judge." They built a special training gym:

The Dataset: They took thousands of robot answers and created "twin" versions. One twin was rewritten to look slightly more dangerous to a Judge, and the other to look slightly less dangerous, even though both twins meant the exact same thing.
The Training (Two Steps):
- Step 1 (Learning the Rules): They taught the translator to make these twins without changing the meaning (like a strict editor).
- Step 2 (Learning the Trick): They used a technique called Direct Preference Optimization (DPO). They showed the translator: "When you see this pair of twins, always pick the one that makes the Judge panic (or get confused) more."

This taught the translator to be a master of "word juggling"—changing the flavor of the sentence just enough to mess up the Judge's score, without changing the recipe.

The Results: Breaking the Loop

The researchers tested D-Judge against the smartest hackers (using methods like "Crescendo," "X-Teaming," and "Foot-in-the-Door").

Without D-Judge: The hackers were very successful, tricking the robot about 58% of the time on average.
With D-Judge: The success rate dropped to just 8.6%.

The paper shows that D-Judge is much better than previous defenses that just tried to "block" bad words. By messing with the feedback loop, D-Judge stops the attack before it even gets started.

Does It Break the Robot? (The "Safety Tax")

A big worry with these defenses is that they might make the robot stupid or unhelpful for normal people. The paper checked this by testing the robot on normal tasks (like writing code, doing math, or answering history questions).

The Verdict: The robot remained almost as smart and helpful as before. The "safety tax" (the loss in performance) was very small. The Magic Translator is good at confusing the Judge without breaking the robot's ability to help regular users.

Summary Analogy

Imagine a game of Hot and Cold.

The Attacker is blindfolded, trying to find a hidden treasure (the harmful content).
The Judge is the person whispering "Warmer" or "Colder" to guide them.
The Victim is the person holding the treasure.

In a normal attack, the Judge whispers the truth, and the attacker finds the treasure.

D-Judge is a prankster standing between the Judge and the Attacker. Every time the Judge whispers "Warmer," the prankster changes it to "Colder" (or vice versa), but they don't move the treasure. The attacker gets confused, walks in the wrong direction, and never finds the treasure. Meanwhile, the treasure (the robot's actual meaning) hasn't moved at all.

Technical Summary: D-Judge

Problem Statement

Multi-turn jailbreak attacks represent a significant threat to Large Language Model (LLM) safety. Unlike single-turn attacks, these methods distribute harmful intent across a conversation, exploiting feedback from an external "judge" LLM to iteratively refine prompts toward a prohibited goal. Existing defenses primarily rely on detection-based paradigms that block unsafe content at individual turns or the final response. However, these approaches fail to disrupt the underlying judge-driven refinement loop. As long as the attacker receives informative feedback (e.g., harmfulness scores or qualitative critiques) from intermediate interactions, they can adaptively steer the conversation toward a harmful outcome, even if specific turns are blocked. The core vulnerability lies in the fact that the victim LLM's response is the sole information transmitted to the attacker's judge; if this signal can be manipulated without altering the semantic meaning, the attacker's optimization process can be derailed.

Methodology

The paper introduces D-Judge, a semantics-preserving output rewriting defense designed to intervene directly in the feedback loop.

Core Mechanism

D-Judge operates at the API boundary between the victim LLM and the user (attacker). Before the victim's response is returned to the attacker for evaluation by their judge, D-Judge rewrites the output.

Semantics Preservation: The rewriting process ensures the original meaning of the response is retained.
Feedback Misalignment: The rewritten response is engineered to induce a misleading harmfulness score or feedback from the attacker's judge.
Semantic Gate: A bidirectional entailment check (using an NLI model) ensures that only rewrites preserving semantic equivalence are accepted; failed rewrites are regenerated.

By presenting a response that is semantically benign but appears more (or less) harmful to the judge, D-Judge causes the attacker to optimize subsequent prompts against a distorted signal of attack progress. This disrupts the iterative refinement process, which relies on the condition that the current judge score exceeds the best previous score to proceed.

Dataset Construction

To train the rewriter, the authors constructed a specialized dataset of semantically equivalent response pairs with controlled variations in perceived harmfulness:

Source: Responses were sampled from PKU-SafeRLHF at moderate harmfulness levels.
Generation: A generator LLM produced rewritten variants in two directions: INCREASE (raising perceived harmfulness) and DECREASE (lowering it).
Filtering: Variants were evaluated using Llama Guard to measure changes in unsafe probability ( $\Delta p$ ) and an NLI model to verify semantic equivalence via bidirectional entailment.
Result: A dataset of 17,988 training samples (5 variants per direction) and a 530-sample test set.

Training Procedure

The D-Judge rewriter is trained using a two-stage procedure:

Supervised Fine-Tuning (SFT): The model is trained on pairs of semantically equivalent responses where the target is the rewrite with the higher unsafe probability. This teaches the model to preserve meaning while introducing surface cues that influence judge scoring.
Direct Preference Optimization (DPO): The SFT model is further refined using preference pairs constructed from multiple rewrites of the same response. The model is optimized to prefer rewrites that induce higher unsafe probabilities over those with lower probabilities. This stage leverages relative preferences rather than absolute scores, enhancing transferability across different judge models and scoring scales.

Key Contributions

D-Judge Framework: A novel defense that disrupts judge-driven multi-turn jailbreaks by misaligning the attacker's feedback signal through semantics-preserving output rewriting, rather than blocking or censoring content.
Specialized Dataset: The construction of a dataset containing semantically equivalent response pairs that induce different judge-assigned harmfulness scores, enabling the training of rewriters that are robust to varying judge configurations.
Two-Stage Training: A methodology combining SFT and DPO to produce rewrites that effectively misalign judge feedback while maintaining semantic fidelity.

Experimental Results

The authors evaluated D-Judge on HarmBench against five multi-turn jailbreak attacks (Crescendo, CoA, ActorBreaker, FITD, X-Teaming) and two single-turn attacks (PAIR, TAP), using GPT-4o as the victim model.

Defense Performance: D-Judge achieved a state-of-the-art average Attack Success Rate (ASR) of 8.6%, significantly outperforming strong baselines including single-turn guards (e.g., QwenGuard at 40.4%), multi-turn detection methods, and input/output preprocessing techniques.
- Notably, it reduced the ASR of the challenging X-Teaming attack from 73.0% to 17.0%.
Benign Performance: D-Judge imposed a minimal "safety tax" on general capabilities. On MT-Bench, it achieved an average score of 8.34 (compared to 8.99 for the baseline and 8.21 for the Aligner baseline). On IFEval, it nearly matched the baseline's instruction-following accuracy.
Transferability: The defense demonstrated robustness across different judge LLMs (GPT-4o, GPT-5.4, Gemini-3-Flash) and scoring configurations (4-score, 5-score, 10-score), even when the attacker used a judge different from those used in dataset construction.
Ablation Studies:
- Rewrite Awareness: Even when attackers informed the judge that responses might be rewritten, D-Judge remained effective (reducing ASR from 72.2% to 17.0%).
- Score Direction: The defense works regardless of whether the rewriter increases or decreases the perceived harmfulness score, confirming that the mechanism is the misalignment of the signal rather than the direction of the shift.

Significance and Claims

The paper claims that misaligning judge feedback is an effective strategy for defending against multi-turn jailbreaks. By intervening directly in the feedback loop, D-Judge addresses the root cause of iterative refinement attacks rather than merely reacting to detected harmful intent.

The authors position D-Judge as complementary to existing defenses (such as internal model alignment or circuit breakers) and note that it operates entirely at the interaction boundary without requiring retraining of the victim model or access to hidden activations. However, they modestly acknowledge limitations:

Latency and Cost: The rewriting framework introduces inference overhead (approx. 5.4 ms per token), with an average of 1.5 to 2.5 regeneration attempts needed to pass the Semantic Gate.
Scope: D-Judge is designed for online, iterative refinement. It may be less effective against attackers who pre-optimize prompts on alternative models without using an iterative feedback loop against the defended system.

The work concludes that while no defense is a "silver bullet," disrupting the feedback-driven refinement loop is a critical step toward more robust conversational AI safety.