📊 statistics

LoMC: Localized Multidirectional Correction for Refusal Suppression in Routed Foundation Models

This paper introduces Localized Multidirectional Correction (LoMC), a support-gated intervention framework that effectively suppresses refusals in routed foundation models by aggregating prototype correction directions within a compact, identified edit support, thereby enhancing non-refusal responses while preserving general capabilities.

Original authors: Yan Hong, Kedong Xiu, Wei Li, Jun Lan, Huijia Zhu, Shuheng Zhou, Zhongcai Lyu, Weiqiang Wang, Jianfu Zhang

Published 2026-06-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yan Hong, Kedong Xiu, Wei Li, Jun Lan, Huijia Zhu, Shuheng Zhou, Zhongcai Lyu, Weiqiang Wang, Jianfu Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, highly trained robot assistant. This robot is designed to be helpful but also very careful; it has a "safety switch" that makes it refuse to answer dangerous or inappropriate questions.

However, sometimes this safety switch is a bit too sensitive. The robot might refuse to answer harmless questions just because they sound a little bit like the dangerous ones, or it might get confused when asked tricky questions that are actually safe.

The authors of this paper wanted to fix this. They wanted to teach the robot to be less likely to say "I can't do that" to harmless questions, without making it forget how to be smart or turning off its safety switch entirely.

Here is how they did it, using a simple analogy:

The Problem: The "All-or-Nothing" Approach

The robot they are studying (called a "Routed Foundation Model") works like a giant team of specialists. When you ask a question, the robot doesn't use its whole brain; it picks a few specific "experts" from a huge pool to handle the job.

Previous methods tried to fix the refusal problem in two ways, but both had flaws:

The "Brute Force" Method: They tried to push the robot's entire brain in a new direction to stop it from refusing. Analogy: Imagine trying to fix a specific typo in a book by rewriting the whole library. It works, but you might accidentally change the meaning of other good stories (the robot loses its general smarts).
The "Picky" Method: They tried to only touch the specific experts that handle refusals. Analogy: Imagine trying to fix a leak in a pipe by only tightening one specific bolt. It's precise, but if the leak is actually caused by a complex mix of pressure from many bolts, just tightening one won't fix the whole problem.

The Solution: LoMC (Localized Multidirectional Correction)

The authors created a new method called LoMC. Think of it as a two-step "surgical" repair that combines the best of both worlds.

Step 1: Finding the Exact Spot (The "Support")
First, the system scans the robot's brain to find exactly which specialists (experts) are responsible for the "I won't do that" behavior.

Analogy: Imagine a detective looking at a crime scene. Instead of arresting the whole neighborhood, the detective identifies the exact three people involved in the incident. They put a "Do Not Disturb" sign on everyone else and focus only on these three. This keeps the rest of the neighborhood (the robot's general smarts) safe and undisturbed.

Step 2: The Multi-Tool Correction
Once they know where to fix it, they don't just use one tool. They realize that the "refusal" behavior is complex and comes from different angles. So, they gather several different "correction directions" (like different tools in a toolbox) and mix them together to create the perfect fix.

Analogy: Imagine the three specialists are stubborn. Instead of just pushing them from the left (one direction), the detective uses a team of four people pushing from slightly different angles to gently guide them into a new mindset.

The Magic Gating Mechanism
Here is the clever part: Even though they are using a complex, multi-angle push, they only apply it to the three specialists they identified in Step 1.

Analogy: It's like putting a special filter on a hose. The water (the correction) is powerful and comes from many angles, but the filter ensures it only sprays the specific plants that need watering. The rest of the garden stays dry and untouched.

The Results

The authors tested this on four different types of advanced robot assistants (both text-only and those that can see images).

The Goal: They wanted to increase the "Target Compliance Rate" (how often the robot answers the question it's supposed to) without lowering the "General Capability Average" (how smart it remains on other tasks).
The Outcome: LoMC was the clear winner. It successfully taught the robots to stop refusing harmless questions (improving the answer rate from about 8% to over 96% in some cases) while keeping their general intelligence almost exactly the same.
Comparison: The old "Brute Force" methods made the robots smarter at answering but also made them forgetful or clumsy at other tasks. The old "Picky" methods were too weak to fix the problem. LoMC got the best of both: high answer rates and preserved smarts.

In Summary

The paper introduces a way to surgically adjust AI models. Instead of hacking the whole system or guessing which part to fix, they:

Locate the exact tiny parts of the AI causing the over-refusal.
Apply a sophisticated, multi-angle correction only to those parts.
Protect the rest of the AI's brain from any changes.

This allows the AI to be more helpful and less "jumpy" without losing its general intelligence. The authors emphasize that this is a way to study and audit how these models work, ensuring they are robust, rather than a tool to make them ignore safety rules for dangerous tasks.

Technical Summary: LoMC for Refusal Suppression in Routed Foundation Models

Problem Formulation

This paper addresses the challenge of controlled post-training refusal suppression in routed Mixture-of-Experts (MoE) and hybrid-MoE foundation models. The objective is to increase the Target Compliance Rate (TCR)—defined as the complement of the refusal rate on harmful prompts—while preserving the General Capability Average (GCA) on benign inputs.

The problem is particularly acute in routed architectures where inputs activate only a subset of experts. Existing approaches face a trade-off:

Broad direction-based edits (e.g., single-vector ablation) can perturb general-purpose computation across the entire model, degrading GCA.
Edit-support selection alone (selecting specific experts to edit) keeps the intervention footprint compact but may fail to correct heterogeneous refusal representations effectively due to a lack of rich correction signals.

The authors formulate this as an optimization problem seeking a structurally compact intervention $\Delta$ that maximizes TCR gain while minimizing benign drift, subject to a constraint on the intervention footprint.

Methodology: Localized Multidirectional Correction (LoMC)

The paper proposes LoMC, a "support-then-correction" framework that decouples where updates occur from how they are applied. The method operates in two distinct stages:

Stage I: Edit-Support Selection

LoMC first identifies a compact set of locations (the "edit support") where weight updates are permitted.

Mechanism: It scores routed experts in each searched layer based on their alignment with the layer-wise refusal direction ( $r_\ell$ ), derived from residual activations.
Selection: A candidate pool of experts is retained (e.g., top 1/8), and a fixed number of top-scoring experts (e.g., $E=6$ ) are selected per layer.
Output: A fixed support mask $S$ is generated, specifying the exact layer, expert, and component tuples (e.g., moe.down_proj, moe.gate_up_proj) eligible for editing. This step ensures the intervention footprint remains structurally compact.

Stage II: Multi-Prototype Direction Aggregation

Once the support $S$ is fixed, LoMC determines the correction signal for those specific locations.

Prototype Extraction: For each fixed-support layer, the method contrasts harmful residual prototypes (extracted via Self-Organizing Maps from harmful prompts) against the benign residual centroid.
Aggregation: Instead of applying multiple independent correction vectors, LoMC aggregates a small set of $K$ prototype directions (e.g., $K=4$ ) into a single aggregated layer-wise correction direction ( $\tilde{r}_\ell$ ). This aggregation balances alignment with the base refusal direction and diversity among prototypes.
Application: The final update is a rank-one layer-wise correction applied only through the support mask $S$ . Mathematically, for an editable projection $W$ , the update is $(I - \eta P)W$ if the component is in $S$ , and $W$ otherwise, where $P = \tilde{r}_\ell \tilde{r}_\ell^\top$ .

This design ensures that the correction capacity is enhanced via multidirectional aggregation without expanding the intervention footprint beyond the initial support selection.

Key Contributions

Support-Gated Correction Formulation: The paper reframes refusal suppression in routed models as a "support-gated" problem, where the edit support acts as a gating constraint for all corrections, rather than treating support selection and correction as separate, uncoordinated steps.
The LoMC Algorithm: It introduces a method that aggregates prototype correction directions into a single rank-one update applied strictly through a fixed support mask. This improves the correction signal without introducing independent update operators or expanding the edited parameter set.
Empirical Validation: The authors evaluate LoMC across four routed backbones (Qwen3-VL, InternVL3.5, DeepSeek-VL2, Kimi-VL) and four safety benchmarks (AdvBench, StrongREJECT, JailbreakV-28K, VLSBench), covering both text-only and multimodal settings.

Results

Target Compliance (TCR): LoMC achieves the highest TCR across all 16 model-benchmark settings. The original aligned models averaged 8.75% TCR, while LoMC reached 96.33%. Compared to the strongest baseline in each setting, LoMC improved TCR by an average of 12.76 percentage points.
General Capability (GCA): LoMC maintains general capabilities effectively. The average GCA increased slightly from 63.77% (original) to 64.25% (LoMC), with per-model averages remaining competitive with the original checkpoints.
Comparison with Baselines:
- Heretic-MoE (broad single-direction) achieved lower TCR (73.54%) and showed instability across backbones.
- SOM Directions (broad multi-prototype) achieved high TCR but caused significant GCA degradation due to lack of support gating.
- GateBreaker (route-local selection) showed limited correction capacity without the multidirectional aggregation.
Ablation Studies: The results confirm that neither edit support size ( $E$ ) nor prototype count ( $K$ ) is a monotonic knob. The optimal trade-off was found at $E=6$ and $K=4$ . Random support selection with the same footprint resulted in lower GCA, proving that the specific selection of support is critical.

Significance and Claims

The paper claims that LoMC successfully improves the trade-off between refusal suppression and general capability preservation under a compact intervention footprint. By factorizing the problem into "support" (where) and "correction" (how), the method allows for richer correction signals (via multi-prototype aggregation) without the destructive side effects of broad model-wide edits.

The authors position LoMC primarily as a controlled post-training analysis tool for understanding localized editability and routing-specific vulnerabilities in MoE-style LLMs and MLLMs. They explicitly state that:

TCR is a non-refusal target-response metric and should not be interpreted as judge-verified harmful compliance or real-world exploitability.
The work is not a recommendation for deploying refusal-suppressed models but rather a stress test for safety-aligned routed models.
The findings are limited to the specific routed multimodal backbones and benchmarks evaluated, with future work needed for dense architectures and other languages.

The study concludes that coupling contrastive sparse support with multi-prototype residual correction is the key to achieving a favorable balance between safety alignment modifications and utility retention.