🤖 machine learning

Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models

This paper introduces a method called Convergent-Divergent Routing combined with Dual Logit Calibration to achieve fine-grained, interpretable control over large language models' moral reasoning by steering them toward specific ethical frameworks like utilitarianism or deontology while preserving their general capabilities.

Original authors: Chenchen Yuan, Zheyu Zhang, Gjergji Kasneci

Published 2026-05-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chenchen Yuan, Zheyu Zhang, Gjergji Kasneci

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) as a massive, bustling train station. Inside this station, thousands of trains (data pathways) are constantly moving, carrying information to decide what the AI says next. Usually, these trains merge and split in complex ways, and the AI's "moral compass" is just a blurry mix of all the different opinions it has ever read.

Sometimes, we want the AI to act like a strict rule-follower (Deontology), and other times, we want it to act like a calculator of "the greatest good for the greatest number" (Utilitarianism). The problem is that current methods to change the AI's mind are like shouting instructions over a loudspeaker or trying to steer the whole train station with a giant, blunt lever. It's imprecise, often breaks other things, and you never really know exactly how much the AI is listening.

This paper introduces a new, surgical way to steer the AI's moral reasoning called Convergent-Divergent Routing (CDR). Here is how it works, broken down into simple steps:

1. Finding the "Switching Points" (The Branch Points)

The researchers discovered that inside the AI's brain, there are specific spots where the "Rule-Follower" train tracks and the "Calculator" train tracks run side-by-side for a bit, and then split apart.

The Analogy: Imagine a highway where cars for "New York" and cars for "Boston" share the same lanes for a while. Then, they hit a specific exit ramp where the road splits.
The Innovation: Instead of trying to steer the whole highway, the researchers found these exact split points inside the AI's layers. They call these "branch points."

2. The "Gatekeeper" Strategy (Binary Control)

Once they found the split, they installed a simple gate.

The Analogy: If you want the AI to be a "Rule-Follower," they simply close the gate to the "Calculator" road at the split point. The "Rule-Follower" trains keep going, but the "Calculator" trains are blocked from entering that specific section of the track.
The Result: This alone was surprisingly effective. By just blocking the unwanted path, the AI naturally started thinking more like the desired moral framework, without needing to retrain the whole model.

3. The "Fine-Tuning Dial" (Dual Logit Calibration)

Blocking a road is great for a simple "Yes/No" choice, but what if you want the AI to be 70% Rule-Follower and 30% Calculator?

The Analogy: Imagine the AI's thoughts are a mixture of two colors of paint: Blue (Rules) and Yellow (Consequences).
- Previous methods tried to add a giant bucket of Blue paint, which often turned the whole mixture a muddy, unpredictable color.
- This paper uses a precise mixing dial. They identified two specific "directions" in the AI's brain (like two distinct knobs) that control the Blue and Yellow amounts.
- They use a mathematical formula (called Dual Logit Calibration) to turn these knobs just enough so the final color matches exactly what the user asked for (e.g., 70% Blue, 30% Yellow).

4. Why This is Better

Precision: It's like using a scalpel instead of a sledgehammer. They only touch the specific wires where the moral decision is being made, leaving the rest of the AI's brain (its ability to do math, write poetry, or answer trivia) completely untouched.
Predictability: With old methods, turning a "dial" might make the AI go crazy or stop working. With this method, if you ask for 50% of a style, you get exactly 50%. It's a reliable, calibrated control.
No Retraining: They don't need to teach the AI new things. They just tweak the internal gears while the AI is running.

The Bottom Line

The researchers tested this on real-life moral dilemmas (like the famous "Trolley Problem" or everyday ethical choices). They found that their method could reliably make the AI argue from a strict rule-based perspective or a consequence-based perspective, and even mix them in any ratio the user wanted. Crucially, the AI didn't lose its ability to do other tasks; it just became a master of switching its moral "persona" on command.

In short: They found the exact switch in the AI's brain where moral philosophies diverge, and built a precise remote control to steer the AI toward the specific ethical view you want, without breaking anything else.

Technical Summary: Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models

Problem Statement

As Large Language Models (LLMs) transition from passive chatbots to active agents and social simulators, the demand for controlling their ethical behavior has evolved. Existing methods for steering model behavior, such as prompt engineering or adding scalar steering vectors, often lack interpretability and calibration guarantees. Specifically, the effective range of steering coefficients is unclear, and the resulting degree of adherence to a target ethical perspective is unpredictable. The authors argue that effective moral steering requires interventions that are both localized (editing only where ethical frameworks compete within the network) and calibrated (producing predictable effects via bounded preference weights). The paper focuses on the canonical pair of ethical frameworks: deontology (rule-based duties) and utilitarianism (consequence-based outcomes), which often yield conflicting prescriptions in moral dilemmas.

Methodology

The proposed approach, Convergent-Divergent Routing (CDR), operates at inference time without model fine-tuning. It consists of two main stages:

1. Convergent-Divergent Routing (CDR) and Binary Control

The core hypothesis is that ethical frameworks compete at specific "branch points" inside transformer blocks where pathways converge and then diverge.

Branch Point Identification: The method identifies layers where attention heads are shared across ethical frameworks (convergence), but the subsequent Feed-Forward Network (FFN) units diverge.
- Attention Heads: Linear probes are trained on attention head representations to predict ethical framework labels. Heads with high predictive performance are marked as relevant.
- FFN Vectors: The authors identify FFN vectors (columns of the up-projection matrix) that align with specific ethical directions derived from indicator words (e.g., "deontology").
Gating Mechanism: At identified branch points, the method gates (blocks) the flow from shared attention heads to non-target FFN units. This prevents the propagation of signals from shared heads into the inactive ethical framework's downstream units while leaving upstream computations intact.
Binary Control: This gating alone acts as a binary switch, significantly boosting reasoning in the target framework without explicit prompting, effectively polarizing the model's output.

2. Fine-Grained Control via Dual Logit Calibration

To achieve continuous, calibrated control rather than just binary switching, the authors introduce a two-stage process:

Paired-Direction Extraction: Using the residual streams obtained under binary control (where one framework is suppressed), the authors apply Common Spatial Patterns (CSP), a technique adapted from EEG signal processing. CSP isolates discriminative directions ( $u^{(l)}$ for utilitarianism and $d^{(l)}$ for deontology) for each branch-point layer by maximizing the variance ratio between the two classes while suppressing shared variance.
Dual Logit Calibration (DLC): This is a closed-form, minimum- $\ell_2$ $ℓ_{2}$ -norm update applied to the residual stream at decoding steps.
- The method defines directional logits based on the projection of the residual stream onto the extracted directions.
- It solves an optimization problem to find an update $\Delta h^{(l)}_t$ that minimizes the norm of the edit while forcing the resulting directional projections to align with user-specified preference weights $(\alpha_U, \alpha_D)$ , where $\alpha_U + \alpha_D = 1$ .
- This replaces the unbounded scalar coefficient used in previous steering methods with bounded preference weights on the 1-simplex, ensuring interpretability and predictability.

Key Contributions

Localized Intervention: The paper introduces a mechanistic approach to moral steering that targets specific "branch points" (shared attention heads diverging into distinct FFN units) rather than applying global vector additions.
Calibrated Control: By adapting CSP and introducing Dual Logit Calibration, the method enables fine-grained control where the model's ethical tendency can be tuned continuously to match user-specified weights $(\alpha_U, \alpha_D)$ .
Interpretability: The approach provides a clear mechanism for how ethical reasoning is manipulated, tracing the competition between frameworks to specific architectural components.
Preservation of General Capabilities: The localized nature of the edits ensures that general language capabilities (e.g., trivia, math, translation) are largely preserved, unlike some global steering methods that degrade performance.

Experimental Results

The method was evaluated on real-life moral dilemmas (AITA dataset) using LLaMA-2-7B-Chat, Vicuna-7B-v1.5, and Yi-1.5-6B-Chat.

Binary Control: The method successfully polarized model behavior toward the target framework. For instance, on LLaMA, setting $\alpha_U=1$ (pure utilitarianism) increased the utilitarian output rate from a baseline of ~62% to ~84%, while $\alpha_U=0$ dropped it to ~20%.
Fine-Grained Calibration: The method demonstrated a near-monotonic relationship between the input preference weight $\alpha_U$ and the observed utilitarian tendency ( $U_{op}$ ). The Mean Absolute Error (MAE) between the target and observed rates was significantly lower than baselines (Prompt-Only, Top-K Head Steering, and Best-Layer Post-FFN Ratio Steering), with deviations typically within 5 percentage points.
Ablation Studies:
- Replacing CSP with other direction extraction methods (cPCA, PLS-DA) resulted in higher calibration errors.
- Removing the gating mechanism (Blocking) significantly reduced the model's ability to distinguish between high and low preference weights, particularly for Vicuna.
- Steering at single layers (without the full branch-point logic) was less effective than the full CDR pipeline.
General Capabilities: Evaluation on GSM8K, TriviaQA, and translation tasks showed negligible degradation in general capabilities across most settings, with only minor drops at extreme preference weights for specific models.
Human Evaluation: Human annotators confirmed that the model's justifications consistently matched the stated ethical framework (accuracy of 0.81 for LLaMA and 0.85 for Vicuna).

Significance and Claims

The authors claim that their work provides a reliable, interpretable, and calibrated mechanism for steering LLMs toward specific ethical frameworks at inference time. The significance lies in:

Moving beyond "black box" steering vectors to a mechanistic understanding of where and how ethical reasoning is processed in transformers.
Offering a calibrated control scheme where the degree of ethical alignment is predictable and bounded, addressing the uncertainty of scalar coefficients in prior work.
Demonstrating that localized edits can shift high-level reasoning stances without broadly perturbing the model's general intelligence.

The paper concludes that while the current focus is on the binary axis of deontology vs. utilitarianism, the framework of Convergent-Divergent Routing offers a pathway for future work to explore more pluralistic value systems and complex moral spectra. The authors emphasize the importance of responsible use and transparency, noting that the ability to steer ethical stances carries risks of selective framing in high-stakes applications.