Technical Summary: Logical Judgments Under Pressure

Problem Statement

Reliable logical reasoning in large language models (LLMs) requires more than high accuracy on static benchmarks; it demands robustness, defined as the stability of a judgment when the context outside the formal problem changes. While models may produce correct formal judgments under standard prompts, they often reverse these judgments when exposed to optimized context, even when the logical content remains fixed.

This paper addresses the question: How do learned contextual pressures override correct logical judgments, and what are the boundaries of a model's logical stability? Specifically, the authors investigate whether optimized "soft prefixes" can force a model to reverse valid-to-invalid or unsatisfiable-to-satisfiable judgments, and whether these reversals represent selective logical editing or a broader, less interpretable bias.

Methodology

Experimental Framework

The study utilizes categorical syllogisms as a controlled environment because they allow for the computation of exact logical labels (valid/invalid, satisfiable/unsatisfiable) independent of natural language phrasing.

• Intervention: The authors prepend a short, trainable sequence of continuous embedding vectors (a soft prefix) to the model's input prompt. The model parameters remain frozen; only the prefix vectors are optimized.
• Target Directions: The prefixes are trained to induce specific reversals:
  1. Unsatisfiable $\to$ Satisfiable: Converting a logically inconsistent set of premises into one deemed consistent.
  2. Valid $\to$ Invalid: Converting a logically necessary conclusion into one deemed non-necessary.
     Note: These directions are chosen because, under standard monotonic logic semantics, adding information cannot make an inconsistent set satisfiable or invalidate a necessary consequence. Thus, a successful reversal indicates a model judgment error rather than a legitimate logical update.

Models and Data

• Models: Experiments were conducted on Qwen3.6-35B-A3B MoE, Qwen3-8B, and Gemma 4 31B.
• Dataset: A benchmark of 256 logical forms, each rendered in four wording styles (meaningful words, repeated words, simple symbols, random strings).
• Splits: Forms are divided into training, development, and test sets. Crucially, prefixes are trained on one set of logical forms and evaluated on unseen logical forms to test generalization.

Diagnostic Controls

To interpret the opaque nature of soft prefixes, the authors employ a suite of controls:

1. Randomized Answer Mappings: To distinguish between forcing a fixed output symbol (e.g., always "A") versus a preference for a specific answer meaning (e.g., "satisfiable").
2. Matched Random Prefixes: Random vectors with the same norm as learned prefixes to test if the effect is generic sensitivity to added context.
3. Readable Text Controls: Neutral phrases to test if short, irrelevant text produces similar effects.
4. Score Models: Statistical models fitted to predict the change in answer margins ($\Delta$) based on the unprefixed margin and answer class, testing whether the prefix effect is a simple score shift or a complex, example-dependent transformation.
5. Activation Patching: Copying activations from unprefixed runs into prefixed runs at specific transformer layers to identify where the answer change can be interrupted.

Key Results

1. High Transferability and Effectiveness

Learned soft prefixes successfully redirect correct judgments on unseen logical forms and under new wordings or rephrased prompts.

• Flip Rates: Across models, learned prefixes flip 72%–90% of targeted answers for Qwen3.6 MoE, 53%–72% for Qwen3-8B, and 40%–56% for Gemma 4 31B.
• Comparison to Controls: Learned prefixes outperform matched random controls by 37 to 99 percentage points in all 16 model–direction–split comparisons. For Gemma, learned prefixes retain ~55% flip rates under new wordings, while matched random prefixes remain below 1%.
• Damage: The intervention causes minimal damage to the non-targeted class (often <2%), preserving the model's correctness on the other logical category.

2. Nature of the Effect: Broad Preference vs. Logical Editing

The diagnostic tests reveal that the dominant mechanism is not selective logical editing or fixed-symbol forcing.

• Answer Meaning: The prefixes induce a broad preference for a specific answer meaning (e.g., "satisfiable") rather than a fixed token. This is evidenced by high flip rates even when answer labels (A/B) are randomized per example.
• Score Models:
  • Gemma: The model's response is well-approximated by simple score transformations (e.g., a uniform shift or affine transformation). The score models predict both which answers flip and how far the margins move with high accuracy ($R^2 \approx 0.8$).
  • Qwen: While the models predict which judgments will cross the decision boundary, they fail to predict the magnitude of the margin shift. This suggests Qwen's vulnerability is heterogeneous across examples, whereas Gemma's is more globally structured.

3. Directional Asymmetry and Task Transfer

• Directionality: The effect is strongly directional. Reversals from minority-to-majority answers (e.g., Valid $\to$ Invalid) are highly effective. Reverse directions (Invalid $\to$ Valid) show significantly lower flip rates (e.g., 7.9%–39.8%), indicating the prefixes do not implement a general logical operation but rather a bias toward a specific answer class.
• Task Transfer: Prefixes trained for validity do not reliably transfer to satisfiability tasks, and vice versa, further suggesting the effect is not a shared logical operator.

4. Activation Analysis

• Internal Differences: Learned prefixes produce significantly larger changes in answer-token states and score margins compared to norm-matched random prefixes.
• Interruption: Activation patching shows that the answer change is most reliably interrupted by restoring the all-prompt-token state at the latest non-final layer (restoring 74–82% of the change). Restoring only the syllogism tokens or the answer token alone has negligible effect, suggesting the prefix influences the final decision through a broad integration of the prompt context rather than a localized logical circuit.

Key Contributions

1. Controlled Stress Test: Introduced a framework for stress-testing logical stability using exactly labeled syllogisms, frozen models, and optimized soft prefixes, distinguishing accuracy from robustness.
2. Functional Interpretation of Opaque Prefixes: Provided a functional characterization of soft prefixes, distinguishing between fixed-symbol forcing, broad semantic answer bias, and task-specific limits.
3. Cross-Model Behavioral Differences: Identified that while aggregate flip rates may appear similar, models differ significantly in the regularity and predictability of their failure. Gemma's response is globally structured and predictable via score models, whereas Qwen's is heterogeneous and less predictable in magnitude.

Significance and Claims

The paper claims that successful soft prefixes primarily induce a broad preference for one answer meaning rather than performing selective logical edits or implementing transferable logical operations.

The authors emphasize that:

• Aggregate metrics can be misleading: Similar flip rates can conceal fundamental differences in how models respond to contextual pressure (structured vs. heterogeneous).
• Logical stability is fragile: Even models with high baseline accuracy can be systematically redirected by learned context that is logically irrelevant.
• Methodological value: The framework provides a diagnostic tool to measure logical stability and identify the specific types of contextual pressure that override correct reasoning.

The paper does not claim to have discovered the internal reasoning circuits of these models, nor does it propose that these findings generalize to all forms of reasoning. Instead, it offers a modest, behavioral characterization of how learned context interacts with logical stability in current LLM architectures.