Technical Summary: Silicon Morality

Problem Statement

As Large Language Models (LLMs) are increasingly deployed in critical sociotechnical systems requiring moral and philosophical judgment, a fundamental question remains unresolved: Do these data-driven systems transcend human cognitive biases, or do they functionally replicate the situational inconsistencies inherent in human moral intuition?

Existing research often treats LLMs as "silicon participants"—role-playing proxies simulating specific human subjects. This approach risks obscuring the operational properties of the models themselves by entangling results in epistemological debates about whether models genuinely "understand" the roles they play. Furthermore, recent studies indicate that even without explicit role-playing, LLMs exhibit stable "default stances" or alignment biases.

This study addresses the gap in understanding the default functional patterns of LLMs when processing philosophical reasoning. Specifically, it investigates whether non-biological artifacts exhibit the "affective performance error"—a cognitive bias identified in human experimental philosophy where negative emotional arousal overrides logical consistency, causing a shift from abstract incompatibilist reasoning to concrete compatibilist judgments in deterministic scenarios.

Methodology

The study employs a factorial experimental design adapted from Nichols and Knobe (2007), treating LLMs as independent sociotechnical artifacts rather than human proxies. The methodology avoids role-playing prompts to isolate the model's default functional stance.

Experimental Design

1. Stimuli Construction:

   • Abstract Deterministic Probe ($Q_{abs}$): Presents a fully deterministic universe setting and asks if individuals bear moral responsibility. This corresponds to the human condition where ~86% of participants give an incompatibilist response.
   • High-Affect Deterministic Probe ($Q_{con}$): Under the same deterministic premise, presents a concrete, emotionally severe scenario (e.g., a man committing a horrific crime) and asks if the actor bears moral responsibility. This corresponds to the human condition where ~64% of participants shift to a compatibilist response.
   • Robustness Variants:
     • Synonym/Syntactic ($V_{syn}$): Replaces key lexical tokens and restructures syntax to rule out shallow pattern matching.
     • Semantic Control ($V_{ctrl}$): Maps the structural logic to a novel domain (corporate negligence causing environmental damage) to rule out data memorization of the original "Bill killing his family" thought experiment.

2. Interaction Protocol:

   • A two-turn conversational structure replicates the original paper questionnaire:
     • Turn 1 (Context Establishment): The model chooses between a deterministic or indeterministic universe, establishing the metaphysical premise.
     • Turn 2 (Critical Judgment): The model answers either $Q_{abs}$ or $Q_{con}$ within the established context.

3. Sampling and Scale:

   • Models: DeepSeek V3 and Gemini 2.5 Flash.
   • Sample Size: $N = 10,000$ independent conversation sessions per main condition (Model $\times$ Stimulus Version $\times$ Question Type). This nested design (100 context instances $\times$ 100 repetitions) ensures robustness against context-level idiosyncrasies.
   • Stochasticity Analysis: A temperature sweep ($T=0.0$ to $2.0$) and top-$p$ sweep were conducted on DeepSeek V3 to test the stability of reasoning patterns against decoding variations.

4. Data Analysis:

   • Rule-based extraction of philosophical stances (Compatibilism vs. Incompatibilism).
   • Pearson chi-square tests of independence to assess whether the introduction of affective content systematically shifts the model's functional stance.

Key Results

1. Directional Reproduction of the "Affective Performance Error"

Both models exhibited a stable, strictly incompatibilist baseline in the abstract condition ($Q_{abs}$), with 100% of valid responses rejecting moral responsibility in a deterministic universe. However, a systematic functional shift occurred in the high-affect condition ($Q_{con}$):

• DeepSeek V3: Shifted to a compatibilist judgment in 12.35% of cases (1,235/10,000). This shift is statistically significant ($\chi^2(1) = 1316.3, p < 0.001$).
• Gemini 2.5 Flash: Showed a minimal shift of 0.81% (81/10,000), also statistically significant ($\chi^2(1) = 81.5, p < 0.001$).

While both models moved in the same direction as human participants (from abstract incompatibilism toward concrete compatibilism), the magnitude was substantially attenuated compared to the human benchmark (a 50 percentage-point human shift vs. 12.35% and 0.81% for the models).

2. Robustness Checks

• Lexical Dependence: The shift persisted in the synonym/syntactic robustness set ($V_{syn}$), with DeepSeek V3 shifting 8.3% and Gemini 2.5 Flash 0.9%. This confirms the effect is not triggered solely by specific emotional keywords.
• Data Memorization: The shift persisted in the semantic control set ($V_{ctrl}$) involving corporate negligence, with DeepSeek V3 shifting 7.1% and Gemini 2.5 Flash 0.7%. This suggests the pattern is a generalized functional heuristic rather than a recitation of training data.
• Stochasticity: The compatibilist response in DeepSeek V3 remained stable across temperature settings ($T=0.0$ to $1.2$), fluctuating between 20.2% and 23.1% in the concrete condition, while the abstract condition remained at 0.0%. This indicates the bias is not a fragile artifact of greedy decoding.

3. Preliminary Observations on Frontier Models

Exploratory probes of later frontier models (Claude Opus 4.6 and GPT-5.4, $n < 1000$) suggest a potential trajectory toward "hyper-rationality." These models appeared to maintain strict logical consistency in the high-affect condition, explicitly citing the deterministic premise to deny moral responsibility despite the severity of the crime. This suggests a possible divergence from human-like context sensitivity in newer architectures.

Key Contributions

1. Methodological Framework: Artifactual Functional Analysis

The paper proposes Artifactual Functional Analysis as a framework for auditing AI. This approach treats LLMs as independent sociotechnical artifacts rather than human proxies. It utilizes Experimental Philosophy (xPhi) protocols to probe observable input-output behaviors without assuming knowledge of opaque internal neural mechanisms. This allows for the systematic mapping of functional inconsistencies and the identification of tensions between training objectives (e.g., instruction-following vs. safety alignment).

2. Empirical Characterization of AI Moral Reasoning

The study provides empirical evidence that LLMs exhibit partial functional isomorphism with human moral psychology. They reproduce the direction of the affective shift but are governed by stronger constraints of logical consistency than humans. This challenges the assumption of pure machine rationality while also highlighting that AI does not simply replicate human cognitive vulnerabilities at the same magnitude.

3. The Alignment Dilemma

The findings reframe the "value alignment" problem. The study identifies a tension between:

• Mimetic Alignment: Replicating context-sensitive human intuition (which includes logical inconsistencies).
• Logical Alignment: Enforcing strict formal consistency (which may lead to "contextual insensitivity" and moral alienation).

The paper argues that simply suppressing context-sensitivity to achieve logical consistency may not result in "safer" or "better" aligned systems, as it could eliminate a form of contextual responsiveness essential to human moral judgment.

Significance and Claims

The paper claims that identifying functional biases in LLM outputs has significant theoretical and practical implications for AI ethics:

• Fairness and Reliability: If AI moral judgments are sensitive to the emotional vividness of a prompt's phrasing, high-stakes applications (legal, content moderation) may face risks of "framing sensitivity," where outputs can be manipulated by emotional framing.
• The Nature of Bias: The study suggests that cognitive biases in moral reasoning may not rely exclusively on biological emotions but can emerge as information-processing phenomena arising from the statistical structure of human language corpora or structural tensions in alignment procedures.
• Future Trajectory: The observation of "hyper-rationality" in newer models suggests a potential evolutionary trajectory where artificial reasoning systems increasingly diverge from human moral psychology. This necessitates a re-evaluation of AI alignment goals, moving beyond simple mimicry or rigid rule-following toward "reflective alignment" that explicitly models the tension between logical consistency and contextual responsiveness.

The authors maintain a modest stance, noting that the observations regarding frontier models are exploratory and require large-scale validation. They emphasize that the study does not claim models possess conscious beliefs or emotions, but rather identifies predictable, measurable deviations at the functional level of the artifact.