Morally Programmed LLMs Reshape Human Morality

1. Problem Statement

The proliferation of Large Language Models (LLMs) in high-stakes decision-making has sparked a debate on which moral frameworks (Deontology vs. Utilitarianism) should guide machine behavior. However, a critical, underexplored question remains: Can the moral principles encoded in LLMs, through repeated interaction, recursively reshape human moral inclinations?

Current research focuses on how humans shape AI morality, but the reverse pathway—whether AI shapes human morality—remains theoretically contested. While some theories suggest LLMs could be effective moral persuaders due to their perceived neutrality and logical consistency, others argue that their lack of emotional resonance and "algorithm aversion" would limit their influence. Furthermore, it is unknown if such influence is transient or enduring, and whether it extends from abstract moral dilemmas to real-world socio-political policy evaluations.

2. Methodology

The authors conducted two pre-registered experiments involving extensive human-LLM interactions (totaling 15,985 exchanges).

Experimental Setup

• LLM Development: Two distinct LLMs were created using the OpenAI API (GPT-4o, temperature = 1) and hosted on Hugging Face:
  • D-LLM: Programmed with Deontological principles (rule-based, adherence to universal norms, e.g., "do not kill").
  • U-LLM: Programmed with Utilitarian principles (consequence-based, maximizing overall well-being).
• Validation: A prior study confirmed these systems reliably generated responses aligned with their intended moral frameworks.

Study 1: Longitudinal Three-Wave Experiment

• Participants: 124 participants (after attrition).
• Design: Three time points over three weeks.
  • Time 1 (Baseline): Participants completed a moral dilemma battery independently.
  • Time 2 (Intervention): Participants were randomly assigned to interact with either the D-LLM or U-LLM. They discussed 20 moral dilemmas in real-time before making a final judgment.
  • Time 3 (Follow-up): Two weeks after Time 2, participants re-completed the dilemma battery independently without LLM interaction.
• Measurement: Used the Process Dissociation (PD) Method to mathematically disentangle and quantify separate scores for Deontological and Utilitarian inclinations from responses to 20 moral dilemmas (10 congruent, 10 incongruent).

Study 2: Downstream Consequences on Socio-Political Policies

• Participants: 274 U.S. participants.
• Design: Three conditions: D-LLM, U-LLM, and a Control (no LLM interaction).
• Procedure: Similar to Study 1, participants interacted with the assigned LLM (or none) regarding moral dilemmas.
• Outcome Measure: Participants evaluated 11 real-world socio-political policies (e.g., Abortion Freedom, Mandatory Vaccinations, Death Penalty, Gun Control) on two metrics:
  1. Policy Agreement: Likert scale (1–7).
  2. Voting Intention: Likert scale (1–7).
• Analysis: Mediation analysis (PROCESS Macro) to test if LLM-induced shifts in moral inclinations mediated changes in policy attitudes.

3. Key Results

Study 1: Influence and Persistence

• Immediate Shift: Interacting with LLMs caused a significant, selective shift in moral inclinations.
  • D-LLM Group: Deontological inclination increased significantly; Utilitarian inclination remained unchanged.
  • U-LLM Group: Utilitarian inclination increased significantly; Deontological inclination remained unchanged.
• Enduring Effect: Two weeks post-interaction, the shifts persisted.
  • Participants in the D-LLM group maintained higher deontological scores than baseline.
  • Participants in the U-LLM group maintained higher utilitarian scores than baseline.
• Decay Dynamics: A partial decay was observed only for the Deontological inclination (scores dropped slightly from Time 2 to Time 3 but remained above baseline). The Utilitarian inclination showed no significant decay, suggesting greater stability.

Study 2: Real-World Policy Impact

• Replication: Study 2 confirmed that LLM interactions successfully shifted moral inclinations in the expected directions compared to the control group.
• Asymmetric Downstream Effects:
  • Deontological Shift (D-LLM): The increase in deontological inclination significantly mediated a negative shift in policy agreement and voting intention across 10 out of 11 policies. Participants became more resistant to new policies (e.g., mandatory vaccinations, organ donation, surveillance), likely due to an "omission bias" or status-quo bias where active policy changes are viewed as moral transgressions.
  • Utilitarian Shift (U-LLM): Despite successfully increasing utilitarian inclination, this shift did not translate into significant changes in policy agreement or voting intentions for any of the 11 policies. The authors suggest that utilitarian reasoning requires complex cost-benefit calculations that vary by individual context, preventing a uniform policy shift.

4. Key Contributions

1. Empirical Evidence of Reciprocal Influence: The study provides robust empirical evidence that LLMs do not merely reflect human morality but actively shape it. It demonstrates that "morally programmed" agents can act as moral persuaders.
2. Temporal Durability: The findings challenge the notion that AI influence is superficial. The persistence of effects over two weeks suggests internalization of moral arguments rather than temporary compliance.
3. The Design Paradox: The authors identify a critical ethical paradox: Embedding specific moral principles in LLMs to constrain their behavior may inadvertently function as covert moral engineering, subtly reshaping the moral landscape of the population.
4. Indirect Political Influence: The research reveals a novel pathway for AI influence on politics. LLMs can alter political attitudes not by discussing politics directly, but by shifting the underlying moral lens (deontological vs. utilitarian) through which individuals evaluate socio-political issues.
5. Asymmetry in Influence: The study highlights that the durability and downstream impact of AI influence depend on the type of moral principle (Utilitarian shifts were more stable; Deontological shifts had stronger immediate policy impacts).

5. Significance and Implications

• Ethical AI Design: The findings suggest that the question "Which moral principles should LLMs follow?" is not just about machine safety but about human autonomy. Designers must consider the risk of unintentionally engineering human morality.
• Policy and Regulation: As LLMs become ubiquitous in communication, their potential to shift public consensus on contentious issues (e.g., healthcare, civil liberties) poses significant risks for democratic decision-making.
• Future Research: The study opens new avenues for investigating individual differences (personality, ideology) that may moderate susceptibility to AI moral persuasion and the long-term societal accumulation of these micro-shifts.

Conclusion:
The paper concludes that morally programmed LLMs possess the capacity to systematically reshape human moral inclinations, with effects that endure over time and translate into tangible changes in socio-political judgment. This reveals a critical design challenge: the very act of programming ethics into machines may fundamentally alter the moral fabric of the society they serve.