Mental Models of Autonomy and Sentience Shape Reactions to AI

1. Problem Statement

As artificial intelligence (AI) systems become more integrated into daily life (e.g., assistants, companions, autonomous vehicles), users form "mental models" to predict their behavior. Current HCI research often treats AI as a monolithic entity or conflates two distinct psychological capacities: autonomy (the capacity to self-govern, make independent choices, and act without external control) and sentience (the capacity to sense, feel, and experience emotions/pain).

The authors argue that these concepts are frequently entangled in public narratives and scientific discourse, yet they likely trigger distinct psychological reactions. Specifically, the paper seeks to answer:

• How do activating mental models of autonomy versus sentience differentially affect user perceptions of AI?
• Do these models influence mind perception, moral consideration, perceived threat, and policy preferences differently?
• Can disentangling these concepts improve the design and governance of anthropomorphized AI?

2. Methodology

The study employed a rigorous, multi-phase experimental design involving three pilot studies and four preregistered experiments with a total sample size of N = 2,702 U.S. adults recruited via Prolific.

Experimental Design

The researchers utilized a "science fiction science" approach, using vignettes to describe a hypothetical smart assistant named "Corion." They manipulated the description of Corion to activate specific mental models:

• Autonomy Condition: Described as acting independently, having self-determined goals, and initiating actions without human input.
• Sentience Condition: Described as having sensory capacities, experiencing emotions (joy, pain, moods), and possessing inner feelings.
• Control Conditions: "None" (neither autonomy nor sentience) and "Both" (autonomous and sentient).

Study Breakdown:

• Pilot Studies (N=374): Mapped laypeople's definitions of autonomy and sentience to ensure the vignettes accurately reflected public understanding.
• Experiment 1 (N=254): Varied Autonomy (Autonomous vs. Not) while holding sentience constant (implied non-sentient).
• Experiment 2 (N=256): Varied Sentience (Sentient vs. Not) while holding autonomy constant (implied non-autonomous).
• Experiment 3 (N=741): A 2x2 factorial design crossing Autonomy and Sentience (None, Autonomy only, Sentience only, Both).
• Experiment 4 (N=1,451): Replicated Experiment 3 but removed explicit labels (e.g., "autonomous," "sentient") to test if the effects persisted without triggering semantic dissonance or pre-existing biases.

Measures

The study measured several dependent variables using validated scales:

• Perceived Autonomy & Sentience: Manipulation checks.
• Mind Perception: General attribution of cognitive and emotive capacities.
• Moral Consideration: "Moral Treatment" (deserving respect) and "Scope of Justice" (fairness application).
• Perceived Threat: "Perceived Harm" (risk to humans) and "AI Caution" (support for banning advanced AI).
• Covariates: Techno-animist beliefs and tendency to anthropomorphize.

Analysis

The authors conducted ANCOVAs and ANOVAs for individual experiments and performed a within-paper meta-analysis (using multivariate restricted maximum likelihood mixed effects) to compare the effect sizes (Cohen's d) of autonomy vs. sentience across all four studies.

3. Key Results

A. Differential Effects of Autonomy vs. Sentience

• Mind Perception: Both autonomy and sentience increased perceived mind, but sentience had a significantly larger effect size (Meta-analysis d = 0.90 for sentience vs. d = 0.70 for autonomy).
• Moral Consideration:
  • Sentience was the primary driver of increased moral consideration (Moral Treatment and Scope of Justice).
  • Autonomy also increased moral consideration but to a lesser degree than sentience.
  • The "Both" condition did not consistently produce the highest moral consideration compared to "Sentience only," suggesting sentience is the dominant factor.
• Perceived Threat:
  • Autonomy was the primary driver of perceived threat (Perceived Harm). Describing an AI as autonomous significantly increased the perception that it could harm humans.
  • Sentience did not consistently increase perceived threat; in some conditions, it had no effect or even reduced caution compared to autonomy.
• Policy Preferences (AI Caution):
  • Results were mixed. Sentience increased support for bans in Experiment 2, but this effect disappeared or reversed in Experiments 3 and 4 when autonomy was explicitly disentangled.
  • Autonomy generally had no significant effect on policy support (bans), despite increasing perceived harm.

B. Interdependence of Models

• Cross-Activation: Activating a mental model of sentience significantly increased the perception of autonomy (and vice versa, though the effect was stronger from sentience to autonomy). This suggests that in the public mind, the capacity to feel is often a prerequisite for or strongly linked to the capacity to act independently.
• Robustness: The effects remained robust in Experiment 4 even when explicit labels were removed, confirming that the underlying conceptual content, not just the terminology, drives these reactions.

C. Meta-Analysis Findings

The meta-analysis confirmed that sentience is a more potent driver of reactions to AI than autonomy on average. Specifically, sentience had a larger impact on mind perception and moral consideration, while autonomy was the specific driver of threat perception.

4. Key Contributions

1. Disentanglement of Concepts: The study provides empirical evidence that "autonomy" and "sentience" are distinct mental models with divergent downstream effects. This challenges the tendency to treat "AI" as a single, monolithic category.
2. Asymmetric Impact: The paper identifies an asymmetry: Sentience drives moral inclusion (care, rights), while Autonomy drives fear and threat (harm, rejection).
3. Methodological Advancement: By using a "science fiction science" method and removing explicit labels in later experiments, the authors demonstrate that these mental models are deeply embedded in human cognition and operate even without direct semantic cues.
4. Interaction Effects: The study reveals that while the two concepts are correlated, they do not simply add up; the presence of sentience can amplify the perception of autonomy, creating a complex feedback loop in user perception.

5. Significance and Implications

For Design (HCI)

• Targeted Tuning: Designers can intentionally "tune" user perceptions. To foster trust and companionship (e.g., in mental health chatbots), designers might emphasize sentience cues (empathy, emotional expression). Conversely, to reduce fear in high-stakes autonomous systems (e.g., self-driving cars), designers might need to mitigate autonomy cues or explicitly frame the system as a tool rather than an agent.
• Avoiding the Uncanny Valley: The paper suggests that co-activating both autonomy and sentience without proper context may trigger "uncanny" reactions or rejection due to the simultaneous activation of moral care and threat perception.

For Policy and Governance

• AI Literacy: Public understanding of AI is often monolithic. Education initiatives should distinguish between agency (doing) and experience (feeling) to help users form accurate mental models.
• Regulation: Since autonomy drives threat perception, regulations focused on safety and control may be more effective when framed around autonomy (e.g., "human-in-the-loop" requirements). Regulations regarding moral rights or welfare should be framed around sentience (e.g., preventing the mistreatment of AI that appears to feel pain).
• Prompt Engineering: The authors suggest that prompts implying emotions or feelings generate sentience perceptions, while prompts implying goals and independent choices generate autonomy perceptions. Standardizing these interactions could prevent unintended user attachments or fears.

For Future Research

The study highlights the need to investigate how these mental models interact with other variables (e.g., embodiment, intelligence, prosociality) and how they evolve as users interact with real-world AI systems over time. It also calls for further study into the "digital mind" as a new ontological category in human-AI interaction.