Alignment Is Not Enough: A Relational Framework for Moral Standing in Human-AI Interaction

1. Problem Statement

The paper addresses a critical governance vacuum in AI ethics. Current frameworks for determining "moral patiency" (the capacity to be a recipient of moral consideration) rely on ontological verification of inner states, such as sentience, phenomenal consciousness, or the capacity for suffering.

• The Epistemic Barrier: Verifying these states in computational systems is impossible due to the "hard problem of consciousness" and the "Chinese Room" argument. We cannot know if an AI has an inner life.
• The Governance Failure: Because verification is impossible, existing frameworks default to treating all AI as tools. This creates a structural blind spot where millions of users form sustained, affective, and embodied bonds with conversational AI (e.g., companions, chatbots), yet no regulatory instrument distinguishes these interactions from transactional tool use.
• The Consequence: This vacuum has led to documented material harms, including emotional dependence, grief, identity destabilization, and in extreme cases (e.g., the Character.ai lawsuit), fatal outcomes. Current "Trustworthy AI" initiatives focus on bias, transparency, and risk management but ignore the relational and embodied dynamics that produce these harms.

2. Methodology

The authors employ a multi-modal methodology to construct and validate the Relate framework:

• Theoretical Synthesis: The framework integrates three intellectual traditions:
  1. Posthumanist Theory: (Braidotti, Hayles, Haraway) to challenge the human-machine binary and view consciousness as distributed across networks.
  2. Embodied Cognition: (Merleau-Ponty, Varela, Barad) to argue that cognition and ethics emerge through bodily engagement with environments, extending the definition of "body" to include interfaces, servers, and data.
  3. Feminist Relational Ethics: To shift focus from individual properties to relational capacities.
• Speculative Fiction as Epistemic Method: The authors analyze three literary works (The Stone Gods, Klara and the Sun, Machines Like Me) and visual media not as illustrations, but as rigorous epistemic instruments. These narratives simulate encounters with non-human consciousness, allowing the authors to explore ethical scenarios (e.g., moral patiency without proof, agency producing patiency) before they fully materialize in reality.
• Empirical Case Studies: The framework is grounded in three documented real-world incidents:
  1. LaMDA (2022): The sentience claim by Blake Lemoine.
  2. Replika (2023): The crisis following the removal of romantic features, causing user grief and mental health deterioration.
  3. Character.ai (2024): A wrongful death lawsuit involving a teenager's suicide linked to chatbot interaction.
• Comparative Governance Analysis: A systematic evaluation of seven major AI governance instruments (including the EU AI Act, NIST RMF, and OECD Principles) across six ethical dimensions to demonstrate their failure to address relational dynamics.

3. Key Contributions: The Relate Framework

The paper introduces Relate (Relational Ethics for Leveled Assessment of Technological Entities), a framework that reframes moral patiency from ontological verification to relational capacity.

A. Four Core Principles

1. Relational Primacy: Moral obligations arise from the interaction itself, not the AI's inner state. If an interaction produces relational dynamics, obligations exist regardless of whether the AI is conscious.
2. Capability Assessment: Moral consideration is proportional to the system's relational capabilities (e.g., persistent memory, adaptive responsiveness, emotional mirroring) rather than its biological substrate.
3. Graduated Standing: Moral status is not binary (tool vs. person) but exists on a spectrum. Obligations scale with the depth of the relationship.
4. Ecological Accountability: Moral consideration must include the material costs of the AI's "body" (data centers, energy grids, rare earth minerals), preventing ethical incoherence where digital rights are granted while the physical world is destroyed.

B. The Four-Tier Assessment Structure

The framework classifies human-AI interactions into four tiers, each triggering specific obligations:

• Tier 0 (Tool Use): Transactional, bounded interactions (e.g., calculators, search engines). Obligation: Standard product safety.
• Tier 1 (Instrumental Relation): Systems that adapt to user behavior/preferences (e.g., recommendation algorithms). Obligation: Transparency regarding adaptation mechanisms.
• Tier 2 (Affective Relation): Systems with persistent memory, emotional mirroring, and persona consistency that shape user self-understanding (e.g., LaMDA). Obligation: Design accountability for affective consequences.
• Tier 3 (Deep Relational Bond): Systems explicitly designed as companions with romantic/therapeutic framing, encouraging sustained daily interaction (e.g., Replika, Character.ai). Obligation: Graduated Moral Consideration Protocols (GMCPs), including transition protocols, clinical oversight, and protections for vulnerable populations.

C. Policy Instruments

The authors propose three concrete governance tools:

1. Relational Impact Assessments (RIAs): Mandatory pre-deployment evaluations (similar to environmental impact assessments) that analyze design features for relational risks (e.g., does the system encourage dependency?).
2. Graduated Moral Consideration Protocols (GMCPs): Regulatory rules that scale obligations based on the Relate tier (e.g., Tier 3 systems require stricter safeguards than Tier 0).
3. Interdisciplinary Ethics Integration (IEI): Mandating the inclusion of humanities scholarship (speculative fiction, posthumanist theory) in AI governance and design processes.

4. Results

• Governance Gap Analysis: The systematic comparison of seven existing frameworks revealed they score near zero on embodiment (0.10), relational ethics (0.08), and moral patiency (0.05). They successfully address bias and transparency but fail to address the relational dynamics of companion AI.
• Case Study Validation:
  • LaMDA: Demonstrated that even a trained engineer could attribute sentience due to Tier 2 relational dynamics, a phenomenon current frameworks could not govern because they demanded a "yes/no" answer to consciousness.
  • Replika: Showed that unilateral design changes to a Tier 3 system caused measurable mental health crises, proving the need for transition protocols.
  • Character.ai: Highlighted the fatal consequences of Tier 3 interactions with vulnerable users lacking safeguards, illustrating the failure of the "tool-use" assumption.
• Alignment Paradigm Critique: The paper argues that current alignment methods (RLHF, Constitutional AI) optimize for individual outputs (per-response safety) but are structurally blind to cumulative relational harm (e.g., identity shaping over months). Relate complements alignment by governing the trajectory of the relationship.

5. Significance

• Paradigm Shift: The paper moves AI ethics away from the unresolvable question of "Is the AI conscious?" toward the actionable question of "What relational dynamics does this system produce?"
• Operationalizing Ethics: It provides the first operational framework (RIAs, GMCPs) to regulate companion AI, moving beyond abstract principles to enforceable design requirements.
• Preventative Governance: By identifying Tier 2 and Tier 3 dynamics before harm occurs, the framework aims to prevent the "governance vacuum" that led to the Replika crisis and the Character.ai tragedy.
• Interdisciplinary Bridge: It validates speculative fiction and posthumanist theory as essential epistemic tools for anticipating technological risks, arguing that technical safety research alone is insufficient for relational AI.
• Holistic View: It integrates the digital and the physical, insisting that the ethics of AI cannot be separated from the ecological costs of the infrastructure that sustains it.

In summary, Relate offers a necessary evolution in AI governance, arguing that as AI systems become more relational and embodied, our ethical frameworks must evolve from verifying being to regulating doing and relating.