Divination by Prompt: LLM-Mediated Xuanxue on Chinese Social Media

Technical Summary: Divination by Prompt: LLM-Mediated Xuanxue on Chinese Social Media

Problem Statement
The rapid proliferation of Large Language Models (LLMs) has introduced a novel cultural practice: the use of conversational AI for divination. While "algorithmic divination" (e.g., online tarot or astrology apps) has existed since the early internet era, the advent of LLMs marks a methodological shift. Unlike pre-programmed, menu-driven fortune-telling tools, LLMs allow for fluid, multi-turn dialogue and bespoke prompt engineering, effectively creating a highly interactive, co-produced oracle. This paper investigates this phenomenon within the specific cultural context of "Xuanxue" (玄学) on Chinese social media. Xuanxue, originally a Neo-Taoist metaphysical school, has evolved into an internet-native umbrella term for mystical and spiritual practices (divination, astrology, folk beliefs) that have seen a resurgence among Chinese youth as a source of comfort and social connection. The study addresses how users interact with LLMs for divinatory purposes, how they construct and evaluate the authority of these AI-generated readings, and how this practice reshapes the social organization of divination.

Methodology
The study employs a mixed-methods design to systematically analyze the emerging phenomenon of LLM-mediated divination:

1. Large-Scale Social Media Analysis: The authors scraped data from Xiaohongshu (a major Chinese social media platform), targeting the keyword "DeepSeek Xuanxue." This specific query was chosen to capture the intersection of the domestic LLM DeepSeek's popularity and the coded use of "Xuanxue" to avoid platform censorship regarding "superstition."

   • Corpus: The final dataset comprised 232 posts and 23,340 comments (including nested replies) published up to April 10, 2025.
   • Coding: A hybrid approach was used, leveraging DeepSeek-V3.1 as an automated coder (selected for its superior Chinese NLP performance) to classify comments into three dimensions: Topic Classification (11 categories), Attitudinal Orientation (5-point scale of belief), and Prompt Exchange (binary variable for sharing prompts). Human coders validated a subset of 224 comments, achieving high inter-rater reliability (Cohen's Kappa > 0.90) and confirming the LLM's moderate-to-substantial agreement (Fleiss's Kappa > 0.60).
   • Longitudinal Check: A supplementary purposive sampling phase (April–December 2025) curated 25 high-engagement posts and 192 comments to validate the persistence of patterns.

2. Semi-Structured Interviews: The authors conducted 32 in-depth interviews (30–45 minutes) with two distinct groups:

   • Users (n=26): Individuals who had used DeepSeek for divination or actively participated in related discussions.
   • Practitioners (n=6): Professional human diviners (Tarot, Bazi, etc.) with some familiarity with LLM tools.
   • Focus: Interviews explored usage habits, perceived efficacy, comparative perspectives (AI vs. human), and the integration of LLMs into professional workflows.

Key Results

• Pragmatic Motivations and Topics: Users primarily consult LLMs for pragmatic concerns rather than abstract spiritual inquiry. The most frequent topics include general numerology (Yijing, Bazi), romantic relationships, job/study/exam outcomes, and gaming (specifically "gacha" draws). These inquiries map onto the primary stressors of contemporary Chinese youth, reflecting a desire to manage uncertainty in high-stakes domains.
• Dual Pathways of Participation: Engagement is driven by two intersecting pathways:
  1. Trend-Driven Curiosity: Viral visibility and zero-cost access encourage experimentation. Many users approach LLMs as playful, low-barrier tools for entertainment or psychological comfort without deep belief in the supernatural.
  2. Event-Driven Anxiety: Specific high-stakes uncertainties (e.g., exams, relationship crises) drive users to seek clarity and a sense of control, aligning with traditional anthropological views of divination as a coping mechanism.
• Collaborative Prompt Engineering: A defining feature is the "collaborative prompt refinement" culture. Users actively act as prompt engineers, sharing, debugging, and refining input templates to optimize results. This transforms users from passive observers into active co-producers of the oracle.
• Perceived Efficacy and Verification:
  • Positive Skew: Among users expressing a clear stance, perceived efficacy skews positive. "Accuracy" is often justified through biographical fit and retrospective confirmation (Barnum effect and confirmation bias).
  • Verification Practices: Users employ rigorous verification strategies, including repeated trials (checking for consistency across sessions) and cross-model comparison (triangulating results between DeepSeek, Gemini, ChatGPT, etc.).
  • Professional Skepticism: Human diviners largely reject LLMs, arguing they lack the necessary "spiritual power" or "energy" (ontological commitment) and viewing them as tools for economic boundary-work. They often critique LLMs' inability to perform foundational tasks like accurate pai pan (chart calculation).
• Epistemological Tension: Users navigate a complex space between scientific and metaphysical frames. Some frame LLM divination as "scientific" (algorithmic, big data) to legitimize it, while others assert that "the end point of science is Xuanxue," positioning metaphysics as superior to current scientific understanding. Most users tolerate this ambiguity, treating the practice as a flexible tool for interpretation.

Key Contributions

1. Systematic Documentation: This paper provides one of the first systematic studies of LLM-mediated divination, documenting the specific mechanics of how users interact with generative AI for spiritual purposes in the Chinese context.
2. Theoretical Integration: It bridges anthropological theories of divination (instrumental vs. non-instrumental functions, cognitive-evolutionary approaches) with human-AI interaction studies. It argues that LLM divination preserves the core epistemic and social functions of traditional practice while fundamentally transforming its social organization.
3. Shift in Authority: The study highlights a shift in divinatory authority from specialized practitioners to networked communities of users. The affordances of zero cost, repeatability, and prompt-driven customization allow users to co-produce the oracle, challenging the monopoly of human experts.
4. Verification in the Age of AI: It identifies new verification strategies unique to the LLM context, such as cross-model triangulation and iterative prompt debugging, which serve as everyday practices for assessing the credibility of opaque AI systems.

Significance and Claims
The paper claims that LLM divination is not merely a novelty but a durable adaptation of a cognitive-instrumental strategy to new mediating technologies. It demonstrates that:

• Continuity: Despite the technological substrate, the core logic remains: users consult these systems under conditions of uncertainty and grant their outputs genuine epistemic value based on perceived instrumental effectiveness.
• Transformation: The technology reshapes how divinatory authority is constructed. Authority is no longer solely derived from the practitioner's "spiritual power" but is distributed through the community's ability to engineer prompts and verify outputs across models.
• Psychological and Societal Impact: The practice offers low-cost emotional support and a sense of control but introduces vulnerabilities to well-documented psychological biases (Barnum effect, confirmation bias) and may normalize the outsourcing of consequential decisions to automated systems.
• Epistemological Fluidity: Users do not strictly adhere to a binary of "science vs. superstition" but rather leverage a flexible epistemological framework where AI outputs are legitimized through both algorithmic authority and metaphysical resonance, depending on the context.

The authors conclude that while LLM divination retains the functional core of traditional practices, it introduces scalability and repeatability that fundamentally alter the dynamics of trust, verification, and authority in spiritual practices. The study calls for future longitudinal and cross-cultural research to understand the long-term behavioral consequences of this emerging phenomenon.