🧬 biology

Mystical and affective features of naturalistic psychedelic experiences: A comparison of dictionary-based and LLM-based text analysis methods

This study compares dictionary-based and large language model (LLM) text analysis methods on 7,303 naturalistic psychedelic reports, finding that while dictionary counts replicate prior U-shaped patterns of mystical vocabulary, LLMs provide a more monotonic and contextually nuanced assessment of mystical experience intensity across key dimensions.

Original authors: Aleksander Milczanowski

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aleksander Milczanowski

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Human beings have long sought to understand the moments when the mind steps outside its usual boundaries. In the study of consciousness, these moments are often called altered states, and when they involve a profound sense of connection, timelessness, or sacred meaning, they are described as mystical experiences. For decades, scientists have tried to measure these fleeting, internal events, often relying on questionnaires filled out in clinical labs. But for many people, these experiences happen outside the hospital, in the quiet of a home or the wild of nature, and they are shared not in data sheets but in stories. The internet has become a vast library of these personal narratives, where thousands of individuals describe their journeys with psychedelic substances in their own words. The challenge for researchers has always been how to read these thousands of stories without losing the nuance of what the writer actually felt. Traditional tools for reading text often work like a simple word counter, looking for specific keywords to tally up a score. But human language is complex; a word like "fear" might signal a terrifying nightmare in one story, or a necessary step toward a profound breakthrough in another. The context matters, and a simple count cannot always tell the difference.

A recent study set out to test a new way of reading these stories, comparing the old method of counting words against a modern approach using artificial intelligence. The researchers gathered 7,303 written accounts of psychedelic experiences from an online forum dedicated to discussions about psilocybin mushrooms. These reports were anonymous, written by people who had already taken the substance, and each writer had assigned their own experience a level of intensity, ranging from mild to very strong. The team first applied the traditional method, using a custom list of words associated with mystical themes—terms like "divine," "unity," or "revelation"—to see how often they appeared in each story. They also measured the emotional tone of the writing, checking whether the stories felt generally positive or negative. This approach confirmed what a previous study had found: the frequency of these mystical words did not simply rise as the intensity of the experience grew. Instead, the words appeared most often in the mildest reports and the most intense ones, forming a U-shaped pattern, while the positive emotional tone of the writing actually dropped as the intensity of the experience increased.

The researchers then turned to a large language model, a type of artificial intelligence capable of understanding the broader meaning and context of a sentence, rather than just spotting isolated words. They asked the AI to read each story and rate it on four specific dimensions of mystical experience: a sense of oneness with everything, the feeling of gaining deep insight or truth, the experience of timelessness, and a sense of sacredness. The AI also rated the overall emotional tone. The results were strikingly different from the word-count method. Instead of the U-shaped curve, the AI's ratings for all four mystical dimensions rose steadily and consistently as the reported intensity of the experience increased. The most intense stories received the highest scores for unity, insight, timelessness, and sacredness. This suggests that while the simple word counter might be good at spotting the presence of mystical vocabulary, it struggles to measure how strong that experience actually is. The AI, by understanding the full context of the narrative, seemed to capture a more direct relationship between the intensity of the trip and the depth of the mystical feeling.

The study also looked at how the AI handled the emotional side of these experiences. The AI's assessment of the emotional tone matched well with the results from the traditional word-counting tools, showing that the writing became less positive and more mixed as the intensity of the experience grew. This aligns with what is known from clinical research: higher doses of psychedelics can lead to more profound effects, but they also carry a higher risk of difficult or challenging moments. The AI's ability to detect this shift in tone, while simultaneously recognizing the increase in mystical depth, suggests it can hold two complex truths at once. The researchers checked the reliability of their AI method by running the same stories through eight different large language models. The models agreed with each other to a high degree, suggesting that the results were stable and not just a fluke of one specific program.

Ultimately, the study does not claim that the artificial intelligence is perfect or that it has solved the mystery of human consciousness. The researchers are careful to note that the AI's scores still need to be compared against ratings from human experts to be fully validated. However, the findings offer a powerful new tool for understanding the psychedelic experience. They show that the way we analyze text can change what we see. A simple count of words might miss the forest for the trees, failing to see that the most intense experiences are also the most mystical. By using a method that reads for meaning rather than just counting, researchers can now look at thousands of natural stories and see a clearer picture of how the human mind transforms under the influence of these substances. This approach opens the door to studying the subjective world of altered states on a scale that was previously impossible, using the very words people use to describe their own journeys.

Technical Summary: Mystical and Affective Features of Naturalistic Psychedelic Experiences

Problem Statement
First-person narrative accounts of altered states of consciousness shared online provide a naturalistic complement to clinical psychedelic research. However, traditional text-analysis methods relying on static word dictionaries face significant limitations when applied to complex phenomenological reports. These lexical approaches are often blind to negation, fail to capture modifying terms (e.g., "sort of," "not"), and struggle to distinguish between terrifying and meaningful experiences that share similar vocabulary (e.g., themes of death and rebirth). Consequently, keyword frequency may not accurately reflect the intensity or contextual meaning of mystical experiences. While Large Language Models (LLMs) offer the potential to evaluate text within its broader semantic context, their application to psychedelic phenomenology remains under-explored. This study addresses the gap between established dictionary-based methods and emerging LLM-based contextual analysis in quantifying mystical and affective dimensions of psychedelic experiences.

Methodology
The study analyzed a corpus of 7,303 English-language psychedelic experience reports collected from the "Trip Reports" section of Shroomery.org. The data collection followed the workflow of a prior study by Žuljević et al. [24], filtering for reports with at least 50 verified English words and removing duplicates.

The analysis employed a dual-method approach:

Dictionary-Based Replication: A custom 77-term dictionary targeting mystical language was applied to calculate a "mystical language score" (percentage of matching terms relative to total word count). Emotional valence was assessed using VADER, a rule-based sentiment analysis tool, and a fine-tuned ModernBERT model.
LLM-Based Feature Extraction: A generative LLM (GLM-5) was prompted via a structured JSON schema to rate each report on four specific dimensions of mystical experience—Unity, Noetic Insight, Time Transcendence, and Sacredness—as well as an overall Sentiment score. Ratings were assigned on a 0–10 scale (0 indicating absence, 1–10 indicating intensity) based on textual evidence. The model was instructed to provide a rationale for its ratings.
Reliability and Statistical Analysis: Inter-model reliability was assessed across eight different LLMs on a 500-report subsample using Krippendorff's alpha. Statistical associations with self-reported experience intensity (Levels 1–5) were tested using Chi-square tests, Kruskal-Wallis tests with Dunn's post-hoc comparisons, and Spearman correlations, with Bonferroni corrections for multiple comparisons.

Key Results

Replication of Prior Findings: The dictionary-based analysis successfully replicated the U-shaped distribution of mystical language scores across self-reported experience intensity levels, where mystical vocabulary frequency was higher at low (Level 1) and high (Level 5) intensities compared to intermediate levels. Positive sentiment (VADER) declined as intensity increased.
Divergence in LLM Ratings: In contrast to the U-shaped lexical pattern, LLM-derived ratings for Unity, Noetic Insight, Time Transcendence, and Sacredness showed a monotonic increase with experience intensity. Level 5 consistently yielded the highest scores, differing significantly from all lower levels.
Convergence and Correlation:
- Dictionary-based mystical language scores correlated moderately with all four LLM-derived dimensions ( $\rho$ = 0.35–0.49).
- LLM-derived sentiment scores converged with both VADER and ModernBERT sentiment measures ( $\rho$ = 0.505 and 0.651, respectively).
- Among the mystical dimensions, Time Transcendence showed the strongest association with self-reported intensity ( $\rho$ = 0.313) and the steepest monotonic increase, yet it had the weakest correlation with sentiment measures ( $\rho$ = 0.055 with LLM sentiment).
Reliability: Inter-model reliability across the eight LLMs was satisfactory to tentative, with Krippendorff's alpha ranging from 0.766 (Unity) to 0.871 (Insight). Pairwise correlations between models were consistently high ( $\rho$ = 0.83–0.90).

Significance and Claims
The paper claims that the divergence between lexical frequency and LLM-derived contextual ratings suggests these methods capture partially distinct aspects of mystical experience. The author posits that the U-shaped distribution observed in dictionary-based analysis likely reflects a limitation of the metric—specifically, that normalized word frequency is less suited to estimating the intensity of a construct than a contextual rating system. Conversely, the monotonic increase in LLM scores aligns more closely with theoretical expectations of dose-dependent mystical experiences, where higher doses (proxied by intensity levels) yield more profound effects.

The study concludes that LLM-based contextual scoring serves as a potentially useful exploratory complement to established NLP methods. While dictionary approaches remain efficient for detecting the presence of mystical vocabulary, LLMs offer a mechanism to evaluate specific phenomenological dimensions within the broader narrative context. The author emphasizes that while the procedural robustness of the LLM approach is supported by high inter-model agreement and convergence with sentiment measures, the correspondence of these scores to human judgments of phenomenological constructs requires further validation against human coders. The work establishes a documented corpus of naturalistic reports as a resource for future mixed-methods research into psychedelic phenomenology.

Technical Summary: Mystical and Affective Features of Naturalistic Psychedelic Experiences

More like this