Artificial Intelligence for Sentiment Analysis of Persian Poetry

This study demonstrates that modern large language models, particularly GPT-4o, can effectively analyze the sentiment and metrical structures of Persian poetry by two prominent poets, revealing that Rumi's works generally convey happier sentiments and utilize meters more diversely than Parvin E'tesami's, thereby validating the use of AI for bias-reduced semantic literary analysis.

The Gist

Imagine you have two giant libraries. One is filled with the ancient, mystical poems of Rumi, a 13th-century Persian master who wrote about love, God, and the soul. The other is filled with the works of Parvin E'tesami, a brilliant 20th-century poet who often wrote about social justice, sorrow, and the struggles of life.

For centuries, humans have read these poems to understand their feelings. But what if we could ask a super-smart robot to read them all at once and tell us: "Is this poem happy or sad?" and "Does the rhythm of the poem change how it feels?"

That is exactly what this paper does. The authors used Artificial Intelligence (AI)—specifically advanced "Large Language Models" (like the brains behind modern chatbots)—to act as a digital literary critic.

Here is the story of their experiment, explained simply:

1. The Experiment: Teaching Robots to Feel

The researchers gave the AI a massive task: read thousands of poems from both Rumi and Parvin. They asked the AI to rate every single poem on a scale of 1 to 5:

• 1 = Deeply Sad (like a rainy Tuesday).
• 3 = Neutral (like a cloudy day).
• 5 = Joyful (like a sunny festival).

They used four different types of AI "brains" to do this. Some were generalists (trained on many languages), and some were specialists (trained specifically on modern Persian).

2. The Big Surprise: The "Modern" Robot Failed

The researchers had a hunch that the AI trained specifically on modern Persian (like news articles and social media comments) would be the best at understanding the old, complex poetry.

They were wrong.
It's like hiring a mechanic who only knows how to fix 2024 electric cars to restore a 17th-century wooden carriage. The "modern" AI got confused by the old-fashioned words and metaphors.

The winner was GPT-4o, a massive, general-purpose AI. It didn't know every single ancient word perfectly, but it was smart enough to understand the vibe and the emotion of the poems better than the specialized ones. It was the closest match to what human scholars felt.

3. The Verdict: Rumi is the "Party Poet," Parvin is the "Deep Thinker"

Once the AI analyzed the poems, a clear pattern emerged:

• Rumi's poems generally scored higher on the happiness scale. They felt more energetic, hopeful, and joyful.
• Parvin's poems generally scored lower. They felt more serious, melancholic, and grounded in reality.

The AI confirmed what humans have suspected for years: Rumi's work is often a celebration of the soul, while Parvin's work often reflects the weight of the world.

4. The Rhythm Connection: The "Musical Instrument" Analogy

This is the most fascinating part. Persian poetry isn't just about words; it's about meter (the rhythm, like a drumbeat). The researchers asked: "Does a specific rhythm always make a poem happy or sad?"

• Parvin's Approach: She used her rhythms like a steady drum. She mostly used the same beats to express similar, serious emotions. Her poems were consistent.
• Rumi's Approach: He used rhythms like a jazz musician. He could take the same drumbeat and make it sound like a joyful dance one minute and a sorrowful lament the next.

The AI measured this using a concept called Entropy (which is just a fancy word for "variety").

• Rumi had high entropy: He could use one rhythm to express everything from deep sadness to wild joy.
• Parvin had lower entropy: Her rhythms were more predictable in their emotional tone.

5. Why This Matters

Before this study, analyzing poetry required a human expert to sit down, read, and interpret. It was slow and subjective (two people might disagree).

This paper proves that AI can now do this heavy lifting.

• It can read thousands of poems in seconds.
• It can spot patterns humans might miss (like how Rumi uses rhythm to trick our emotions).
• It reduces human bias.

The Bottom Line

The study is like handing a magnifying glass to a robot and asking it to look at a painting. The robot couldn't explain the philosophy of the painting perfectly, but it could accurately tell us: "This painting is mostly blue and sad," and "That painting is mostly red and energetic."

It shows that while AI isn't a replacement for human poets or scholars, it is a powerful new tool that can help us understand the emotional landscape of literature in ways we never could before. It bridges the gap between ancient art and modern technology, proving that even a robot can appreciate the joy in Rumi's verses.

Technical Summary

Here is a detailed technical summary of the preprint paper "Artificial Intelligence for Sentiment Analysis of Persian Poetry."

1. Problem Statement

The paper addresses the challenge of applying modern Large Language Models (LLMs) to the sentiment analysis of classical Persian poetry. While AI has made strides in analyzing general text and even modern poetry, classical Persian literature presents unique difficulties due to:

• Linguistic Complexity: The presence of archaic vocabulary, complex metaphors, and amphibologies (ambiguities).
• Subjectivity: Sentiment in poetry is inherently subjective, making "Ground Truth" difficult to establish even among human experts.
• Data Scarcity: A lack of pre-trained models specifically fine-tuned on classical Persian poetic structures.
• Research Gap: Prior to this study, no research had utilized modern transformer-based models (BERT and GPT) to analyze the sentiment of classical Persian poets like Rumi and Parvin E'tesami, nor had there been an investigation into the correlation between poetic meters (vazn) and sentiment.

2. Methodology

Dataset

• Sources: Poems were sourced from the Ganjoor online repository.
• Subjects:
  • Jalāl al-Dīn Muḥammad Rūmī (Rumi): Specifically the Divan-i Shams, chosen for its vast diversity of meters (over 50, including 13 invented by Rumi) and archaic language.
  • Parvin E'tesami: A modern Persian poet, chosen to test model performance on 20th-century texts and to compare female poetic expression using classical meters.
• Preprocessing: Verses were concatenated into unified text. Due to token limits, longer poems were split into chunks, analyzed individually, and averaged to generate a global score.

Models Evaluated

Four transformer-based models were tested in a zero-shot inference setting (no domain-specific fine-tuning on the poetry dataset):

1. BERT Multilingual (Uncased): A generic 102-language model.
2. Pars-BERT (Uncased): A BERT model fine-tuned specifically on modern Persian datasets (e.g., Wikipedia, Digikala e-commerce reviews, SnappFood).
3. GPT-4o-mini: A smaller, efficient generative model.
4. GPT-4o: A state-of-the-art multimodal generative model.

Evaluation Framework

• Sentiment Scale: A 1-to-5 ordinal scale (1 = Sad/Negative, 3 = Neutral, 5 = Happy/Positive).
• Human Ground Truth: 100 poems were rated by 4 annotators (2 scholars, 2 general literature experts).
  • Aggregation: Four strategies (Mean, Median, Mode, Dawid-Skene) were compared. The Mean method yielded the highest alignment with human consensus (Average Quadratic Weighted Kappa $\approx$ 0.78) and was selected as the Ground Truth.
  • Inter-annotator Reliability: Krippendorff's Alpha was 0.6, indicating moderate agreement and high task subjectivity.
• Metrics:
  • Nominal Fleiss' Kappa: To measure model repeatability.
  • Quadratic Weighted Kappa (QWK): To measure alignment with the human Ground Truth.
  • Entropy ($H(X)$): To quantify the diversity of sentiments expressed within a specific poetic meter.
  • Standard Deviation & Polarization: To analyze emotional variability and the ratio of polarized (extreme) vs. neutral sentiments.

3. Key Results

Model Performance

• Repeatability: All models showed high consistency (Fleiss' Kappa > 0.93).
• Accuracy vs. Ground Truth:
  • GPT-4o was the top performer with a QWK of 0.60 and absolute accuracy of 33%.
  • GPT-4o-mini achieved a QWK of 0.50.
  • BERT-based models performed poorly (QWK < 0.05 for Pars-BERT, 0.04 for Multilingual).
• Human Baseline: Even human scholars only achieved 47–61% exact accuracy against the aggregated Ground Truth, highlighting the inherent ambiguity of the task.
• Key Insight: While GPT-4o struggled with exact intensity (e.g., predicting 4 instead of 5), it successfully identified directional polarity (Positive vs. Negative). BERT models failed to capture the nuances of classical poetry.

The "Domain Shift" Phenomenon

• Pars-BERT Failure: Contrary to expectations, Pars-BERT (fine-tuned on modern Persian) performed worse than the generic Multilingual BERT. The authors attribute this to a domain shift: the model was over-fitted to modern, colloquial, or commercial text (social media, e-commerce), causing it to fail on the archaic, metaphorical structures of Rumi's poetry.

Sentiment Analysis Findings

• Rumi vs. Parvin: Across all models, Rumi's poems consistently received higher sentiment scores (happier) than Parvin E'tesami's.
• Meter Correlation:
  • Rumi: Utilized specific meters (e.g., R24, R34, C12) to convey predominantly happy sentiments. Over 60% of poems in certain meters were classified as happy.
  • Parvin: Even in meters shared with Rumi, her average sentiment scores were below 3 (neutral/sad), indicating a predominantly somber tone.
• Entropy & Diversity:
  • Rumi exhibited significantly higher entropy in meter usage, meaning he used the same meters to express a wide spectrum of emotions (from 1 to 5). Some meters reached an entropy of ~2.25 (near maximum).
  • Parvin showed lower entropy, using meters more consistently for specific emotional tones.

4. Key Contributions

1. First Application of LLMs to Classical Persian Poetry: Demonstrated the feasibility of using GPT-4o for semantic analysis of Divan-i Shams and Parvin E'tesami's work without human intervention.
2. Validation of Generative vs. Encoder Models: Established that Generative LLMs (GPT-4o) significantly outperform Encoder-based models (BERT) for interpreting complex, metaphorical literature, likely due to better contextual understanding and training on diverse corpora.
3. Warning on Fine-Tuning: Provided empirical evidence that fine-tuning on modern, out-of-distribution (OOD) data can degrade performance on classical/archaic tasks.
4. Computational Humanities Framework: Developed a robust pipeline for mapping poetic meters to sentiment, proving that AI can replicate human scholarly intuition regarding the emotional diversity of Rumi's work.

5. Significance and Conclusion

The study confirms that LLMs can effectively reduce human bias in large-scale literary analysis by providing consistent, automated semantic evaluations.

• Confirmation of Scholarly Intuition: The AI results validated the long-held scholarly view that Rumi's work is more emotionally diverse and energetic (happier) than Parvin E'tesami's, and that Rumi's mastery of meter allows for a broader emotional range.
• Limitations: The models still face a "performance ceiling" due to the lack of pre-training data on classical metaphors and the inherent subjectivity of poetry.
• Future Outlook: The paper advocates for the development of language models specifically pre-trained on classical literary heritage to fully unlock the potential of AI in Digital Humanities, moving beyond structural analysis to deep conceptual understanding.