Constrained Semantic Decompression in LLMs through Persian Proverb-Conditioned Story Generation

Technical Summary: Constrained Semantic Decompression in LLMs through Persian Proverb-Conditioned Story Generation

Problem Definition
The paper formalizes the challenge of transforming dense, abstract cultural knowledge into engaging narratives as a constrained semantic decompression task. Specifically, it investigates the ability of Large Language Models (LLMs) to generate stories conditioned on Persian proverbs. Unlike open-ended creative writing, this task requires models to faithfully instantiate the underlying moral logic and causal structure encoded in a proverb while adhering to constraints such as narrative coherence, length, and age-appropriateness for children. The authors posit that this serves as a rigorous testbed for evaluating "abstraction-to-realization" capabilities, probing whether models can bridge the gap between abstract cultural wisdom and concrete narrative form.

Methodology
To study this problem, the authors introduce the Proverb Aligned Narrative Dataset (PAND), a curated corpus of 150 instances. Each instance pairs a Persian proverb with its explicit moral meaning and a human-written short story for children. The dataset was constructed by crawling Persian literary resources and manually verifying that all content predates the LLM era to prevent synthetic contamination.

The study evaluates a range of proprietary (GPT-4.1) and open-weight models (Gemma 3 12B/27B, Mistral Small 3.2 24B) across three distinct prompting regimes:

1. Pure: Models generate stories using only the proverb and word limits, testing intrinsic reasoning via strategies like Zero-Shot, Persona, Moral Chain-of-Thought (CoT), and Outline CoT.
2. Surface-Assisted: Prompts are augmented with surface-level cues from human reference stories (e.g., prefix conditioning or keyword inclusion) to test reliance on lexical signals versus semantic grounding.
3. Feedback-Guided: An iterative refinement loop involving three personas: a Writer (generates a draft), a Critic (scores and provides feedback based on specific criteria), and an Editor (revises the story). This regime tests whether explicit reasoning and iterative critique can mitigate decomposition failures.

Evaluation employs a hybrid framework combining:

• Human-Calibrated LLM-as-a-Judge: A calibrated Gemini 2.5 Pro model assesses five subjective metrics: Relatedness, Creativity, Fluency, Suitability for Children, and Overall quality.
• Structural Metrics: Objective measures including Lexical and Semantic Diversity, Novelty, Surprise, and Readability (Flesch-Dayani) to quantify narrative structure and deviation from human baselines.

Key Results
The experiments reveal a persistent decompression gap: current LLMs frequently achieve high surface-level fluency and child-suitability scores but fail to faithfully instantiate the deeper moral and causal structures of the proverbs.

• Fluency vs. Grounding: Models often generate grammatically correct and engaging text that lacks the intended moral alignment. Smaller open-weight models exhibit particularly low "Relatedness" scores compared to human benchmarks.
• Prompting Impact: Strategies that explicitly separate semantic interpretation from narrative realization, particularly Moral CoT, consistently yield higher Relatedness and Overall scores. Conversely, surface-assisted strategies (e.g., Prefix-Conditioned) often constrain semantic diversity, leading to lower novelty and surprise.
• Iterative Refinement: The Feedback-Guided regime demonstrates that decomposition errors are not necessarily due to a lack of knowledge but rather difficulties in translating abstract meaning into narrative form. Iterative refinement, especially when using a stronger model (Gemma 3 27B) as a critic for a weaker writer (Gemma 3 12B), significantly improves semantic grounding and creative quality, allowing smaller models to approach the performance of single-pass proprietary baselines.
• Human Validation: A blinded pairwise preference study confirmed that human annotators preferred the refined stories over initial drafts in 65% of cases, validating the LLM judge's assessment of improvement.

Significance and Contributions
The paper makes four primary contributions:

1. Task Formalization: It defines proverb-conditioned story generation as a constrained semantic decompression task, offering a new framework for evaluating abstraction-to-realization in LLMs.
2. Dataset Creation: It introduces PAND, the first Persian dataset pairing proverbs with explicit meanings and human-written stories for evaluating culturally grounded narrative generation.
3. Empirical Findings: It identifies a systematic decoupling between surface fluency and semantic grounding, highlighting a fundamental limitation in current models' ability to preserve moral logic during generation.
4. Mitigation Strategies: It demonstrates that explicit reasoning (CoT) and iterative refinement can partially repair these failures, suggesting that the "decompression gap" is a solvable challenge through better prompting and refinement policies rather than an insurmountable lack of cultural knowledge.

The authors conclude that while state-of-the-art models show strong linguistic competence, their ability to transform compressed cultural knowledge into grounded narratives remains a significant challenge. They suggest that future work could extend this framework to other forms of compressed knowledge (e.g., fables, legal maxims) and explore internalizing the writer-critic dynamic through training rather than prompting alone.