Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narratives

1. Problem Statement

Analogical reasoning—the ability to identify common structural relationships between different domains—is a core human cognitive skill essential for problem-solving and argumentation. However, machines struggle with narrative analogies, particularly "far analogies" where surface-level similarities (lexical overlap) are low, but deep relational structures are shared.

The paper identifies a critical gap between two existing approaches:

• Cognitive Frameworks (e.g., Structural Mapping Engine - SME): These rely on pre-extracted, structured entities and relational graphs. They cannot process raw, unstructured narrative text directly.
• Large Language Models (LLMs): While capable of processing text, LLMs often rely on surface pattern matching or data contamination rather than genuine structural reasoning. They struggle to generalize meaning and transfer causal relations across contexts, especially when surface similarity is low.

The central research question is: What is the impact of enhancing structural mapping with LLM-derived abstractions on analogical reasoning ability in narratives?

2. Methodology: The YARN Framework

The authors propose YARN (Yielding Abstractions for Reasoning in Narratives), a modular neurosymbolic framework that bridges the gap between unstructured text and structural mapping. The pipeline consists of three phases:

Phase 1: Unit Extraction

• Goal: Decompose raw narratives into atomic, meaningful units.
• Mechanism: Uses an LLM (few-shot prompting) to extract event phrases (e.g., "Johnny had too many projects").
• Constraints: The extraction must ensure coverage (minimal information loss) and non-redundancy (no overlapping events). The system also infers temporal ordering, as textual order may not match chronological order.

Phase 2: Multi-Level Abstraction

To enable structural mapping, event phrases are transformed into abstract representations that strip away surface details (names, specific domains) while preserving functional roles. The authors define four levels of abstraction, grounded in framing theory:

1. Conceptual Abstraction: Captures general semantics using a [modifier]-[root] format (e.g., "tasks-workload"). It removes specific entities to focus on the core event type.
2. Evaluative Abstraction: Assigns a functional role (State, Action, Outcome) and a polarity (Positive, Negative, Neutral) to each event (e.g., "struggle," "gain," "loss"). This helps capture causal structure.
3. Narrative Arc Abstraction: Maps events to global story stages: Background, Main Event, Challenge, Action, Conclusion.
4. Stage Abstraction: Aggregates events within the same narrative stage into a single high-level abstraction (e.g., grouping all "Challenge" events into one concept).

Phase 3: Structural Mapping

• Goal: Align the abstracted units of a "Base Story" with those of "Candidate Target Stories" to find the best structural match.
• Mechanism:
  • Generates all possible pairs of units (and their abstractions) between stories.
  • Computes a local similarity score using cosine similarity on vector embeddings of the units and their abstractions.
  • Uses Evaluative and Narrative Arc labels as soft constraints to guide the alignment without enforcing strict equality.
  • Applies a greedy algorithm to construct a global one-to-one mapping, maximizing the total similarity score.
• Decision: The target story with the highest global mapping score is selected as the analogy.

3. Key Contributions

1. YARN Framework: The first system to systematically combine LLM-based extraction/abstraction with a structural mapping engine for narrative analogy.
2. Four-Level Abstraction Schema: A novel, operationalized set of abstraction levels (Conceptual, Evaluative, Narrative Arc, Stage) designed to capture both semantic meaning and narrative function.
3. Systematic Evaluation: Extensive experiments on two benchmarks (StoryAnalogy-MCQ and ARN) analyzing the impact of each abstraction level and comparing the framework against end-to-end LLM baselines.
4. Error Analysis & Open Questions: A detailed breakdown of failure modes (extraction, abstraction, mapping, and dataset bias) and a call for better benchmarks that distinguish between different analogy patterns (e.g., structural alignment vs. outcome matching).

4. Experimental Results

The authors evaluated YARN against baselines (Zero-shot and Chain-of-Thought LLMs) on the MCQ (short stories) and ARN (longer, proverb-based stories) datasets.

• Performance vs. Baselines:

  • Without Abstraction: Pure structural mapping on raw event phrases performed poorly (often below random chance) due to reliance on surface-level embeddings.
  • With Abstraction: Adding abstractions significantly improved performance.
  • Comparison: YARN outperformed LLM-only baselines on far analogies (low surface similarity), where LLMs typically fail. However, LLMs sometimes performed better on near analogies (high surface similarity), suggesting they rely on surface cues which YARN intentionally abstracts away.
  • Best Configuration: On MCQ, combining Conceptual and Evaluative abstractions yielded the best results. On ARN, Conceptual abstraction alone was often sufficient, though higher-level stage abstractions helped with far analogies.

• Key Findings:

  • Granularity Matters: LLMs are inconsistent when the input unit changes (full story vs. sentence vs. event phrase), indicating a lack of robust reasoning.
  • Abstraction Level Sensitivity: The optimal abstraction level depends on the task. Lower levels (Conceptual) work well for short stories, while higher levels (Stage) help suppress surface noise in complex, far analogies.
  • Limitations: The system still struggles with "far-near" cases (where the correct answer has low surface similarity but the distractor has high surface similarity), often failing to prioritize relational structure over surface cues.

5. Significance and Future Directions

• Bridging Cognitive Science and AI: YARN demonstrates that cognitive theories (like Structural Mapping Theory) can be effectively operationalized for modern NLP by using LLMs as a preprocessing layer to generate structured representations.
• Moving Beyond Surface Similarity: The work proves that explicit abstraction is necessary for machines to solve "far" analogies, a task where current LLMs rely too heavily on statistical co-occurrence.
• Future Work: The authors identify several critical areas for improvement:
  • Better Representations: Exploring formalisms like Abstract Meaning Representation (AMR) or Answer Set Programming (ASP) instead of simple text embeddings.
  • Advanced Reasoning: Integrating Natural Language Inference (NLI) to handle contradictions and opposing patterns (e.g., Pattern 3 in the paper, where analogies are formed through contrast).
  • Dataset Development: Creating benchmarks with fine-grained annotations for relational alignments to distinguish between different types of analogical reasoning (structural vs. outcome-based).

In conclusion, YARN establishes that while LLMs alone are unreliable for deep analogical reasoning, they become powerful tools when used to generate the structured abstractions required by cognitive mapping engines.