💬 NLP

Agenda-based Narrative Extraction: Steering Pathfinding Algorithms with Large Language Models

This paper introduces agenda-based narrative extraction, a method that integrates large language models into pathfinding algorithms to steer storyline construction toward user-specified perspectives while maintaining high narrative coherence and supporting multiple storylines.

Original authors: Brian Felipe Keith-Norambuena, Carolina Inés Rojas-Córdova, Claudio Juvenal Meneses-Villegas, Elizabeth Johanna Lam-Esquenazi, Angélica María Flores-Bustos, Ignacio Alejandro Molina-Villablanca, Joshu

Published 2026-04-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Brian Felipe Keith-Norambuena, Carolina Inés Rojas-Córdova, Claudio Juvenal Meneses-Villegas, Elizabeth Johanna Lam-Esquenazi, Angélica María Flores-Bustos, Ignacio Alejandro Molina-Villablanca, Joshua Emanuel Leyton-Vallejos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive library of news articles about a single event, like a political protest. You want to read a story that connects these articles into a smooth, logical timeline. But here's the problem: different people want to read different versions of that story. One person wants to focus on the protesters' demands, while another wants to focus on the government's reaction.

Traditional computer tools usually do one of two things:

The "Strict Librarian": It builds the most logically perfect story possible, ignoring what you specifically want to read. It's coherent, but boring and one-sided.
The "Keyword Hunter": It finds articles that contain specific words you typed in. It's flexible, but the story often jumps around and makes no sense because it's just matching words, not understanding the meaning.

This paper introduces a new method called Agenda-Based Narrative Extraction. Think of it as hiring a Smart AI Tour Guide to lead you through that library.

The Core Idea: The Tour Guide with a Compass

Imagine you are at a giant, chaotic museum (the news corpus). You want to walk a path from the entrance (the start of the event) to the exit (the end of the event).

The Old Way (Narrative Trails): The museum has a pre-painted line on the floor that represents the "safest" and most logical path. You must follow it. It's a great walk, but you can't choose to see the dinosaur exhibit if the line goes past the art gallery.
The New Way (Agenda-Based): You tell the AI Tour Guide your "Agenda."
- Agenda A: "I want to see everything about the protesters' demands."
- Agenda B: "I want to see everything about the police response."

The Tour Guide (powered by a Large Language Model, or LLM) looks at the next few rooms (candidate articles) and asks: "Which of these rooms fits the user's request best?"

How It Works (The "Smart" Part)

The AI doesn't just look for keywords. It understands meaning.

The Keyword Trap: If you ask for "Regime Crackdown," a simple keyword search might fail if the articles say "police arrested people" or "violent suppression" but never use the exact phrase "Regime Crackdown." The AI guide, however, understands that "arrests" and "suppression" are the crackdown. It picks those articles even without the exact words.
The Coherence Check: The AI is also a stickler for logic. It won't jump from "Protest starts" to "Election results 10 years later" just because it fits the agenda. It ensures the story flows smoothly, like a good movie, not a jumbled pile of clips.

The Experiment: Testing the Guide

The researchers tested this on 418 news articles about the 2021 Cuban protests. They gave the AI three types of instructions:

Simple: "Find articles about freedom." (Easy, keywords work here).
Semantic: "Find articles about the regime violently suppressing protesters." (Hard, requires understanding context).
Counter (The Lie Detector): "Find articles about how the protests were failing and the government was popular." (Impossible, because the news was actually about the opposite).

The Results:

On Simple Tasks: The AI and the Keyword Hunter were about equal.
On Complex Tasks: The AI Tour Guide crushed the Keyword Hunter. It found the right stories 13% better because it understood the meaning, not just the words.
On The "Lie Detector" Task: When asked to find a story that contradicted reality (e.g., "The protests failed"), the AI said, "I can't do that." It gave a very low score. This proves the AI isn't making things up; it can only highlight perspectives that actually exist in the data.

The Trade-Off: Is the story still good?

You might worry that forcing the AI to follow a specific agenda makes the story messy.

The Verdict: The story is almost as smooth as the "Strict Librarian" version. The AI only sacrificed about 2% of the story's logical flow to gain the ability to follow your specific perspective. That's a tiny price to pay for getting the story you actually want to hear.

Why This Matters

This technology bridges the gap between rigid logic and human curiosity.

It allows analysts, journalists, or regular people to explore the same set of facts from different angles without breaking the story.
It prevents "fake news" generation because the AI is constrained by the actual documents available; it can't invent a narrative that isn't supported by the evidence.

In short: This paper gives us a tool to ask our computers, "Show me the story of this event from my perspective," and get back a coherent, logical, and fact-based answer, rather than just a list of random articles or a single, boring version of the truth.

1. Problem Statement

Existing methods for computational narrative extraction face a fundamental trade-off between coherence, interactivity, and multi-storyline support:

Narrative Maps: Supports rich user interaction and generates multiple storylines via coverage constraints but often sacrifices individual path coherence.
Narrative Trails: Achieves high coherence by finding the "maximum capacity path" (optimizing the weakest link) in a coherence graph but produces a single deterministic output with no mechanism for user guidance or alternative perspectives.
The Gap: There is no existing method that allows users to guide the extraction process toward specific perspectives (agendas) while maintaining the high coherence guarantees of pathfinding algorithms.

The authors aim to bridge this gap by introducing agenda-based narrative extraction, enabling users to steer pathfinding algorithms using natural language descriptions of desired perspectives without sacrificing narrative quality.

2. Methodology

The proposed method integrates Large Language Models (LLMs) into the Narrative Trails pathfinding framework.

A. Core Framework: Coherence Graphs

The system builds upon the Narrative Trails algorithm:

Graph Construction: Documents are nodes; edges represent pairwise coherence.
- Embeddings: Documents are embedded using Sentence-BERT, projected to 2D via UMAP, and clustered using HDBSCAN to derive topic distributions.
- Coherence Function ( $\theta$ ): Combines spatial proximity (angular similarity of 2D projections) and topical similarity (Jensen-Shannon divergence of topic distributions).
- Constraints: Edges are restricted to chronologically ordered pairs, forming a Directed Acyclic Graph (DAG).
Pathfinding: The algorithm seeks the Maximum Capacity Path, maximizing the minimum edge weight (bottleneck) between a source and target to ensure no transition falls below a quality threshold.

B. Agenda-Driven Steering (The Innovation)

The authors modify the greedy selection step of the pathfinding algorithm:

Candidate Pooling: Instead of selecting only the single highest-coherence neighbor, the algorithm identifies the top- $k$ candidates based on coherence.
LLM Ranking: If $k > 1$ $k > 1$ , an LLM (Gemma 3 12B) is queried to rank these candidates based on their alignment with a user-provided agenda (a natural language string describing the desired perspective).
- Prompting: The prompt includes the agenda, current narrative context, and candidate titles.
- Output: The LLM returns a JSON ranking.
Re-prioritization: Candidates are pushed into the priority queue based on their LLM rank rather than coherence alone. This allows the algorithm to explore paths that are slightly less coherent but highly aligned with the agenda, while maintaining the ability to backtrack if a path leads to a dead end.

C. Baselines

The method is compared against:

Maximum Capacity (Baseline): Standard Narrative Trails ( $k=1$ ), optimizing purely for coherence.
Keyword Matching: Uses TF-IDF similarity between the agenda and documents to score candidates, combined with coherence scores.

3. Evaluation Setup

Dataset: 418 news articles regarding the 2021 Cuban protests (Dec 2020–late 2021).
Agendas: Six agendas categorized by complexity:
- Simple (Literal): "Freedom Uprising," "Diaspora Solidarity" (keywords appear verbatim).
- Semantic (Inference): "Regime Crackdown," "Government Censorship" (requires inferring relevance from arrests, beatings, etc., without exact keyword matches).
- Counter (Negative Control): "Protests Failing," "Regime Popular" (contradict the source material).
Metrics: Evaluated by two frontier LLM judges (Claude Opus 4.5 and GPT 5.1) on:
- Coherence: Logical flow, thematic consistency, temporal coherence, completeness (1-10 scale).
- Alignment: Agenda support, persuasiveness, evidence strength, etc. (1-10 scale).
Scope: 64 endpoint pairs across 6 agendas.

4. Key Results

A. Alignment Performance

Semantic Agendas: LLM-driven steering significantly outperforms keyword matching. For the "Regime Crackdown" agenda, LLM steering achieved a 13.3% higher alignment ( $p=0.037$ ).
Simple Agendas: Keyword matching performed comparably or better, as literal keyword overlap is sufficient for these cases.
Overall: LLM steering achieved 9.9% higher alignment than keyword matching on semantic agendas ( $p=0.017$ ).

B. Coherence Trade-off

Minimal Cost: The LLM-driven approach reduced coherence by only 2.2% (6.19 vs. 6.33) compared to the agenda-agnostic baseline.
Independence: There was a weak correlation ( $r \approx 0.10$ ) between coherence and alignment, indicating that narratives can be highly aligned with an agenda without sacrificing coherence.

C. Counter-Agenda Validation

All methods scored uniformly low (2.2–2.5) on counter-agendas (e.g., "Regime Popular").
Significance: This confirms the system cannot fabricate narratives unsupported by the source data. It amplifies existing perspectives in the corpus rather than hallucinating false ones.

D. Sensitivity Analysis

Prompting Strategy: Using Chain-of-Thought (CoT) prompting increased alignment by 25.7% compared to direct prompting, though it increased extraction time 5x.
Hyperparameters: Performance was robust across variations in candidate pool size ( $k$ ) and temperature.

5. Key Contributions

Novel Algorithm: Introduction of Agenda-based Narrative Extraction, which integrates LLMs into maximum capacity pathfinding to steer storylines via natural language.
Bridging the Gap: Successfully combines the high coherence of Narrative Trails with the interactivity and multi-perspective capabilities of Narrative Maps.
Empirical Validation: Demonstrated that LLMs can effectively identify semantically relevant documents for complex agendas where keyword matching fails, with negligible cost to narrative coherence.
Ethical Safeguard: Showed that the method is constrained by the source corpus, failing to generate narratives that contradict the underlying data (preventing "fake news" generation via steering).

6. Significance and Implications

Interactive Analytics: This method enables analysts to explore large document collections from specific viewpoints without manually manipulating graph structures.
Multi-Perspective Exploration: Running the same algorithm with different agendas yields distinct, coherent storylines, facilitating the study of how framing affects narrative construction.
Limitations & Future Work: The study is limited to a single corpus (Cuban protests) and relies on LLM judges. Future work requires human evaluation to validate LLM judge assessments and testing on diverse domains.
Ethical Note: While the tool allows for "steering," the authors emphasize that this makes framing explicit and studiable, which is a prerequisite for detecting manipulation in information systems.

In conclusion, the paper presents a robust framework for user-guided narrative extraction that leverages LLMs to navigate the trade-off between coherence and perspective, offering a powerful tool for intelligence analysis, journalism, and information retrieval.