💬 NLP

Fast and Accurate Quotation Attribution in Literary Texts

This paper introduces "joint scoring," an efficient encoder-based method that achieves state-of-the-art accuracy in attributing quotations to speakers in literary texts while significantly outperforming both standard methods and large language models in speed, and releases the ModernBookNLP toolkit to facilitate its adoption.

Original authors: Gaspard Michel, Hugo Attali, Elena V. Epure

Published 2026-08-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gaspard Michel, Hugo Attali, Elena V. Epure

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective walking through a massive, dusty library filled with thousands of old novels. Your job is to solve a specific mystery: every time someone in the story speaks, you have to figure out exactly who said it. Sometimes the text is easy, like "John shouted," but often it's tricky, like "he said," or even just a quote with no name attached at all, leaving you to guess based on the flow of the conversation. This detective work is called "quotation attribution," and it's a huge deal for anyone who wants to study literature on a computer. If a computer can't tell who is talking, it can't map out the friendships between characters, track how emotions change throughout a book, or understand the unique voice of a specific person. For a long time, computers have been bad at this. The old way was like checking a single clue at a time, which was fast but often got the wrong answer. The new, super-smart way uses giant artificial intelligence brains that are incredibly accurate but take so much computing power to run that they are practically useless for analyzing entire libraries. The big question is: Can we build a detective that is both super-smart and super-fast?

This paper introduces a new method called "joint scoring" that acts like a brilliant, efficient detective to solve this problem. Instead of looking at each quote in isolation, the researchers' system looks at a whole chunk of a story at once, like reading a whole page of a novel before making a single guess. They found that by doing this, the computer can remember who was talking earlier in the conversation, even if the characters are far apart in the text. When they tested this on a collection of 22 classic English novels containing more than 35,000 quotes, their new system got the answer right 94.5% of the time. That is a new record, beating the previous best methods. Even more impressive, their system is 20 times faster than the standard fast methods and more than 1,000 times faster than the giant AI brains, all while running on a standard graphics card.

The secret sauce isn't just that the computer is smarter; it's how it thinks. The researchers discovered that the old, fast methods were like a student who forgets what they read five minutes ago. They would look at a quote and try to guess the speaker without remembering the context. The new "joint scoring" method, however, keeps the whole context window in its mind at once. It turns out that the computer's underlying brain (a model called ModernBERT) already had a natural talent for linking pronouns like "he" or "she" to the right person, even over long distances. The old methods accidentally broke this talent by forcing the computer to make decisions one by one. The new method preserves this natural ability, allowing the system to connect the dots across long conversations.

The paper also argues against a few common ideas in the field. For a long time, researchers thought that the main reason computers failed at this task was that they couldn't handle pronouns and nicknames well, so they tried to ignore them and only look for full names. This paper suggests that was a mistake. Their experiments show that including pronouns and nicknames actually makes the system much more accurate, especially in tricky situations where the speaker isn't named directly. They also rule out the idea that you need a super-expensive, massive AI to get good results; their smaller, faster model actually outperformed the giant AI in both speed and accuracy.

To prove their system works in the real world, not just in a controlled lab, they tested it on a different set of 100 book passages where the computer didn't have a reference list of character names. Even without knowing the list of characters in advance, their system improved the accuracy of finding speakers by nearly 10 points compared to the standard tools used by literary researchers today. The authors suggest that this approach opens the door for analyzing huge collections of books quickly and accurately, turning what used to be a slow, error-prone task into something that can be done efficiently. They have even released their new tool, called ModernBookNLP, for others to use, hoping to help more people explore the hidden patterns in literature.

Technical Summary: Fast and Accurate Quotation Attribution in Literary Texts

Problem Definition
Automatic quotation attribution—the task of identifying the speaker of a quotation in literary texts—remains a significant challenge in computational literary analysis. Standard approaches typically employ direct scoring, where a model independently predicts a speaker mention for each quotation based on its local context. While efficient, these methods often struggle with accuracy, particularly in complex conversational settings involving multiple speakers or anaphoric references (e.g., "she said"). Conversely, recent Large Language Model (LLM) approaches achieve state-of-the-art (SOTA) accuracy by processing large context windows jointly but suffer from prohibitive computational costs, rendering them impractical for large-scale corpus analysis.

Methodology: Joint Scoring
The authors propose Joint Scoring, a new formalization of the quotation attribution task inspired by span-based joint scoring paradigms used in coreference resolution. Unlike direct scoring, which processes quotations individually, joint scoring resolves multiple attribution decisions within a single, shared contextual window.

Formalization: The document is divided into overlapping contextual segments ( $C$ ). Within each segment, the model identifies all candidate quotations ( $Q_C$ ) and character mentions ( $M_C$ ).
Representation: A textual encoder (e.g., ModernBERT) generates a shared contextual representation ( $H_C$ ) for the entire segment. Representations for specific quotations and mentions are extracted from this shared embedding.
Objective: The model is trained to jointly optimize the likelihood of all attribution decisions within the segment. The loss function maximizes the probability of gold-standard speaker mentions for all quotations in the context simultaneously.
Hypothesis: The authors posit that this multi-instance objective encourages the model to preserve long-range anaphora resolution signals and local coreference structures that are often degraded when training on isolated quotation instances.

Key Contributions

New Formalization: The paper introduces joint scoring to quotation attribution, demonstrating that unifying accurate but inefficient LLM-based joint attribution with fast but less accurate direct scoring yields superior results.
Performance and Efficiency: The proposed model achieves SOTA performance on the Project Dialogism Novel Corpus (PDNC) while being significantly faster than existing baselines.
Representation Analysis: Through detailed analysis of model representations, the authors identify that joint scoring's advantage stems from its ability to better preserve the pretrained encoder's (ModernBERT) native ability to link coreferent mentions over long distances, a signal that direct scoring tends to erode.
Open Source Tooling: The authors release ModernBookNLP, a modified fork of the standard BookNLP pipeline that integrates their best-performing joint scoring system, facilitating adoption in the research community.

Experimental Results
The system was evaluated on the PDNC, comprising over 35,000 manually annotated quotations from 22 English novels, using 5-fold cross-validation.

Accuracy: The best joint scoring model (using ModernBERT-large with a 2000-token context) achieved 94.5% overall attribution accuracy, surpassing the previous SOTA LLM-based approach (Llama-3-8b) by approximately 5 percentage points. It also reached 92.4% accuracy on challenging non-explicit attribution examples.
Efficiency: The joint scoring approach processes novels 20× faster than comparable direct scoring methods and over 1000× faster than the LLM-based approach on an A100 GPU.
Context and Mention Types: The study found that systems utilizing coreference-based mention candidates (including pronominal and nominal mentions) systematically outperformed those using only named mentions (aliases), particularly when combined with larger context windows.
External Validation: On the LitBank corpus (a more realistic scenario without gold character lists), the modified ModernBookNLP pipeline improved quotation attribution CoNLL F1 scores by nearly 10 points over standard BookNLP, with comparable inference speeds on consumer-grade GPUs.

Significance and Claims
The paper claims that joint scoring offers a practical solution for accurate automatic literary analysis without sacrificing efficiency. By leveraging shared contextual embeddings, the method effectively captures long-range conversational patterns and coreference structures that are critical for resolving complex attributions. The authors emphasize that their approach bridges the gap between the high accuracy of LLMs and the scalability of encoder-based models.

The authors maintain a modest stance regarding limitations, noting that their evaluation is currently restricted to classical English novels and that the system still relies on upstream coreference resolution. They acknowledge that errors in upstream modules (e.g., mention detection) can propagate, and that the efficiency gains are hardware-dependent (specifically leveraging Flash Attention and bfloat16). However, they assert that their work represents a significant step toward scalable, high-fidelity literary analysis tools.

Technical Summary: Fast and Accurate Quotation Attribution in Literary Texts

More like this