💻 computer science

Digging Up Citations: FOSSIL, a Dataset and Workflow for Reference Extraction in Law and the Humanities

This paper introduces FOSSIL, a multilingual dataset and accompanying workflow designed to improve the extraction of footnote-based citations in law and humanities scholarship, demonstrating that a specialized pipeline nearly doubles extraction quality compared to standard tools while highlighting remaining challenges in handling cross-references and mixed-content footnotes.

Original authors: Luca Foppiano, Christian Boulanger

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Luca Foppiano, Christian Boulanger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to organize a massive library. In the Natural Sciences (like biology or physics), books have a very neat, predictable structure: the main story is in the chapters, and all the "source notes" are tucked away in a clean, numbered list at the very back. It's like a recipe where the ingredients are listed separately from the cooking instructions. Computers are very good at reading these recipes.

However, in Law and the Humanities (history, philosophy, literature), the "source notes" are messy. They are buried inside the text itself, often in the bottom margins (footnotes), mixed in with the author's personal thoughts, clarifications, and cross-references. It's like a recipe where the ingredients are hidden inside the sentences like, "Add two cups of flour (which I bought at the market on Tuesday, see page 4) while stirring."

This paper, titled "Digging Up Citations: FOSSIL," is about building a better way to find and organize those hidden ingredients in Law and Humanities books.

Here is the breakdown of what they did, using simple analogies:

1. The Problem: The "Black Box" and the "Messy Basement"

The authors explain that existing computer programs (models) are trained on the neat "Natural Science" style. When they try to read Law and Humanities footnotes, they get confused because the data is mixed up.

They also note that while powerful AI tools (like big commercial chatbots) can sometimes figure this out, they are risky. They are like renting a high-tech excavator from a company that might go out of business tomorrow, taking your data with them. The authors wanted a tool that is open, free, and stays with them forever.

2. The Solution: The "FOSSIL" Project

The team created a complete toolkit to dig up these citations. They call the dataset FOSSIL (Footnote-based Open-access SSH Scientific Instance Labels). Think of this as a massive, organized collection of "before and after" examples that teach computers how to read messy footnotes.

They built four main things:

A Digital Workbench (PDF-TEI Editor): A web tool that lets humans and computers work side-by-side. It shows the original PDF on one side and the structured data on the other, allowing people to fix mistakes and teach the computer what to look for.
A Training Manual (The Workflow): A step-by-step guide on how a team of seven people (including experts and students) cleaned and labeled the data.
The Treasure Trove (The Dataset): They collected 96 academic articles from law, history, and social sciences. These articles contain over 7,600 references hidden in footnotes. Crucially, this data is "open license," meaning anyone can use it without legal trouble.
A Specialized Engine (Grobid Specialization): They took an existing open-source tool called Grobid (which is like a standard library scanner) and gave it a "specialized lens" to focus specifically on the messy footnotes of Law and Humanities.

3. The Challenge: Why is this so hard?

The paper explains that footnotes in these fields are tricky because they do five different things at once:

Just a comment (no citation).
A bio note about the author.
A clean list of books.
A "see also" note pointing to an earlier footnote (like "See note 5").
A chaotic mix of all the above.

It's like trying to sort a pile of mail where some envelopes contain letters, some contain checks, some contain photos, and some contain a mix of all three, all written in different languages and handwriting styles.

4. The Results: From "Missing" to "Found"

The team tested their new specialized engine against the old, standard version.

The Old Way: The standard tool was very picky. It was almost perfect at identifying a reference if it found one (high precision), but it missed 70-80% of the actual references because they didn't look like standard scientific citations (low recall). It was like a metal detector that only beeps for gold coins but ignores silver ones.
The New Way: The specialized "FOSSIL" version found twice as many references.
- It went from finding only 21% of publication dates to finding 71%.
- It went from finding 23% of author names to finding 60%.
- Overall, the quality of the extraction nearly doubled (the score went from 0.36 to 0.72).

5. The Conclusion

The paper concludes that you cannot just "tweak" the settings of a standard tool to fix this problem; you need a tool specifically trained on the messy reality of Law and Humanities footnotes.

FOSSIL is now a stepping stone. It proves that with the right data and a specialized workflow, computers can finally start reading these complex footnotes accurately. The authors plan to expand this to more languages and tackle even harder problems, like automatically connecting "See note 5" back to the original note 5.

In short: They built a better shovel and a map to dig up citations that were previously buried in the mud of academic footnotes.

Technical Summary: FOSSIL – A Dataset and Workflow for Reference Extraction in Law and the Humanities

Problem Statement
Citation extraction has been extensively researched, yet existing models are predominantly optimized for the structured, end-of-document bibliographies typical of the natural sciences. In contrast, scholarship in law and the humanities relies heavily on footnotes to embed bibliographic data alongside commentary, clarifications, and cross-references. This interleaving creates highly heterogeneous content that resists standard pattern-based parsing. While Large Language Models (LLMs) show promise for handling such unstructured data, closed models suffer from reproducibility issues and infrastructure dependencies, while open-weight alternatives require significant hardware resources. Furthermore, the field lacks specialized, gold-standard datasets for Social Sciences and Humanities (SSH) literature, hindering both the evaluation of LLM-based approaches and the training of classical supervised sequence labellers.

Methodology
The authors propose a workflow grounded in the Grobid architecture, which utilizes a cascade of specialized machine-learning models to parse PDF articles into structured TEI-XML. The methodology consists of four integrated components:

Annotation Tool (PDF-TEI Editor): An open-source, web-based application supporting iterative, collaborative annotation. It features a synchronized dual-pane PDF/XML view, built-in TEI schema validation, role-based access control, and a plugin architecture for integrating extraction backends (currently Grobid and LLaMore).
Annotation Workflow: A five-phase process involving a team of seven annotators (including senior researchers and Digital Humanities students). The workflow moves from pre-annotation using the latest Grobid model to correction, verification, senior validation, and feedback integration into a living annotation guide. The process utilized 194 pairwise comparisons on 96 documents, achieving a mean Krippendorff's $\alpha$ of 0.917.
Dataset Construction (FOSSIL): The team constructed FOSSIL (Footnote-based Open-access SSH Scientific Instance Labels), an openly licensed, multilingual dataset comprising 96 scholarly articles from 38 journals in law, humanities, history, and social sciences (1959–2025). The corpus includes approximately 2,400 footnote blocks (roughly 500 discursive and 1,900 bibliographic) containing over 7,600 individual references. Annotations are organized in three layers aligned with the Grobid cascade: article segmentation, reference segmentation, and citation parsing.
Model Specialization: The authors implemented a specialized "flavor" for Grobid that incrementally replaces models in the standard cascade (starting with article segmentation) while reusing the rest of the pipeline. This design allows for targeted evaluation of specific components.

Key Contributions
The paper presents four primary contributions:

PDF-TEI Editor: A production-ready, open-source web tool for collaborative TEI-XML annotation.
Documented Workflow: A reproducible annotation protocol integrating the editor and Grobid, executed by a multi-role team.
FOSSIL Dataset: The first openly licensed, large-scale resource of its kind for legal scholarship, spanning multiple languages (English, Portuguese, Italian, German) and citation styles.
Grobid Specialization: A proof-of-concept baseline demonstrating a domain-specific adaptation of Grobid for footnote-heavy documents.

Results
The authors evaluated the specialized pipeline against the default Grobid configuration using 5-fold cross-validation and end-to-end testing on four scholarly articles.

Article Segmentation: The specialized model achieved an overall F1 score of 81.08 (Precision: 82.91, Recall: 79.41). Performance was strong for <body> (86.85) and <page> (90.12) but lower for <footnote> (54.95) and <front> (34.54), where physical page space overlap creates inherent classification ambiguity.
End-to-End Extraction: The default Grobid model exhibited high precision (0.89–0.94) but extremely low recall (0.21–0.27) for key fields (author surname, journal title, publication date), resulting in F1 scores between 0.35 and 0.42. This indicated a fundamental mismatch with footnote-based citation styles.
Specialized Performance: The specialized "footnotes-ref" flavor substantially corrected this imbalance. Recall improved significantly across all fields (e.g., author surname recall rose from 0.23 to 0.60; publication date from 0.21 to 0.71), while precision remained acceptable. Consequently, the micro-averaged F1 score nearly doubled, rising from 0.36 to 0.72. The most significant gain was observed in publication date extraction (F1 0.35 → 0.80), attributed to better handling of variable date conventions and abbreviated formats.

Significance and Claims
The paper claims that FOSSIL is the first openly licensed resource of this scale specifically designed for legal scholarship and footnote-heavy citation practices. The evaluation confirms that domain-specific specialization of Grobid's sequence-labelling models substantially improves citation parsing for law and humanities, addressing the primary failure mode of existing tools: extremely low recall.

The authors assert that the low recall of standard models cannot be resolved through post-processing alone, necessitating the dataset and workflow contributions presented. While the specialized model shows substantial gains, the paper modestly notes that significant "headroom" remains, particularly regarding cross-reference resolution and mixed-content footnotes. Future work is proposed to extend the dataset to additional languages and legal traditions, tackle cross-reference resolution, and explore LoRA-based fine-tuning of LLMs and end-to-end neural architectures.