Ontology-Guided Citation Intent Classification for Interpretable Scholarly Information Discovery
This paper proposes an ontology-guided framework that integrates explicit semantic features from the Citation Context and Reason Ontology (CCRO) with contextual language representations to enable interpretable citation intent classification, achieving predictive performance comparable to fine-tuned SciBERT while providing transparent, instance-level audit trails for scholarly information discovery.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the vast library of human knowledge, a citation is more than just a footnote or a reference list entry; it is a conversation. When a scientist writes a paper and mentions the work of another, they are doing one of several things: they might be building a new method on top of an old one, they might be comparing their results to see who did better, or they might be pointing out a flaw in a previous study. For decades, the academic world has measured the importance of a paper simply by counting how many times it was cited. This is like judging a book solely by the number of times it appears on a reading list, without ever knowing if the reader loved it, hated it, or just used it as a paperweight. While counting citations tells us how much attention a work received, it fails to capture the nuance of why it was mentioned. To truly understand how science evolves, we need to know the intent behind the reference.
Researchers have long tried to teach computers to read these citations and understand the intent, moving from simple word counts to sophisticated artificial intelligence that can grasp the context of a sentence. However, these powerful AI systems often act as "black boxes," making decisions that are difficult for humans to inspect or trust. A new study by Muhammad Bilal Rehman and his colleagues at institutions in Pakistan and China proposes a different path. They built a system that combines the deep understanding of modern AI with a clear, human-readable set of rules about what citations mean. Their goal was not just to predict the intent accurately, but to do so in a way that allows a human to see exactly why the computer made its choice.
The team focused on a massive collection of 8,736 sentences from the field of computational linguistics, where every sentence was carefully labeled by human experts to identify its specific purpose, such as "incorporating" a method, "extending" a theory, or "criticizing" a finding. They created a framework that feeds the text into four different types of analysis simultaneously. One part looks at the specific grammar and word types, another counts how unique certain words are, a third uses a specialized AI model trained on scientific text to understand the broader meaning, and the fourth uses a structured map of knowledge called an ontology. This ontology acts like a dictionary of citation reasons, checking if specific phrases like "builds upon" or "differs from" appear in the text.
The researchers found that their hybrid system achieved predictive performance comparable to the most advanced AI models currently available, matching the accuracy of a fine-tuned SciBERT baseline without establishing a statistically significant advantage. Crucially, they discovered that while the specialized AI model (the contextual stream) remained the largest and most information-dense driver of the system's decisions, the structured knowledge map provided essential complementary information. When they removed the structured knowledge map from the system, the accuracy dropped significantly, proving that the explicit rules were providing unique value that the AI alone was missing. The study showed that the system could successfully combine the deep, intuitive understanding of neural networks with the clear, logical structure of human-defined rules.
Perhaps the most valuable result was the system's ability to explain itself. Because the different parts of the analysis remained separate, the researchers could trace every single prediction back to its source. For any given citation, the system could show which specific words triggered a rule, which grammatical patterns were detected, and how the AI interpreted the context. In about 72 percent of the cases where the system used these explicit rules, the final decision matched the rule's suggestion. When the system made a mistake, the researchers could look at the internal record and see exactly which part of the analysis had gone wrong, such as a specific word being misinterpreted or a rule failing to fire. This transparency means that scholars can trust the system not just because it is accurate, but because they can see the evidence behind its conclusions.
The study also highlighted the challenges of working with real-world academic data. The collection of sentences was heavily skewed, with nearly 70 percent of them simply discussing previous work in a general way, while only a tiny fraction involved strong criticism or the incorporation of new methods. The researchers found that simply balancing the data to give equal weight to these rare categories improved the system's ability to spot them, but it also made the system more likely to guess those rare categories even when it shouldn't. This trade-off between finding rare events and being precise is a common hurdle in analyzing scientific literature.
Ultimately, the work demonstrates that we do not have to choose between powerful AI and human understanding. By guiding the machine with a clear map of what citations mean, the researchers created a tool that is both accurate and transparent. This approach offers a way forward for building better tools to explore scientific literature, allowing researchers to search for papers not just by topic, but by how they were used—finding studies that were criticized, those that were extended, or those that served as a foundation for new discoveries. The study suggests that the future of scholarly discovery lies in systems that can think deeply about text while remaining open to human inspection.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.