Assessing the impact of Open Research Information Infrastructures using NLP driven full-text Scientometrics: A case study of the LXCat open-access platform

This paper proposes a domain-agnostic, NLP-driven scientometric framework that uses full-text analysis to quantify the scholarly impact of open research information infrastructures beyond traditional citations, using the LXCat platform as a case study to reveal fine-grained patterns of data usage and research evolution.

The Gist

The "Library of Ingredients" for Plasma Science: A Simple Breakdown

Imagine you are a world-class chef trying to invent a brand-new type of sustainable fuel. To do this, you don't just need a recipe; you need a massive, perfectly organized warehouse filled with thousands of specific ingredients—the exact saltiness of a certain sea salt, the precise acidity of a specific lemon, and the exact smoke point of a rare oil.

If that warehouse is messy, or if the ingredients are scattered across a thousand different websites, you’ll spend all your time searching and none of your time cooking.

In the world of science, specifically in a field called Low Temperature Plasma (LTP) research, scientists face this exact problem. They need "ingredients" (data like electron collision rates and gas properties) to run their "recipes" (computer simulations and experiments).

LXCat is that massive, community-run warehouse. It’s a digital platform where scientists share the essential "ingredients" needed to understand plasma—the stuff that powers everything from microchips to medical treatments.

---

The Problem: The "Citation" Illusion

Usually, when people want to know if a library or a warehouse is useful, they look at citations. It’s like saying, "This cookbook is important because 100 people mentioned it in their own books."

But there’s a flaw: Just because someone mentions a cookbook doesn't mean they actually used the recipes inside. They might just be acknowledging it exists. To truly know if LXCat is changing the way science is done, the researchers in this paper realized they needed to look deeper. They didn't just want to see if people mentioned LXCat; they wanted to see if people were actually using the ingredients inside it.

The Solution: The "Digital Detective" (NLP)

To solve this, the researchers built a Digital Detective using Natural Language Processing (NLP).

Think of this detective as a super-fast reader that can scan 400 scientific papers in seconds. Instead of just looking for the name "LXCat," the detective looks for "clues" in the text:

1. The Ingredients: Which specific gases (like Nitrogen or Oxygen) are scientists actually talking about?
2. The Tools: Are they using specific digital "blenders" (like a software called BOLSIG+) to process the data?
3. The Specific Shelves: Which specific databases within the LXCat warehouse are being pulled off the shelves most often?
4. The Global Map: Which countries are the most active "chefs" in this kitchen?

What They Discovered

By letting the Digital Detective scan the literature, they found things a simple citation count would have missed:

• The "Staple" Ingredients: They found that Nitrogen and Oxygen are the "salt and pepper" of this field—used constantly.
• The "Power Tools": They saw a strong connection between the data in LXCat and the software tools used to process it, proving that LXCat isn't just a storage unit; it’s a vital part of the "machinery" of science.
• The Growing Kitchen: They saw that while the US has long been a leader, the "cooking" is becoming a global event, with more and more countries joining the community.
• New Flavors: They used "Topic Modeling" (a way of grouping ideas) to see that LXCat isn't just used for one thing. It’s being used for everything from cleaning the environment to powering spacecraft.

Why This Matters

This paper isn't just about plasma; it's a blueprint for a new way to measure impact.

The researchers have created a "measuring tape" that can be used for any scientific resource. Whether it’s a database of human genes or a map of the stars, we can now use this NLP "detective" to see if these resources are actually fueling real progress or if they are just sitting on the shelf gathering digital dust.

In short: They moved from counting "likes" to measuring "actual usage," proving that LXCat is the essential pantry that keeps the kitchen of plasma science running.

Technical Summary

Technical Summary: Assessing the Impact of Open Research Information Infrastructures using NLP-driven Full-text Scientometrics

1. Problem Statement

Traditional scientometrics relies heavily on citation-based metrics (e.g., citation counts, h-index) to evaluate the impact of scientific resources. However, for Open Research Information (ORI) infrastructures—such as data repositories and specialized platforms—citation counts are insufficient. They fail to capture how data is actually used, which specific datasets are most critical, how computational tools are coupled with data, and how research themes evolve in response to available data. In the field of Low-Temperature Plasma (LTP) science, the fragmentation of data historically hindered reproducibility and modeling; while the LXCat platform addressed this, its deep functional impact on scientific workflows remained unquantified.

2. Methodology

The authors propose a scalable, domain-agnostic, NLP-driven scientometric framework. Using the LXCat platform as a case study, they implemented a multi-stage pipeline:

• Corpus Construction: The researchers identified 403 full-text peer-reviewed articles citing three foundational LXCat publications via the Scopus database.
• Data Processing:
  • PDF Conversion: Used the Marker framework (a GPU-accelerated transformer model) to convert PDFs into structured JSON (for metadata/affiliations) and Markdown (for text mining).
  • Text Cleaning: Removed non-linguistic elements (tables, math expressions, and reference sections) to create clean plain-text (TXT) files.
• NLP-Driven Extraction Modules:
  • Chemical Entity Recognition: Utilized ChemDataExtractor and PubChem to identify and normalize gas species (e.g., converting "CO2" and "carbon dioxide" to a single canonical form).
  • Database & Solver Mention Extraction: Employed sentence segmentation and keyword filtering to detect mentions of specific LXCat databases and the BOLSIG+ solver, ensuring author names were not confused with database names.
  • Geographic Mapping: Scanned author affiliations in the first two pages of documents to map research impact to specific countries using ISO standards.
  • Topic Modeling: Applied BERTopic (using all-mpnet-base-v2 embeddings and HDBSCAN clustering) to identify latent research themes within the corpus.

3. Key Contributions

• A New Scientometric Paradigm: Moves beyond "visibility" (citations) to "usage-aware" metrics (how entities, tools, and data interact).
• Integrated Pipeline: A complete, open-source software pipeline for full-text mining, including PDF-to-structured-text conversion and domain-specific entity extraction.
• Domain-Agnostic Framework: While tested on LTP/LXCat, the methodology is designed to be transferable to any field (e.g., mapping chemical species to proteins in biology or datasets to genes in genomics).

4. Results

• Temporal & Geographic Trends: LXCat shows growing integration into research, with a widening international user base (notably increasing participation from China, the UK, and emerging regions like India and Brazil).
• Data Usage Patterns:
  • Species: Nitrogen ($N_2$) and Oxygen ($O_2$) are the most central species, with clear temporal shifts in the demand for specific gases.
  • Databases: Identified a shift from reliance on a few established databases (e.g., Phelps, Biagi) toward a more diverse engagement with multiple datasets, indicating a trend toward cross-validation.
  • Tool-Data Coupling: Revealed a strong functional link between the BOLSIG+ solver and specific datasets, proving that LXCat acts as an "epistemic infrastructure" that connects data to computational workflows.
• Thematic Evolution: BERTopic identified eight distinct research clusters, ranging from fundamental "Electron Collision & Cross Sections" to applied domains like "CO2 Conversion," "Plasma Combustion," and "Space Propulsion."

5. Significance

This work demonstrates that open-access platforms are not merely static repositories but active drivers of scientific practice. By quantifying the "implicit" usage of data, the study provides a roadmap for:

• Infrastructure Design: Helping developers identify gaps in data coverage (e.g., underrepresented species).
• Policy & Funding: Providing evidence-based metrics for policymakers to evaluate the ROI of open-science investments.
• Sustainability: Offering a data-driven method to monitor the health, adoption, and evolution of scientific ecosystems.