Non-synchronism in Global Usage of Research Methods in Library and Information Science from 1990 to 2019

Technical Summary: Non-synchronism in Global Usage of Research Methods in Library and Information Science (1990–2019)

Problem Statement
Library and Information Science (LIS) is an interdisciplinary field where research methods are derived from various disciplines. While the importance of understanding these methods for disciplinary independence and development is well-established, there is a lack of comprehensive, quantitative understanding regarding how research method usage varies across different countries. Previous studies have largely focused on general international trends, specific regions, or single countries, often relying on manual coding. Furthermore, with the emergence of new research methods, the evolution of international differences and trends over the last two decades remains unclear. This study addresses the gap in understanding the "non-synchronism" or asynchronous development of research methods across 81 countries over a 30-year period.

Methodology
The study employs a large-scale, automated quantitative analysis of academic literature to map the distribution and evolution of research methods.

• Data Source: The dataset comprises 5,281 full-text research articles published between 1990 and 2019 in three internationally representative journals: Journal of the Association for Information Science and Technology (JASIST), Journal of Documentation (JDoc), and Library and Information Science Research (LISR). Articles from 81 countries were included; multi-authored papers were attributed to all relevant countries.
• Coding Scheme: The authors adopted a coding scheme based on Chu & Ke (2017), categorizing methods into 15 distinct types (e.g., Bibliometrics, Content Analysis, Experiment, Questionnaire) plus an "Other" category.
• Data Expansion: To address class imbalance in the training data (where some methods had fewer than 300 samples), the authors expanded the dataset. They used TextRank to extract keywords from existing articles, selected top keywords for underrepresented methods, and retrieved additional abstracts from Web of Science. These were manually annotated, resulting in an expanded training set of 2,799 articles.
• Automatic Classification Model: A deep learning model named CogFT (Full-text cognitive-based classification) was developed.
  • Architecture: Inspired by CogLTX, the model partitions full-text articles into 128 blocks. A Summary Network (using SciBERT and Self-Attention) identifies blocks most relevant to research methods, filtering out extraneous content. The top four blocks are merged and fed into a Classification Network (SciBERT with a Sigmoid output layer) for multi-label classification.
  • Noise Reduction: The authors utilized Confident Learning to identify and manually verify potentially noisy labels in the dataset, updating the training data to improve model accuracy.
  • Performance: The final model achieved a Macro-F1 score of 0.8299 and a Micro-F1 score of 0.8171.
• Analytical Techniques:
  • Similarity Analysis: Research method distributions were vectorized. UMAP was used for 2D visualization, and K-means clustering (based on Euclidean distance) grouped countries with similar method profiles.
  • Statistical Testing: The Kolmogorov-Smirnov (K-S) test was used to identify statistically significant differences in method distributions between countries.
  • Temporal Analysis: The Shannon Variety Index measured the diversity of methods and topics over time. Dynamic Time Warping (DTW) and hierarchical clustering were used to group countries based on the trajectory of method variety changes. Jensen-Shannon divergence measured the distance between national and international method distributions for specific topics.

Key Results

• Publication Concentration: The top 20 countries account for over 90% of the total articles. The United States is the dominant contributor (approx. 37-40% of authors), followed by the UK, China, and Canada.
• Method Distribution Differences: Significant heterogeneity exists in method usage. For instance, Belgium and the Netherlands show a heavy preference for bibliometrics (over 45-60% of papers), whereas Norway predominantly uses questionnaires. The top five methods in most countries account for over 60% of their publications, indicating a reliance on a limited set of methods.
• Statistical Divergence: K-S tests confirmed significant differences in method distributions between many country pairs, particularly between high-volume and low-volume publication countries.
• Temporal Trends:
  • Multiple Methods: The overall use of multiple research methods has increased significantly over 30 years. However, this trend is driven largely by countries with lower publication volumes (especially post-2008), while top publishers like the US and UK show less clear upward trends in multi-method usage.
  • Method Variety: Most countries show an upward trend in the variety of research methods used. This correlates with an expansion in research topics.
  • Clustering: Countries were clustered into three groups based on method variety trajectories: (1) Developed countries (US, UK, Canada) with consistently high variety; (2) Rapidly developing countries (e.g., China, Australia, Finland) showing a sharp increase in variety; and (3) Special countries (Belgium) with stable, tradition-influenced patterns.
• Convergence: While national differences persist, the Jensen-Shannon divergence between national and international method distributions for specific topics (e.g., "Information seeking behavior") has decreased over the last three decades, suggesting a gradual convergence toward international norms.

Significance and Contributions
The paper claims several contributions to the field of LIS:

1. Global Perspective: It provides the first large-scale, automated analysis of research method usage across 81 countries over 30 years, moving beyond single-country or regional studies.
2. Methodological Innovation: The study demonstrates the efficacy of using deep learning (CogFT) and full-text analysis for automatic classification of research methods, overcoming the limitations of manual coding and abstract-only analysis.
3. Disciplinary Insight: By highlighting the "non-synchronism" in method usage, the study offers a framework for understanding the unique developmental paths of LIS in different national contexts. It suggests that disciplinary development is influenced by local economic, social, and cultural factors, as well as disciplinary traditions.
4. Practical Guidance: The findings can guide national-level discipline development by identifying gaps or over-reliance on specific methods. Furthermore, understanding these differences can foster better transnational scientific cooperation and academic exchange by aligning expectations and methodologies.

The authors conclude that while differences in research methods remain, the field is becoming more integrated, with a gradual decrease in the divergence between national practices and international trends.