Exploring the relationship between team institutional composition and novelty in academic papers based on fine-grained knowledge entities

Technical Summary: Exploring the Relationship Between Team Institutional Composition and Novelty in Academic Papers

Problem Statement
While the composition of author teams is recognized as a significant factor influencing the novelty of academic papers, existing research exhibits two primary limitations. First, studies on team composition have predominantly focused on individual attributes such as team size, gender balance, and international collaboration, largely neglecting the impact of the institutional types (academic vs. industrial) to which authors belong. Second, current measures of novelty often remain at a general or macro level, failing to capture fine-grained nuances. Traditional methods rely on broad entity combinations (e.g., journal references or topic pairs) without distinguishing specific knowledge elements like methods, datasets, tools, or metrics. This lack of granularity makes it difficult to analyze specific types of novelty or provide detailed explanations of how different institutional biases manifest in research outcomes.

Methodology
The study employs a quantitative approach using data from the Natural Language Processing (NLP) field as a case study. The methodology proceeds through four distinct steps:

1. Data Collection and Team Categorization:

   • Source: 15,337 papers published between 1979 and 2021 were collected from three top-tier NLP conferences: ACL, EMNLP, and NAACL.
   • Institutional Classification: Author affiliations were extracted and mapped using the Global Research Identification Database (GRID). Institutions were categorized into eight types (e.g., education, corporations, healthcare) and subsequently aggregated into three team composition types:
     • A&A: Purely Academic institutions.
     • I&I: Purely Industrial institutions.
     • A&I: Mixed Academic and Industrial collaborations.
   • Filtering: Single-authored papers were excluded, resulting in a final dataset of 14,812 papers.

2. Fine-Grained Entity Extraction:

   • The study identified four specific types of knowledge entities relevant to NLP research: Methods (algorithms/models), Datasets (corpora), Tools (software/frameworks), and Metrics (evaluation standards).
   • A cascaded SciBERT+BiLSTM model was trained and utilized to extract these entities from the full text of the papers. This model achieved an F1 score of 86.27, outperforming general pre-trained models like BERT and XLM-RoBERTa.

3. Novelty Measurement:

   • Combinatorial Innovation: Novelty was defined based on the theory of combinatorial innovation, where novelty arises from atypical combinations of knowledge elements.
   • Semantic Distance: The semantic distance between pairs of extracted entities was calculated using cosine distance on word embeddings.
   • Scoring: Entity pairs falling within the top 10% of semantic distances (indicating the least common/atypical combinations) were identified. The proportion of these atypical combinations in a paper served as its Novelty Score. Papers in the top 10% of scores were classified as "novel papers."

4. Bias Analysis and Regression:

   • Fine-Grained Bias: The study calculated the contribution rate of the ten possible pairwise combinations of the four entity types (e.g., Method-Method, Method-Metric, Method-Tool) to the overall novelty of papers.
   • Statistical Analysis: Regression models were employed to analyze the relationship between team institutional composition and novelty scores, controlling for variables such as publication year, research task, team size, international collaboration, and institutional prestige.

Key Results

• Publication Trends: Academic institutions (A&A) remain the dominant producers of NLP papers. However, mixed academic-industrial collaborations (A&I) have shown a remarkable surge in publication volume since 2010, outpacing the growth rate of pure academic papers.
• Institutional Impact on Novelty: Regression analysis reveals that, compared to purely industrial teams (I&I), both purely academic teams (A&A) and mixed teams (A&I) produce papers with significantly higher novelty scores. Specifically, mixed teams (A&I) demonstrate the highest positive impact on novelty.
• Fine-Grained Novelty Bias:
  • Overall: Method-related entity combinations contribute the most to novelty across all institution types.
  • Mixed Teams (A&I): These teams show a significantly higher contribution to novelty through Method-Metric combinations. This suggests a tendency to integrate diverse methodological approaches with evaluation indicators.
  • Industrial Teams (I&I): These teams exhibit a significantly higher contribution to novelty through Method-Tool combinations, reflecting a focus on leveraging innovative tools to solve practical application problems.
  • Academic Teams (A&A): These teams show a balanced but distinct preference for Method-Data and Method-Method combinations.

Significance and Contributions
The paper claims two primary contributions to the field of scientometrics and team science:

1. Institutional Perspective on Novelty: The study fills a gap in literature by shifting the focus from individual author attributes to institutional composition. It provides empirical evidence that the type of institution (academic, industrial, or mixed) significantly influences the novelty of research outcomes, highlighting the critical role of industry-academia collaboration in generating novel knowledge.
2. Fine-Grained Novelty Measurement: The research advances novelty measurement beyond binary or scalar assessments. By introducing a fine-grained novelty bias measurement based on specific knowledge entities (methods, data, tools, metrics), the study reveals distinct "innovation signatures" for different institutional types. This allows for a more nuanced understanding of how different teams innovate (e.g., industrial teams focusing on tool integration vs. mixed teams focusing on method-indicator synthesis).

Limitations
The authors acknowledge several constraints:

• Data Scope: The study is limited to the NLP field and three specific conferences, which may not fully represent the broader spectrum of scientific research.
• Data Imbalance: The dataset contains a significantly higher number of academic papers compared to industrial papers, which could introduce bias.
• Contextual Nuance: The entity extraction and combination analysis were performed across the entire document without distinguishing specific sections (e.g., methodology vs. results), potentially affecting the precision of the novelty interpretation.
• Causality: The study identifies correlations between institutional composition and novelty but does not establish a definitive causal relationship.

Conclusion
The study concludes that while industrial institutions are increasingly active in publishing, their highest novelty potential is realized through collaboration with academic institutions. The findings suggest that interdisciplinary cooperation between academia and industry is a key driver for high-novelty research, with different institutional types contributing unique biases to the types of knowledge combinations they produce. This provides guidance for fostering effective industry-university-research cooperation to enhance the quality and innovativeness of scientific output.