Technical Summary: Enhancing Academic Paper Recommendations Using Fine-Grained Knowledge Entities and Multifaceted Document Embeddings

1. Problem Statement

The rapid expansion of global scholarly output has intensified the burden of literature review, making it difficult for researchers to efficiently locate papers that align with their specific interests. Existing academic paper recommendation systems primarily rely on coarse-grained similarities based on general topics, fields, or historical interactions (e.g., collaborative filtering, citation networks). While effective for broad discovery, these approaches fail to address scholars' fine-grained needs, such as locating papers that tackle the same research task using different methods, or papers that utilize specific materials or metrics within the same domain. Furthermore, current methods often underutilize fine-grained knowledge entities present in text and lack the ability to dynamically model semantic compositions among heterogeneous entity types to reveal diverse research perspectives.

2. Methodology

The proposed framework introduces a novel recommendation method that integrates multidimensional information through fine-grained knowledge entities and multifaceted document embeddings. The methodology consists of three primary stages:

2.1 Construction of a Fine-Grained Scientific Knowledge Graph (FG-SKG)

The system constructs a heterogeneous information network to capture detailed research elements.

• Entity Extraction: Four primary types of fine-grained knowledge entities are identified from paper titles and abstracts: Task (what is being studied), Method (how it is conducted), Material (resources/datasets used), and Metric (evaluation indicators).
• Model Architecture: A SciBERT + BiLSTM + Cascade architecture is employed for entity recognition. This model uses SciBERT for token-level semantic encoding, a BiLSTM for contextual dependency capture, and a cascade decoder (entity tagger + entity-type tagger) with a CRF layer to ensure valid label transitions and high boundary precision.
• Relation Extraction: Instead of relying on static mappings, the system uses dependency syntax templates to extract high-confidence relations between entities. Key relations include achievedBy (Task–Method), usedBy (Material–Task), evaluatedBy (Task–Metric), and related (intra-type). This approach prioritizes precision to mitigate error propagation.
• Graph Structure: The FG-SKG integrates paper nodes, entity nodes, and three edge types: paper-paper citations, paper-entity links, and entity-entity relations.

2.2 Multidimensional Document Embedding

To represent papers holistically, the method generates embeddings that combine textual content, citation context, and entity semantics.

• Paper Nodes: The SPECTER model (Transformer-based) is used to encode paper titles and abstracts, capturing citation-aware semantic representations.
• Entity Nodes: GPT-3.5 is utilized to generate embeddings for the extracted knowledge entities, leveraging its sequence modeling capabilities for short, definitional contexts.
• Vector Composition: Paper representations are formed by concatenating the SPECTER paper vector with aggregated entity vectors (sum-pooled Task, Method, and Material/Metric vectors). This allows for multi-granular representations, such as:
  • Overall: [Task, Method, Material, Paper]
  • Task-focused: [Task, Paper]
  • Task-Method: [Task, Method, Paper]

2.3 Vector-Based Recommendation and Ranking

Recommendations are generated by computing similarity between query and candidate paper vectors, employing learned convex weights to balance different semantic dimensions.

• Similarity Computation: The system calculates cosine similarity across different vector combinations (e.g., overall, task-only, task-method).
• Weighted Aggregation: A target vector is formed as a convex combination of various representations ($p = \sum w_i p_i$). The weights are learned on a development set to optimize ranking metrics (e.g., MAP).
• Diversity Control: To address diverse information needs (e.g., "same task, different method"), the system employs a task-aware refinement step. Within a subset of task-similar papers, signals are reweighted to filter for papers with different methods or materials, promoting diversity while maintaining relevance.

3. Key Contributions

1. Fine-Grained Recommendation Framework: The paper proposes a method that aligns recommendations with scholars' specific and diverse literature needs by dynamically composing fine-grained entities (Task, Method, Material, Metric) rather than relying solely on topic-level similarity.
2. Multidimensional Embedding Strategy: It introduces a systematic approach to integrating heterogeneous signals (text, citations, and structured knowledge entities) into unified document representations, enhancing both semantic coverage and representation fidelity.
3. Systematic Evaluation: The study provides a comprehensive evaluation on the STM-KG dataset (covering 10 scientific domains), demonstrating that the proposed method outperforms baseline models. It also includes ablation studies on embedding models, entity types, and weighting strategies.

4. Experimental Results

The method was evaluated using the STM-KG dataset and the AMiner v12 dataset.

• Entity Extraction: The proposed SciBERT + BiLSTM + Cascade model achieved a micro F1-score of 85.10 for entity identification, outperforming baseline models (BiLSTM+CRF, SciBERT, SciBERT+CRF) and zero-shot/few-shot GPT-3.5 approaches.
• Recommendation Performance:
  • The proposed method achieved a Top-50 Precision of 27.3% on the STM-KG dataset.
  • This represents a 6.7 percentage-point improvement over existing baseline approaches (e.g., Baseline 1, Baseline 2, SMIGNN, TIDRec, and LLM-enhanced models).
  • In cross-domain scenarios, the proposed method demonstrated greater stability and smaller performance degradation compared to baselines, attributed to the integration of citation information and structured entity embeddings.
• Ablation Studies:
  • Entity Contribution: Removing entity types revealed that Task entities contribute most to accuracy, followed by Method and Material/Metric.
  • Embedding Models: GPT-3.5 embeddings for entities consistently yielded the best results, followed by SciBERT.
  • Weighting: Learned convex weights significantly improved performance over uniform weighting.
  • Diversity vs. Accuracy: The study highlighted a trade-off; while accuracy-tuned weights (λ=0) maximize MAP/nDCG, diversity-aware re-ranking (λ>0) significantly improves Intra-List Diversity (ILD) and Coverage with a controlled decrease in accuracy.

5. Significance and Claims

The paper claims that its primary significance lies in addressing the joint requirements of fine-grained relevance and diversity in academic literature review. By moving beyond coarse topic matching to model dynamic semantic combinations of heterogeneous entities, the system can:

• Provide recommendations that are not only topically similar but also methodologically or resource-diverse.
• Offer interpretable recommendations where users can trace results back to specific semantic components (e.g., "recommended because it uses the same task but a different method").
• Achieve state-of-the-art performance without relying solely on end-to-end LLM generation, instead using structured knowledge graphs to enhance interpretability and control.

The authors position this work as a structured, interpretable foundation that complements Large Language Models (LLMs), offering a scalable and efficient alternative for large-scale academic recommendation systems while acknowledging that future work could explore hybrid frameworks integrating generative techniques.