Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora

Technical Summary: Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation

Problem Statement
Citation link prediction is a central problem in informetrics and network science, yet its methodological validity is frequently compromised by temporal leakage. Many existing studies construct features from fully aggregated, static citation graphs, inadvertently allowing future information (e.g., a paper's eventual citation count or the final topology of the network) to influence predictions about past citation events. This "future topology leakage" and related forms of leakage artificially inflate predictive performance and obscure the historically available information that actually governed citation behavior. While temporal leakage is a known issue in applied machine learning, citation prediction remains uniquely vulnerable due to the frequent neglect of chronological distance structures in benchmark construction.

Methodology
The study introduces a strongly leakage-mitigated methodological framework designed to evaluate citation formation under temporally rigorous conditions. The core of this approach is the temporally capped graph ($G_{t_A-1}$), where all network-derived quantities are computed exclusively from information available prior to the publication year ($t_A$) of the citing paper.

1. Leakage Taxonomy: The authors formalize four distinct forms of temporal leakage:
   • Future Citation Leakage: The target link exists in the feature computation graph.
   • Future Topology Leakage: Centrality/overlap measures use nodes/edges entering the network at or after $t_A$.
   • Metadata Leakage: Use of retrospectively updated metadata (e.g., future Impact Factors).
   • Semantic Leakage: Text embeddings from pre-trained models that may implicitly encode citation contexts from the future.
2. Data Infrastructure: The framework is applied to ten large-scale bibliographic corpora retrieved from Dimensions.ai, spanning five domains: Science (Protein Folding, CRISPR), Engineering (Additive Manufacturing, Corrosion Protection), BioMed (Neuroblastoma, Osteosarcoma), Social Science (Income Inequality, Organizational Behavior), and Humanities (Film Studies, Memory Studies). All corpora cover the period 1975–2024.
3. Candidate Space & Negatives: To avoid trivial negatives, the study employs temporally and topically constrained negatives. For a citing paper $A$ and a true cited paper $B$, negative candidates are sampled from papers published within $\pm3$ years of $B$ and belonging to the same topical corpus, ensuring the model discriminates between contemporaneous, topically relevant papers.
4. Feature Engineering & Decomposition: Features are grouped into three explanatory blocks for an S-T-N$_{prom}$ decomposition:
   • S (Semantic): Directional semantic alignment using SBERT embeddings of titles and abstracts.
   • T (Temporal): Citation time gap ($t_A - t_B$).
   • N$_{prom}$ (Network Prominence): Chronological distance in-degree and year-capped PageRank.
5. Modeling & Evaluation: Gradient Boosting is used as the primary classifier. Evaluation includes 5-fold cross-validation, strict chronological hold-out tests (training $\le$2015, testing 2016–2020), and top-k ranking experiments (MRR, NDCG@k) to bridge pairwise discrimination and retrieval realism.

Key Results

• Predictive Performance: Under strict leakage-mitigated conditions, Gradient Boosting achieves ROC-AUC scores ranging from 0.530 (Corrosion Protection) to 0.734 (CRISPR). These scores reflect the intrinsic difficulty of the task rather than model failure, as temporal hold-out performance remains consistent with cross-validation.
• Dominance of Chronological Distance (T): The decomposition reveals that chronological recency (T) is the dominant explanatory component across almost all corpora.
  • $AUC(T)$ alone ranges from 0.577 to 0.824.
  • The marginal contribution of T beyond S ($\Delta T$) is positive in all ten corpora.
• Redundancy of Semantic Alignment (S): Adding directional semantic alignment to a model that already includes chronological distance reduces predictive power ($\Delta S < 0$) in all ten corpora. In four of the ten corpora, cited papers exhibited marginally lower directional semantic alignment than non-cited candidates.
• Negligible Contribution of Prominence (N$_{prom}$): Once T is known, citation-network prominence adds negligible predictive signal ($\Delta N|_{ST} \approx 0$ or negative in 8/10 corpora). This is not due to multicollinearity, as T and N$_{prom}$ are largely independent in most datasets.
• Semantically Surprising Citations: When citations occur despite weak semantic alignment, they tend to be temporally proximate. No consistent prominence pattern emerges for these cases, except in specific corpora like Income Inequality where prominence acts as the primary separator.

Significance and Claims
The paper positions its primary contribution as methodological, providing a rigorous evaluation framework that prioritizes historical validity over maximizing accuracy. The study claims that:

1. Chronological distance is the primary driver of citation formation within narrowly defined, keyword-based scholarly communities, outweighing semantic alignment and network prominence under leakage-controlled conditions.
2. Semantic alignment is largely redundant with chronological recency in these constrained settings, offering little discriminative power beyond the age of the cited paper.
3. Citation prediction is fundamentally a retrieval problem that requires evaluation beyond pairwise discrimination; the study demonstrates that performance degrades as candidate pool complexity increases, highlighting the limits of current feature sets in broader retrieval scenarios.
4. Domain-specific heterogeneity exists: While T-dominance is a consistent pattern, exceptions exist (e.g., Income Inequality where prominence dominates, and Memory Studies where semantic alignment contributes more significantly), suggesting that citation dynamics are corpus-specific rather than governed by a universal law.

The authors explicitly delimit their scope, acknowledging that the findings are specific to the keyword-defined corpora and constrained candidate spaces used. They do not claim these results generalize to full-corpus retrieval or that semantic content is irrelevant to citation formation in broader contexts, but rather that within the specific constraints of their leakage-mitigated framework, time is the most potent predictor.