Technical Summary: MIRAI – Prediction and Generation of High-Impact Academic Research

Problem Statement

The scientific community faces an accelerating crisis of volume and quality. With scientific literature growing near-exponentially—evidenced by a 16.1% year-over-year increase in Nature Index articles and arXiv submissions reaching record highs in 2026—identifying high-impact work has become increasingly difficult. This challenge is compounded by the proliferation of AI-generated text and a contraction in research funding, which necessitates more efficient resource allocation.

Current methods for evaluating research impact suffer from significant limitations:

• Peer Review: Cannot scale to the current publication rate and is subject to bias and reliance on crude quantitative proxies.
• Citation Metrics: Inherently lagging indicators that require years to materialize and often amplify prestige biases rather than intrinsic merit.
• Existing Prediction Models: Often rely on post-publication data (citation history, author networks) or shallow, domain-specific features, limiting their ability to predict impact at the time of publication.

The core research questions addressed are: (1) Can research impact be predicted at the paper level using only content available at publication (title, abstract, date)? (2) Can this predictive signal be leveraged to generatively synthesize novel research directions likely to be high-impact?

Methodology

Data Collection and Graph Construction

The authors constructed a self-contained academic citation graph exclusively from arXiv papers submitted up to April 9, 2026, comprising 2,976,125 papers (98.74% of the corpus at the time).

• Source: Semantic Scholar API was used to retrieve metadata, including titles, abstracts, publication dates, and reference IDs.
• Graph Structure: Papers were represented as a directed citation graph stored in sparse CSR format. References were filtered to include only papers present within the arXiv corpus to ensure the graph remained self-contained for PageRank computation.
• Fields: The dataset is heavily skewed toward Computer Science, Mathematics, and Physics.

Label Definition

To ensure fair comparison across papers of different ages, the authors defined impact labels based on fixed time horizons ($n$-years post-publication):

1. Citation Count: Defined as the number of citations received within the first $n$ years. The target variable is the log-transformed count: $\ln(1 + C_n)$.
2. PageRank: Computed on the citation subgraph at fixed time horizons. To correct for graph size dilution (where raw PageRank decreases as the graph grows), the authors used the transformation $\ln(N \times \text{PageRank})$, where $N$ is the number of papers in the cohort's subgraph. This metric better captures long-term influence and milestone status.

Model Architecture (MIRAI)

MIRAI (Multi-year Inference of Research trends and Academic Impact) is a deep learning framework designed to predict impact scores for 1 to 5 years post-publication.

• Input:
  • Text: Title and abstract concatenated ("{title}\n{abstract}") and embedded using llama-embed-nemotron-8b (4096-dimensional vector).
  • Temporal Data: Publication date converted to a fractional year and normalized via z-score.
• Network: A feedforward neural network (FFNN) with four hidden layers (2048, 1024, 512, 256 dimensions), utilizing LayerNorm, ReLU activation, and 0.3 dropout.
• Output: A 5-vector of predicted impact scores corresponding to 1 through 5-year horizons.
• Training: The model is trained jointly across all horizons using Mean Squared Error (MSE) loss. The loss is averaged over horizons to ensure the 5-year term receives equal weight despite fewer available labels. Optimization uses AdamW with a learning rate of $3 \times 10^{-4}$.

Research Generation Pipeline

A pipeline was developed to generate high-impact research ideas by synthesizing established work with recent developments:

1. Seeding: Pairs of papers are selected: an "old" paper (top 1% ground-truth PageRank, 2016–2021) and a "new" paper (top 5% MIRAI-predicted PageRank, 2026).
2. Synthesis: An LLM (Claude Haiku 4.5) generates a candidate title and abstract by combining the key contributions of the seed pair.
3. Selection: The pipeline generates $N=32$ candidates. The candidate with the highest MIRAI score is selected as the final output.
4. Variants: The study compares the MIRAI-guided pipeline (Variant A) against an LLM-guided pipeline (Variant B) and a random baseline (Variant C).

Key Results

Impact Prediction Performance

Evaluated on papers published in 2021 (the most recent fully resolved test year), MIRAI achieved:

• 5-Year Citation Prediction: Spearman's $\rho$ of 0.6192.
• 5-Year PageRank Prediction: Spearman's $\rho$ of 0.4686.

Key Trends:

• Horizon Effect: Performance increases monotonically with the prediction horizon. Longer horizons provide more stable labels, outweighing the difficulty of predicting further into the future.
• Metric Comparison: Citation count models outperform PageRank models, likely because PageRank depends recursively on the centrality of citing papers, a signal less inferable from text alone.
• Temporal Stability: Performance is stable through 2019 but declines for test years 2020–2023. This is attributed to a distributional mismatch driven by the rapid expansion of Computer Science submissions on arXiv, which the model handles well, while performance drops in Physics and Mathematics.

Comparison with Zero-Shot LLMs

MIRAI was compared against zero-shot prompting of frontier LLMs (GPT-4o and Hermes 3) on ranking papers by 2-year citation impact:

• Global Ranking: MIRAI achieved $\rho = 0.581$, significantly outperforming GPT-4o ($\rho = 0.336$) and Hermes 3 ($\rho = 0.305$).
• High-Impact Retrieval: MIRAI achieved an Average Precision (AP) of 0.426 for the top 5% of papers, compared to 0.179 for GPT-4o and 0.151 for Hermes 3.
• Conclusion: Supervised training on citation data yields substantially stronger ranking performance than zero-shot LLM prompting.

Research Generation Evaluation

In a study of 300 trials comparing the MIRAI-guided pipeline (A), LLM-guided pipeline (B), and random baseline (C):

• LLM Judge (GPT-5.4): Preferred the LLM-guided pipeline (B) over the MIRAI-guided pipeline (A) in 62.0% of trials. However, both curated pipelines (A and B) significantly outperformed the uncurated baseline (C).
• MIRAI Judge: Naturally preferred its own pipeline (A) in 96.3% of A-vs-B comparisons.
• Rubric Analysis: The LLM judge found that MIRAI selection (A vs. C) improved Significance (+0.82) and Clarity (+0.24) but did not significantly improve Novelty or Technical Soundness. Conversely, LLM selection (B vs. C) improved all four criteria.
• Interpretation: MIRAI captures a real but partial signal correlated with significance and clarity, whereas LLM judges appear to capture a broader range of quality dimensions including novelty and technical soundness.

Significance and Claims

The paper positions MIRAI as a step toward scalable, content-driven assessment of scientific impact that operates with low latency, independent of citation history. Its primary contributions are:

1. Demonstrating Feasibility: Showing that high-impact research can be predicted at publication time using only text and date, achieving performance that surpasses zero-shot LLMs.
2. Generative Application: Proposing a pipeline that uses predictive signals to guide the ideation of novel research directions.
3. Equitable Evaluation: Offering a potential alternative to traditional peer review and lagging citation metrics, which are prone to bias and prestige effects.

The authors modestly conclude that while MIRAI provides a powerful signal for filtering and ideation, it does not yet fully capture all dimensions of research quality (such as novelty and technical soundness) as effectively as human or LLM judges. They suggest that a hybrid approach, combining MIRAI's predictive power with other evaluation signals, represents a promising path for navigating and contributing to the rapidly growing scientific literature. The 5-year citation prediction model is made publicly available to facilitate further research.