Autonomous Research Assistant: An AI-Powered System for Automated Paper Discovery, Summarization, Domain Classification, and Citation Generation

Technical Summary: Autonomous Research Assistant

Problem Statement
The academic landscape is currently overwhelmed by the sheer volume of new research, with over 200 million papers indexed in major databases. Researchers face significant challenges in keeping pace with this growth, specifically in three areas: discovering relevant studies efficiently, quickly grasping the core contributions of individual papers, and managing citations across varying formatting standards (e.g., APA, MLA, IEEE). Traditional literature reviews are time-consuming, often taking weeks, while existing tools typically address only isolated parts of the workflow (search, summarization, or citation management), forcing researchers to switch between multiple platforms and lose organizational continuity.

Methodology and System Architecture
The authors present the Autonomous Research Assistant, a full-stack AI web application designed to automate the end-to-end research workflow. The system integrates four primary capabilities: automated paper discovery, AI-powered summarization, machine learning-based domain classification, and multi-format citation generation.

• Architecture: The system is built on a serverless, multi-tier architecture. The frontend utilizes React 18 and TypeScript with Tailwind CSS for the presentation layer, while state management is handled by TanStack Query and React Hook Form. The backend relies on Deno Edge Functions, ensuring low-latency, serverless execution. Data persistence is managed via PostgreSQL with Row-Level Security (RLS) to ensure user data isolation and privacy. Authentication is handled via JWT.
• Paper Discovery & Summarization: The system queries the Semantic Scholar API to access a library of over 200 million papers. It employs a robust retry logic (up to three attempts with exponential backoff) to handle rate limits. If the API fails, an AI fallback mechanism triggers. For summarization, the system leverages Google Gemini 2.5 Flash to generate abstractive summaries (2–3 sentences) and extract 5–7 relevant keywords from paper abstracts in real-time.
• Domain Classification: To categorize papers into six distinct research domains, the authors trained a machine learning model on the Cornell University arXiv Paper Abstracts dataset.
  • Dataset: 18,000 abstracts were selected across six disjoint domains: Computer Vision (cs.CV), Natural Language Processing (cs.CL), Astrophysics (astro-ph.GA), Combinatorics (math.CO), Neuroscience (q-bio.NC), and High Energy Physics (hep-th).
  • Preprocessing: The pipeline includes case normalization, removal of non-alphabetic characters (except hyphens), tokenization, stopword removal (including generic academic terms), and lemmatization. Titles are prepended to abstracts twice to reinforce context.
  • Feature Engineering: TF-IDF vectorization was used with a maximum of 50,000 features, utilizing unigrams through trigrams.
  • Model Selection: Five classifiers were compared: Logistic Regression, Multinomial Naive Bayes, Linear SVM, SGD Classifier, and Random Forest. The Linear SVM with Platt scaling and hyperparameter tuning (C=1.0) emerged as the superior model.
  • Deployment Strategy: Instead of deploying a Python-based sklearn model directly into the Deno environment (which would require complex containerization), the authors utilized a prompt engineering approach. The trained taxonomy and classification logic were encoded into a prompt for Gemini 2.5 Flash, allowing the Edge Function to perform domain classification alongside summarization without a Python runtime.
• Citation Generation: The system automatically generates citations in APA, MLA, and IEEE formats based on retrieved metadata, allowing for single-click export.

Key Results

• Classification Performance: The tuned Linear SVM achieved a test accuracy of 97.25% with a macro F1 score of 0.97 on a stratified 80/20 split (3,600 test samples). This outperformed Logistic Regression (97.06%), SGD Classifier (97.03%), Multinomial Naive Bayes (96.69%), and Random Forest (94.44%).
  • Per-class analysis showed the highest F1 scores for Astrophysics and NLP (0.98), with Computer Vision slightly lower (0.96) due to vocabulary overlap with NLP.
  • The model was validated via 5-fold cross-validation (96.17% ± 0.45%) and a custom test of six hand-crafted abstracts, achieving 100% accuracy.
• System Latency: The end-to-end pipeline for processing 5 papers takes between 8 to 15 seconds. Specific breakdowns include ~1.2 seconds for summarization, ~0.8 seconds for keyword extraction, and ~0.6 seconds for domain classification per paper.
• Efficiency: The TF-IDF + Linear SVM approach trained in under 30 seconds on Google Colab's free tier, requiring no GPU, contrasting with the high computational costs of transformer-based models like BERT or SciBERT.

Significance and Claims
The paper claims that the Autonomous Research Assistant successfully demonstrates that a complete research automation pipeline can be delivered as a production-grade web application with strong security guarantees. Key contributions include:

1. Holistic Workflow: Unifying discovery, summarization, classification, and citation into a single interface, eliminating the need to switch between disparate tools.
2. Cost-Effective ML: Proving that carefully engineered traditional ML approaches (TF-IDF + Linear SVM) can achieve near-deep-learning accuracy (97.25%) for domain classification while being significantly faster to train and cheaper to deploy than transformer-based alternatives.
3. Innovative Deployment: The use of prompt engineering to embed a trained classification taxonomy into an LLM inference call allows for domain classification within a serverless Deno environment without Python dependencies, reducing latency and infrastructure complexity.
4. Productivity Gains: The system reportedly reduces the time required for manual literature reviews by a factor of three (approximately 70% time savings) and automates the tedious task of citation formatting.

The authors maintain a modest stance, acknowledging limitations such as reliance on API metadata for citation accuracy, the current restriction to six broad domains (though expandable), and the lack of full-text PDF analysis in the current prototype. Future work is proposed to include hierarchical classification, PDF parsing via GROBID, and collaborative sharing features.