LiRA: A Multi-Agent Framework for Reliable and Readable Literature Review Generation

1. Problem Statement

The rapid proliferation of scientific publications has made it increasingly difficult to conduct comprehensive and up-to-date Systematic Literature Reviews (SLRs). While existing automation efforts have largely focused on the retrieval and screening phases of SLRs, the writing phase remains under-explored. Current automated writing solutions suffer from two critical issues:

• Readability and Structure: Generated reviews often lack the coherence, logical flow, and structural organization of human-written reviews.
• Factual Accuracy (Hallucination): Models frequently generate plausible-sounding but non-existent citations or misattribute findings, undermining trust in AI-generated scientific content.

There is a lack of "out-of-the-box" solutions that can generate high-quality, citation-faithful reviews without requiring domain-specific pre-training or fine-tuning.

2. Methodology: The LiRA Framework

LiRA (Literature Review Agents) is a multi-agent collaborative workflow built on Large Language Models (LLMs) that emulates the human literature review process. It utilizes LangGraph for orchestration and operates without task-specific fine-tuning. The framework decomposes the writing process into specialized, interacting agents:

• Outline Drafter Agent:
  • Function: Analyzes up to 50 reference abstracts/titles to construct a coherent structural outline (main sections and subsections).
  • Design: Explicitly drafts the structure before writing to avoid the incoherence of single-pass generation. It includes default sections (Introduction, Conclusion) and suggests supporting papers for each section.
• Subsection Writer Agent:
  • Function: Writes individual sections in parallel, conditioned on the outline and a retrieved subset of relevant references (up to 25% of the pool, min 3, max 150).
  • Design: Uses FAISS for retrieval based on section descriptions. It generates ~1,000 words per subsection, enabling long-form synthesis while staying within context limits. The title, abstract, and conclusion are written after the body to mimic human practices.
• Editor Agent:
  • Function: Refines the assembled draft for presentation, style, transitions, and fluency.
  • Constraint: Crucially, this agent does not alter factual content, preserving the integrity of the generated information. It also handles context window overflow by detecting abrupt cuts and prompting the model to continue.
• Reviewer Agent:
  • Function: Evaluates intermediate outputs (outlines, sections) against systematic review guidelines (e.g., completeness, transparency, clarity).
  • Mechanism: If quality thresholds are not met, it provides structured feedback and triggers regeneration. This loop runs up to 3 times to balance refinement with efficiency.
• Citation Grounding Strategy:
  • To mitigate hallucination, agents cite sources using full article titles as semantic anchors during generation, rather than abstract placeholders.
  • Post-generation, these are converted to standard numbered references. Any hallucinated titles are redacted during evaluation to ensure fair comparison.

Implementation Details:

• Architecture: Built on LangGraph with individual memory for each agent to store feedback (similar to the Reflexion framework).
• Communication: Agents return structured documents to shared message pools to optimize information retrieval.
• Prompting: Uses Zero-Shot Chain-of-Thought (CoT) for most agents to improve reasoning, except for the Researcher and Editor.

3. Key Contributions

1. First Agentic LLM Workflow for SLR Writing: LiRA is the first framework to explicitly emulate the human writing process (planning, drafting, editing, reviewing) for automated literature reviews.
2. Integration of Guidelines: It incorporates formally defined guidelines from SLR literature (e.g., analyzing papers before writing) into the agentic workflow, bridging theory and application.
3. State-of-the-Art Baselines: The authors established robust baselines (AutoSurvey, MASS-Survey, Direct Prompting) using the same underlying LLM (gpt-4o-mini) to ensure fair comparison across multiple settings.
4. Focus on Reliability: The framework specifically targets the reduction of citation hallucinations and the improvement of readability, addressing the primary barriers to AI adoption in scientific writing.

4. Experimental Results

LiRA was evaluated on two datasets: SciReviewGen (Computer Science, 125 samples) and a proprietary ScienceDirect dataset (23 domains, 125 samples). It was compared against Direct Prompting (DP), MASS-Survey (MASS), and AutoSurvey (AS).

• Similarity to Human Reviews:
  • LiRA achieved the highest ROUGE-L scores, indicating superior lexical alignment with human-written reviews.
  • While AutoSurvey had slightly higher recall metrics (hsr, her, aer), this was attributed to verbosity (AutoSurvey generated ~50k tokens vs. LiRA's ~22k). LiRA demonstrated the ability to generate concise yet information-dense reviews.
• Writing Quality:
  • Evaluated via Prometheus 2 (LLM evaluator) and Subject Matter Experts (SMEs).
  • LiRA outperformed all baselines in Structure and Relevance.
  • SMEs on ScienceDirect strongly preferred LiRA for its structural coherence, while AutoSurvey was only marginally better in "Coverage" due to its excessive length.
• Citation Quality (CQF1):
  • LiRA achieved the highest Citation Quality F1-Scores (0.76 on SciReviewGen, 0.73 on ScienceDirect), significantly outperforming AutoSurvey (≤0.63).
  • This confirms the effectiveness of the citation-grounding design in reducing hallucinations and ensuring claims are supported by provided references.
• Robustness & Real-World Deployment:
  • Model Variation: Replacing the reviewer agent with a different model (Gemma 3) had negligible impact on performance, suggesting the pipeline is robust to model changes.
  • Retrieval: When tested with retrieved references (simulating a real-world scenario where gold references are unavailable), LiRA maintained performance comparable to the baseline, with only minor statistically significant drops in specific metrics.

5. Significance and Future Work

• Trustworthy Automation: LiRA demonstrates that agentic workflows can produce reliable, readable, and factually grounded scientific writing without domain-specific tuning, addressing a critical gap in AI-assisted research.
• Practical Usability: The framework is "out-of-the-box," requiring no fine-tuning, making it immediately applicable to diverse scientific fields.
• Limitations: The current results rely on gpt-4o-mini, which limits reproducibility due to non-deterministic outputs. The authors suggest using seedable models in future work.
• Future Directions:
  • Creation of open-source datasets for broader evaluation.
  • Expansion of the pipeline to include earlier SLR stages, such as defining search criteria and screening for bias in randomized trials (integrating Cochrane Handbook guidelines).

In conclusion, LiRA represents a significant step forward in automating scientific writing, proving that multi-agent collaboration can effectively balance the competing demands of comprehensiveness, readability, and factual accuracy in literature reviews.