Anagent For Enhancing Scientific Table & Figure Analysis

1. Problem Statement

Scientific research relies heavily on the accurate interpretation of complex multimodal data, specifically tables and figures, which often contain critical information that text alone cannot convey. Current Artificial Intelligence (AI) systems, including Multimodal Large Language Models (MLLMs), struggle to perform high-quality scientific analysis due to several fundamental challenges:

• Heterogeneity: Scientific literature varies widely in authoring formats (LaTeX, XML), rendering formats (PDF, HTML), and dissemination platforms (arXiv, PubMed).
• Complexity: Analysis requires integrating evidence from diverse sources, handling long contexts, and applying domain-specific knowledge.
• Reasoning Depth: Tasks range from shallow summarization to deep inferential reasoning, requiring the synthesis of multimodal data with specialized terminology.
• Error Propagation: Mistakes in structural parsing, numerical extraction, or contextual interpretation can cascade into factual hallucinations and incorrect conclusions.

Existing benchmarks primarily focus on narrow tasks (e.g., QA, caption generation) and fail to capture the full spectrum of scientific analysis writing, which involves varying depths of reasoning, multi-modal integration, and long-context understanding.

2. Methodology

The authors propose a two-pronged approach: a new benchmark (ANABENCH) and a multi-agent framework (ANAGENT).

A. ANABENCH: A Large-Scale Benchmark

To quantify the challenges, the authors introduced ANABENCH, a dataset containing 63,178 instances spanning 9 scientific domains and 170 fine-grained disciplines.

• Complexity Dimensions: The benchmark systematically categorizes tasks along seven dimensions:
  1. Data Type: Table, Figure, or Both.
  2. Domain: 9 broad domains (e.g., CS, Biology, Physics).
  3. Format: LaTeX vs. XML.
  4. Source: General papers vs. Reviews/Surveys.
  5. Analysis Width: Self-contained, Internal, External, or Mixed references.
  6. Analysis Depth: Shallow (summarization) vs. In-depth (inferential).
  7. Objective: Methodology-focused vs. Experiment-focused.
• Construction: Built via a four-stage pipeline (Source Collection, Data Extraction, Instance Construction, and MLLM-assisted Classification) with multi-level filtering to ensure high data quality and real-world distribution.

B. ANAGENT: A Multi-Agent Framework

Inspired by the human scientific workflow, ANAGENT decomposes the analysis task into four collaborative specialized agents:

1. PLANNER: Decomposes the complex input into actionable subtasks, defining the scope (width), depth, and objectives of the analysis.
2. EXPERT: Executes targeted tool calls (using 5 toolkits with 16 tools, e.g., document parsers, search engines, OCR) to retrieve task-specific knowledge and contextual information.
3. SOLVER: Synthesizes the input data, retrieved knowledge, and planning guidance to generate a coherent, scientifically grounded analysis draft.
4. CRITIC: Performs iterative refinement by evaluating the SOLVER's output against a five-dimensional protocol (Content Accuracy, Analytical Completeness, Format Correctness, Clarity & Coherence, Reliability & Faithfulness) and providing feedback for improvement.

C. Optimization Strategies

• Test-Time Optimization: Utilizes Few-Shot Learning (providing exemplars) to enhance agent adaptability without retraining.
• Modular Training: Instead of end-to-end training, the authors employ Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) using Group Relative Policy Optimization (GRPO).
  • Each agent is trained on specialized datasets tailored to its specific role (e.g., PLANNER learns strategy selection; EXPERT learns tool invocation; SOLVER learns generation; CRITIC learns evaluation).
  • This preserves role specialization while enhancing global collaboration.

3. Key Contributions

1. ANABENCH: The first large-scale benchmark specifically designed for scientific table & figure analysis, covering 63k+ instances across 7 complexity dimensions, addressing the lack of comprehensive evaluation tools in this domain.
2. ANAGENT: A novel multi-agent framework that mimics human scientific reasoning through task decomposition, tool-augmented retrieval, and iterative critique, significantly outperforming monolithic models.
3. Modular Training Paradigm: Demonstrates that training agents individually with specialized RL rewards, rather than end-to-end, leads to superior coordination and performance, particularly for long-context and complex reasoning tasks.
4. Comprehensive Evaluation: Introduces specialized metrics and a five-dimensional evaluation protocol (validated by both rule-based metrics and human experts) to rigorously assess scientific analysis quality.

4. Experimental Results

The framework was evaluated across 170 subdomains using various MLLM backbones (e.g., Qwen, InternVL, GPT-4.1-mini, Gemini-2.5-Flash).

• Training-Free Performance: ANAGENT achieved substantial improvements over baselines without fine-tuning, with relative gains of ↑13.43% in overall accuracy (SAVG) in one-shot settings.
• Fine-Tuned Performance: With modular SFT + RL training, ANAGENT achieved massive improvements, with relative gains of ↑42.12% compared to baselines.
• Ablation Studies:
  • Modular vs. End-to-End: Modular training outperformed end-to-end training by ↑33.10%, proving that preserving role specialization is critical.
  • Agent Roles: Removing the PLANNER or CRITIC led to significant performance drops, highlighting the necessity of planning and reflection.
  • Tool Utilization: Fine-tuning significantly improved the success rate and strategic selection of tools (e.g., abstract collectors, search engines).
• Error Reduction: ANAGENT reduced error rates across all 7 identified failure patterns (e.g., hallucinations, domain-specific errors, long-context misunderstandings) by >3% to >32% relative to baselines.

5. Significance and Impact

• Advancing AI Co-Scientists: The work moves AI from simple data extraction to autonomous scientific reasoning, enabling systems to act as "co-scientists" that can interpret complex evidence, synthesize findings, and write rigorous analysis.
• Benchmarking Standard: ANABENCH sets a new standard for evaluating multimodal scientific reasoning, moving beyond simple QA to complex, context-aware analysis.
• Methodological Insight: The paper provides strong evidence that task-oriented decomposition and modular training are essential for handling the heterogeneity and complexity of scientific literature, offering a blueprint for future agentic systems in specialized domains.
• Practical Application: The framework's ability to handle diverse formats (LaTeX/XML) and domains makes it highly applicable for automating literature reviews, hypothesis generation, and scientific communication.

In conclusion, ANAGENT represents a significant leap forward in AI's ability to understand and generate scientific content, leveraging a structured multi-agent architecture and specialized training to overcome the limitations of current monolithic models in complex, multimodal scientific environments.