AgentCAT: An LLM Agent for Extracting and Analyzing Catalytic Reaction Data from Chemical Engineering Literature

This paper introduces AgentCAT, a large language model agent that overcomes data bottlenecks in chemical engineering by extracting catalytic reaction data from literature into a dependency-aware knowledge graph, enabling robust schema-governed extraction and natural language-based interactive analysis across approximately 800 peer-reviewed publications.

The Gist

Imagine you are a master chef trying to recreate a famous dish, but the recipe is hidden inside a library of 800 different cookbooks. Each book describes the dish slightly differently: one mentions the oven temperature, another talks about the specific type of flour, and a third explains why the dough rises, but none of them give you the full picture in one place. Worse, if you just grab a random sentence from each book, you might end up mixing up the ingredients from a cake with the instructions for a soup.

This is the exact problem chemical engineers face with catalytic reactions (the chemical "recipes" that turn raw materials into useful products like fuel or plastics). The data is scattered, complex, and deeply interconnected.

Enter AgentCAT, a new "digital research assistant" built by a team of scientists. Here is how it works, explained simply:

1. The Problem: The "Island" of Information

In the past, if a computer tried to read these chemistry papers, it would treat every sentence like an isolated island. It might see "Catalyst A" in one paragraph and "High Temperature" in another, but it wouldn't realize they belong to the same experiment.

• The Analogy: Imagine trying to understand a movie by reading only one random sentence from every page. You might know the hero is wearing a hat, but you'd have no idea if they are fighting a dragon or buying groceries. General AI (like standard chatbots) often makes this mistake, mixing up facts or inventing details that sound plausible but are wrong.

2. The Solution: AgentCAT's "Detective" Approach

AgentCAT isn't just a reader; it's a detective that solves a mystery. It doesn't just copy-paste; it understands the story of the experiment.

• The "Living Blueprint" (Progressive Schema):
  Instead of forcing the AI to fit into a rigid, pre-made form (like a boring tax form), AgentCAT starts with a rough sketch and keeps improving it. As it reads more papers, it learns, "Oh, this new type of experiment needs a new box to fill in." It evolves its own checklist as it goes, ensuring it never misses a crucial detail.

  • Metaphor: Think of it like a detective's whiteboard. At first, they have a few photos and notes. As they find new clues, they add more strings and photos, connecting them until the whole picture makes sense.

• The "Fact-Checker" Loop:
  AgentCAT works in teams. One agent grabs the raw text, another tries to organize it, and a third acts as a strict editor. If the editor finds a mistake (like mixing up a temperature from 1990 with a chemical from 2020), it sends the work back to be fixed immediately.

  • Metaphor: It's like a newsroom where a reporter writes a story, an editor checks the facts, and if something is wrong, the reporter has to go back and verify the source before the story is published.

• The "Web of Truth" (Knowledge Graph):
  Once AgentCAT extracts the data, it doesn't just put it in a spreadsheet. It builds a giant, interactive 3D web (a Knowledge Graph). In this web, every piece of data is connected. If you click on a specific chemical, the web shows you:

  • Who made it?
  • What machine was used?
  • What was the result?
  • What evidence proves it?
  • Metaphor: Instead of a list of phone numbers, imagine a giant spiderweb where every node is a person, and the strings show who knows whom, who works together, and who is related. You can trace a path from "Grandma" to "The President" in three clicks.

3. Why This Matters

Chemical engineering is the bridge between a cool lab experiment and a factory that can make millions of products. But that bridge has been blocked because the data is too messy to cross.

AgentCAT clears the road. It allows scientists to ask questions in plain English, like:

"Show me all the catalysts that work best for making plastic at low temperatures, and tell me which ones were tested in a pulse reactor."

The system instantly searches through 800 papers, connects the dots, and draws a visual map of the answer.

Summary

AgentCAT is a smart, self-correcting robot librarian that:

1. Reads complex chemistry papers.
2. Connects the dots between the "how," "why," and "what" of an experiment.
3. Builds a giant, interactive map of chemical knowledge.
4. Lets scientists ask questions in normal language to discover new ways to make cleaner energy and better materials.

It turns a mountain of confusing, scattered papers into a clear, navigable map for the future of science.

Technical Summary

1. Problem Statement

Chemical engineering faces a long-standing data bottleneck: while vast amounts of catalytic reaction data exist in peer-reviewed literature, they are unstructured, scattered across different sections (synthesis, characterization, mechanism, performance), and highly dependent on specific experimental contexts.

• The Core Challenge: Unlike pure chemistry which focuses on reaction mechanisms, chemical engineering prioritizes scalability and industrial applicability. Data is only interpretable when linked along the catalytic process (Synthesis $\to$ Structure $\to$ Performance).
• Limitations of Existing Tools:
  • General LLMs (e.g., Gemini, Grok): While good at semantic understanding, they fail at causal consistency (linking distant information), fine-grained alignment (matching specific experimental conditions to results), and domain logic (violating chemical constraints). They often hallucinate properties or break long-range dependencies.
  • Existing Extraction Systems: Domain-specific tools (e.g., SLM-MATRIX, Eunomia) lack the ability to handle the complex, heterogeneous dependency structures found in catalytic reaction data (e.g., linking synthesis descriptors to macroscopic outcomes via active sites).

2. Methodology: AgentCAT Architecture

AgentCAT is a multi-agent framework designed to extract, verify, and analyze catalytic data using a schema-governed, closed-loop, plan-then-execute approach. It consists of three progressive stages:

A. Adaptive Information Extraction

This stage addresses the complexity of unstructured text and the risk of hallucination.

1. Progressive Schema Evolution:
   • Instead of a static schema, AgentCAT uses a Human-in-the-Loop mechanism. A Planning Agent collaborates with researchers to create an initial DomainKeyElements schema.
   • A Schema-Evolution Agent iteratively processes stratified samples of papers to refine the schema (adding new entity types or properties) while ensuring backward compatibility. This allows the system to adapt to emerging research paradigms.
2. Two-Phase Evidence-Based Extraction:
   • Phase 1 (Candidate): An agent scans the document to extract verbatim text fragments as potential evidence for specific schema fields without interpretation.
   • Phase 2 (Resolve): The system processes these candidates alongside their original context to populate structured fields. This forces the model to ground outputs in explicit textual evidence, reducing hallucinations.
3. Review and Quality Verdict:
   • A Review Agent evaluates extracted data, assigning a verdict: PASS, MINOR_FIX (formatting), or MAJOR_ERROR.
   • If a MAJOR_ERROR is detected, a re-extraction loop is triggered. Error-reason hints are injected back into the prompt to guide the agent toward correction in subsequent iterations.

B. Knowledge Graph Construction

Extracted data is transformed into a Dependency-Aware Reaction-Network Knowledge Graph (stored in Neo4j).

• Structure: The graph links Catalysts/Active Sites $\to$ Synthesis-Derived Descriptors $\to$ Mechanistic Claims (with Evidence) $\to$ Macroscopic Outcomes.
• Dynamic Labeling: A Label Manager supports dynamic schema expansion for novel concepts while maintaining structural coherence.
• Entity Normalization: A Molecular Normalizer standardizes chemical formulas and nomenclature (e.g., unifying "propylene" and "propene").
• Traceability: Every node is linked to its source PDF identifier, ensuring verifiability.

C. General Querying and Graph Exploration

• General Querying Agent: Allows researchers to interact with the graph using Natural Language. The agent decomposes complex queries into a plan (sub-queries, synthesis, revision) and executes them against the Neo4j database.
• Visualization: Provides a visual interface to trace catalytic mechanisms across multiple papers, enabling cross-document analysis and discovery of non-obvious research directions.

3. Key Contributions

1. Formal Abstraction of the Task: The authors define catalytic data extraction as reconstructing a "partial picture" of the Synthesis-Structure-Performance (SSP) paradigm. They identify three specific failure modes of general LLMs in this domain:
   • Lost-in-the-middle causal linking.
   • Hallucination in fine-grained parameter alignment.
   • Lack of domain logic constraints (violating chemical validity).
2. Multi-Agent Pipeline: A novel architecture featuring progressive schema evolution and a two-phase evidence-grounded extraction (Candidate-then-Resolve) to ensure data fidelity.
3. Dependency-Aware Knowledge Graph: A specialized graph structure that preserves the coupling between synthesis, mechanism, and performance, unlike isolated tuple-based datasets.
4. Comprehensive Evaluation: A robust evaluation on a large-scale corpus demonstrating the system's ability to handle complex, real-world chemical engineering literature.

4. Results and Performance

The system was evaluated on ~800 peer-reviewed chemical engineering publications (Energy Catalysis domain).

• Schema Convergence: The schema evolution process showed rapid stabilization, with new items introduced in the first few rounds and minimal extensions thereafter, proving the system's adaptability.
• Extraction Quality (Expert Evaluation):
  • Three domain experts evaluated 20 randomly selected papers.
  • Accuracy and Readability were consistently high (scores ~28/30).
  • Completeness was robust, with minor variations.
• Reliability at Scale:
  • Across 4,398 extracted sections from 733 papers:
    • 82.0% achieved a PASS verdict.
    • 15.3% required MINOR_FIX.
    • Only 2.7% resulted in MAJOR_ERROR.
  • This low error rate validates the effectiveness of the review-driven re-extraction loop.
• Querying Performance:
  • Tested on 12 queries (Easy, Medium, Hard).
  • Overall Correctness: 86.67%.
  • Error Rates: 10% for Easy/Medium, 20% for Hard queries.
  • Latency: Execution time increases with query complexity (avg. 184s), dominated by the planning phase rather than database execution.

5. Significance

• Solving the Data Bottleneck: AgentCAT provides a scalable, automated solution to the scarcity of structured catalytic data, a critical barrier in chemical engineering.
• Bridging AI and Domain Science: By formalizing the extraction task with domain-specific constraints (SSP paradigm) and evidence grounding, it moves beyond generic LLM capabilities to create a reliable scientific tool.
• Enabling Cross-Paper Analysis: The knowledge graph allows researchers to query across thousands of papers, identifying patterns, optimizing catalyst designs, and accelerating the discovery of new reaction pathways that would be impossible to find via manual literature review.
• Community Impact: The paper offers a formal abstraction of the problem that can guide future AI research in scientific data extraction, moving the field from simple entity recognition to complex, causal reasoning.