🤖 AI

MACReD: A Multi-Agent Collaborative Reasoning Framework for Reaction Diagram Parsing

MACReD is a hierarchical multi-agent framework that leverages specialized agents and multigraph fusion to achieve state-of-the-art performance in parsing complex chemical reaction diagrams by effectively integrating visual perception with chemically consistent reasoning.

Original authors: Chuang Tang, Chenhao Lin, Yin Xu, Hao Wang, Jinrui Zhou, Xin Li, Mingjun Xiao, Enhong Chen

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chuang Tang, Chenhao Lin, Yin Xu, Hao Wang, Jinrui Zhou, Xin Li, Mingjun Xiao, Enhong Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Chemistry Comic Book" Puzzle

Imagine you are trying to read a comic book, but instead of words, the panels are filled with complex drawings of molecules, weird arrows, and tiny scribbles of text. To a human chemist, this is a clear story: "Mix these two ingredients (Reactants), add this heat or catalyst (Conditions), and you get this new thing (Products)."

But to a computer, this is a nightmare. The drawings are messy, the arrows curve in strange ways, and the text is often squeezed into tight corners. Previous computer programs tried to solve this by either:

Following rigid rules: Like a robot that only knows how to read a straight line. If the diagram is curved or messy, the robot gets confused and gives up.
Using a "Super-Brain" (AI): Large AI models are great at understanding general pictures, but when it comes to these specific chemistry puzzles, they often hallucinate. They might mix up which molecule is the start and which is the end, or miss the crucial "arrow" that tells the story direction. They struggle to keep the whole picture logically consistent.

The Solution: MACReD (The Chemistry Detective Squad)

The authors of this paper, MACReD, decided that one giant brain isn't the best way to solve this. Instead, they built a team of specialized detectives (a Multi-Agent Collaborative Framework) that work together in three distinct stages, like a well-oiled factory assembly line.

Think of MACReD as a high-tech newsroom trying to reconstruct a story from a pile of scattered photos and notes.

Stage 1: The Editor-in-Chief (Planning Layer)

Before anyone starts working, the Planning Agent looks at the messy diagram and the user's question. It acts like a smart editor who decides, "Okay, this is a simple one-step reaction, so we just need the molecule expert. But this one is a complex, multi-step tree, so we need the whole team!"

The Magic: It doesn't use a rigid script. It dynamically calls the right experts based on how complicated the picture is, saving time and energy.

Stage 2: The Specialized Detectives (Perception Layer)

Once the plan is set, three specific agents go to work, each with a unique superpower:

The Molecule Detective: This agent finds the drawings of the chemical structures. It's like a photo editor who cuts out the messy background and cleans up the image so the structure is clear. It then translates the drawing into a digital code (SMILES) that computers can read.
The Arrow Detective: Arrows in chemistry are crucial—they tell you the direction of the reaction. But they can be curved, double-headed, or hidden behind text. This agent is trained to spot these tricky arrows and figure out if they mean "go forward," "go back and forth," or "resonate."
The Text Detective: Chemical diagrams are full of tiny notes like "heat," "solvent," or "10 minutes." This agent reads the tiny, messy handwriting and normalizes it (e.g., turning "FeCl3" and "ferric chloride" into the same standard term).

Stage 3: The Chief Investigator (Reasoning Layer)

Now, the team has a pile of clean data: molecules, arrows, and text. But they might still be disconnected. The Reasoning Layer is the Chief Investigator who brings it all together.

The "Multigraph Fusion": Imagine the Chief Investigator has three different maps of the same crime scene:
1. A Spatial Map (where things are located relative to each other).
2. A Chemistry Map (does this reaction make chemical sense? Do the atoms balance?).
3. A Guess Map (what did the AI "Super-Brain" think the story was?).
The Chief Investigator overlays these maps. If the Spatial Map says "A is next to B," but the Chemistry Map says "A and B can't react," the Chief Investigator uses logic to figure out the truth. They prune away the wrong guesses and assemble the final, chemically correct story.

The Results: Why It Wins

The researchers tested MACReD on a famous benchmark called RxnScribe, which is full of these tricky chemistry diagrams.

The Competition: Old rule-based systems were terrible (scoring near zero). Even the most advanced "Super-Brain" AI models (like GPT-4 or specialized chemistry AIs) struggled, often getting the structure wrong or missing the logic.
The Winner: MACReD crushed the competition. It achieved the highest scores ever recorded for this task.
- It was particularly good at Hard Matches (getting the entire structure, including conditions, perfectly right) and Soft Matches (getting the core molecular relationships right).
- It handled complex layouts (like tree-shaped reactions or multi-step graphs) much better than anyone else.

The Bottom Line

The paper claims that by breaking the problem down into a team of specialists who communicate and cross-check each other, rather than relying on one giant AI to do everything at once, we can finally teach computers to "read" complex chemistry diagrams with human-like accuracy. It's not just about seeing the picture; it's about understanding the logic behind the drawing.

Technical Summary: MACReD – A Multi-Agent Collaborative Reasoning Framework for Reaction Diagram Parsing

1. Problem Definition

Parsing chemical reaction diagrams from scientific literature is a critical yet challenging task in AI for Science (AI4Sci). While machine learning drives advancements in retrosynthesis and reaction prediction, these models rely on high-quality, machine-readable datasets that are currently "locked" within complex, multimodal graphics.

Reaction diagrams present three fundamental challenges that existing Vision Language Models (VLMs) and rule-based systems struggle to address:

Visual and Structural Complexity: Diagrams feature high-density interleaving of heterogeneous entities (skeletal structures, varied arrow geometries, text). Spatial proximity does not always correlate with chemical connectivity, and layouts range from simple single-step transformations to complex, multi-step, and branched pathways.
Limited Generalization: Current learning-based methods often rely on specific dataset patterns, failing to generalize across the "stylistic diversity" of scientific literature (e.g., varying arrow head styles, novel diagrammatic formats).
Disjoint Perception and Reasoning: Most existing pipelines separate visual perception from chemical reasoning. This unidirectional design allows perception errors to propagate without correction, undermining global chemical consistency and making it difficult to resolve ambiguities or infer implicit reaction semantics.

2. Methodology: MACReD Framework

To address these challenges, the authors propose MACReD (Multi-Agent Collaborative Reasoning for Reaction Diagram Parsing), a hierarchical framework that tightly couples visual perception and chemical reasoning. Rather than treating parsing as a monolithic end-to-end task, MACReD decomposes the process into three coordinated layers: Planning, Perception, and Reasoning.

2.1 Planning Layer

The Planning Layer acts as a top-level controller that dynamically orchestrates specialized agents based on the input query and diagram features.

Adaptive Routing: Instead of fixed pipelines, the Planning Agent formulates decision-making as a context-conditioned agent routing problem. It analyzes the visual content and task query to predict an ordered execution plan ( $\pi$ ), selecting and sequencing specific agents (e.g., invoking all agents for complex multi-step pathways vs. a minimal subset for simple tasks).
Goal: To enable flexible, scalable reasoning over heterogeneous reaction diagrams while improving computational efficiency.

2.2 Perception Layer

This layer serves as the bridge between raw image pixels and symbolic chemical representations, decomposing the task into specialized agents:

Molecule Agents: Utilize a decoupled two-stage pipeline. A CNN-based module performs object-level localization, followed by a Molecule-Recognition Agent that converts cropped regions into SMILES or molecular graphs using tools like MolScribe and RDKit. VLMs are subsequently used to correct recognition errors via chemical semantic reasoning.
Arrow Agent: Detects and classifies reaction arrows using Oriented Bounding Box (OBB) detection trained on fine-grained geometric annotations. It classifies arrows into functional categories (forward, reversible, resonance) and uses VLMs to cross-check classifications against visual semantics.
Text Agents: Extract and normalize textual annotations (reagents, conditions, yields). This involves detection, transcription (via PaddleOCR), and normalization to map synonymous expressions to structured chemical entities, handling domain-specific symbols and OCR ambiguities.

2.3 Reasoning Layer

The Reasoning Layer integrates perception outputs into a unified representation to perform global reaction reconstruction. It constructs and fuses three intermediate graphs:

Spatial-Semantic Graph ( $G_{space}$ ): A lightweight Graph Convolutional Network (GCN) propagates local spatial cues and visual-semantic coherence across neighboring entities to score relations beyond simple pairwise heuristics.
Chemistry-Aware Graph ( $G_{chem}$ ): Incorporates domain constraints by quantifying chemical affinity using molecular fingerprint similarity (Tanimoto) and formal charge consistency. Edges with low chemical plausibility are pruned, acting as soft inductive biases.
VLM-Induced Reaction Entities Graph ( $G_{VLM}$ ): The VLM generates preliminary reaction hypotheses by processing localized subgraphs (to avoid conflating unrelated steps in dense diagrams). These provide coarse, semantically informed reaction relations.

Multigraph Fusion and Pruning: These three graphs are consolidated into a unified reasoning graph ( $G_{fuse}$ ) via a weighted aggregation of confidence scores from spatial, chemical, and VLM sources. Low-confidence edges are pruned.
Global Reaction Inference: The system enumerates candidate configurations over connected components of the pruned graph, selecting the optimal reaction structure ( $R^*$ ) that maximizes accumulated structural support.

3. Key Contributions

The paper outlines three primary contributions:

MACReD Framework: A hierarchical multi-agent system that coordinates specialized agents (Planning, Perception, Reasoning) to jointly address visual complexity, generalization limits, and the disconnect between perception and chemical reasoning.
Unified Graph-Based Fusion Mechanism: A novel approach that synthesizes spatial structures, chemistry-aware constraints, and VLM-induced entities. This allows for the robust reconstruction of intricate, multi-step, and non-linear reaction pathways while enforcing chemical consistency.
State-of-the-Art Performance: Extensive experiments demonstrate that MACReD achieves superior performance and generalization across diverse diagram layouts, including challenging multi-step and graph-structured pathways.

4. Experimental Results

The framework was evaluated on the RxnScribe benchmark, a comprehensive dataset for reaction diagram understanding.

4.1 Evaluation Metrics

Performance was measured using Hard Match (requiring exact matching of reactants, conditions, and products) and Soft Match (focusing on molecular entities, ignoring text variations). Metrics included Precision, Recall, and F1-score.

4.2 Main Results

Overall Performance: MACReD achieved state-of-the-art results with an F1 score of 75.2% (Hard Match) and 84.6% (Soft Match).
Comparison: This outperforms the previous best baseline, RxnScribe (69.1% / 80.0%), and significantly surpasses general-purpose VLMs (e.g., GPT-4o, Gemini-2.5-Pro) and specialized chemical models (e.g., ChemVLM, Intern-S1-mini), many of which scored near zero on Hard Match.
Layout Robustness: MACReD demonstrated consistent performance across four layout types: Single Line, Multiple Line, Tree, and Graph. While performance naturally degraded with increasing complexity (Graph layouts being the most challenging), the framework maintained competitive accuracy, indicating effective handling of non-linear dependencies.

4.3 Ablation Studies

Ablation studies confirmed the necessity of each component:

Arrow Agent: Removal caused the most severe drop (Hard F1: -39.8%), highlighting the critical role of arrow understanding in defining reaction directionality and boundaries.
Molecule Recognition: Exclusion led to significant drops (Hard F1: -41.8%), confirming that accurate molecular identity is a prerequisite for chemical reasoning.
Multigraph Fusion: Removing this module resulted in a substantial performance decrease (Hard F1: -24.8%), validating the need to jointly integrate heterogeneous evidence sources.
Text Recognition: Removal reduced performance, particularly in Hard Match, showing that textual cues are vital for correct reaction-level grouping.

5. Significance and Claims

The paper claims that MACReD represents a significant step forward in scientific document understanding by moving beyond simple multimodal generation to structured, chemically consistent reasoning.

Beyond Monolithic Models: The results suggest that the challenge of reaction diagram parsing exceeds the capability limits of single monolithic VLMs, even large-scale or domain-specific ones. The task requires auxiliary mechanisms and system-level solutions, such as multi-agent collaboration and explicit structural constraints.
Bridging Perception and Reasoning: By tightly coupling visual perception with chemistry-aware reasoning through a hierarchical graph fusion mechanism, MACReD effectively resolves ambiguities in complex diagrams where spatial proximity does not imply chemical relevance.
Generalizability: The framework's ability to handle diverse layouts and drawing conventions without catastrophic performance degradation highlights its potential for extracting machine-readable chemical knowledge from the vast, unstructured repository of scientific literature.

The authors conclude that their approach validates the effectiveness of multi-agent cooperation and context-aware inference for achieving accurate and chemically coherent reaction reconstruction, laying the groundwork for future advancements in automated synthesis planning and materials discovery.