💬 NLP

MultiVis-Agent: A Multi-Agent Framework with Logic Rules for Reliable and Comprehensive Cross-Modal Data Visualization

This paper introduces MultiVis-Agent, a logic rule-enhanced multi-agent framework that ensures reliable and comprehensive cross-modal data visualization across complex scenarios, achieving superior performance and near-perfect task completion rates compared to existing baselines.

Original authors: Jinwei Lu, Yuanfeng Song, Chen Zhang, Raymond Chi-Wing Wong

Published 2026-01-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jinwei Lu, Yuanfeng Song, Chen Zhang, Raymond Chi-Wing Wong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a custom piece of furniture based on a sketch, a list of materials, and a few specific instructions. You ask a very smart, but sometimes scattered, robot assistant to do it for you.

The Problem: The "Scattered Genius" Robot
Current AI tools for making data charts (visualizations) are like that scattered robot. They are great at following simple instructions like "make a bar chart of sales." But real life is messy. Sometimes you need to:

Show them a picture of a chart you like and say, "Make it look like this, but with my new data."
Give them a snippet of code and say, "Use this style, but fix the colors."
Say, "Actually, change the title and make the bars blue," after they've already made the first draft.

When you ask current AI systems to do these complex, multi-step tasks, they often get confused. They might get stuck in an endless loop of trying to fix an error, they might ignore your reference picture, or they might just crash completely. It's like the robot trying to build the chair, realizing it picked the wrong wood, panicking, and then trying to build the chair again with the wrong wood, over and over again.

The Solution: The "Logic-Guided Construction Crew"
The authors of this paper, MultiVis-Agent, propose a new way to build these charts. Instead of one lone robot trying to do everything, they use a team of specialized workers led by a strict but helpful Project Manager.

Here is how their system works, using simple analogies:

1. The Team of Specialists

Instead of one AI trying to be everything, they have three specific roles:

The Data Detective: This agent looks at your database (the raw numbers) and figures out exactly what data to pull.
The Chart Builder: This agent takes the data and the instructions to actually write the code that draws the chart.
The Quality Inspector: This agent looks at the finished chart to see if it matches your request. If it's wrong, they send it back to the Builder with specific notes on what to fix.

2. The Project Manager (The Coordinator)

This is the brain of the operation. It listens to your request, decides which specialist needs to work, and keeps the whole team on track. It makes sure the Detective talks to the Builder, and the Inspector talks to the Builder.

3. The "Safety Rules" (The Magic Ingredient)

This is the most important part of the paper. The authors realized that even a team of smart agents can get into trouble if they aren't watched. So, they added a four-layer set of "Logic Rules" that act like a safety harness for the team.

Think of these rules like the safety protocols on a construction site:

Rule 1: Know the Job. Before starting, the system checks: "Are we building a new chair, or just fixing an old one?" This prevents confusion.
Rule 2: Check the Tools. Before the Builder uses a tool (like a saw or a drill), the rules check: "Is this tool safe to use with these materials?" This stops the robot from trying to cut wood with a hammer.
Rule 3: Fix Mistakes Gracefully. If the Inspector finds a mistake, the rules say: "Here is exactly how to fix it, and you only get 10 tries." This stops the robot from getting stuck in an infinite loop of panic.
Rule 4: Stop When Done. The rules guarantee that the project will end. If the robot can't fix the chair after 10 tries, it stops and reports the issue, rather than running forever.

What They Built to Test It

To prove their system works, they didn't just guess. They built a giant test kitchen called MultiVis-Bench.

It contains over 1,000 different challenges.
Some challenges are simple ("Make a chart").
Some are complex ("Make a chart that looks like this picture but uses this new data").
Some require fixing ("The chart is wrong, change the title").

The Results: A Much Safer, Smarter System

When they tested their "Logic-Guided Crew" against other AI systems:

Reliability: The new system finished the job 99.6% of the time. The old systems often got stuck or crashed (finishing only about 74% of the time).
Success Rate: The code they wrote actually worked 94.6% of the time. The old systems only succeeded about 65% of the time.
Quality: The charts looked better and matched the user's complex requests much more often (scoring 75.6% vs. roughly 60% for others).

The Bottom Line

The paper argues that to make AI useful for real-world data work, we can't just rely on the AI being "smart." We need to wrap that intelligence in a safety net of mathematical rules.

By combining the creativity of AI with the strict discipline of logic rules, MultiVis-Agent creates a system that is flexible enough to handle complex, multi-step requests but reliable enough that you won't have to worry about it crashing or getting stuck in an endless loop. It turns a chaotic, unpredictable robot into a dependable construction crew.

Technical Summary: MultiVis-Agent

1. Problem Statement

Real-world data visualization tasks extend far beyond simple text-to-chart generation. They frequently involve complex, multi-modal requirements, including the need to adapt reference styles from images, modify existing code snippets, and engage in iterative refinement based on user feedback. Current automated visualization systems suffer from three fundamental limitations:

Single-Modality Input: Most systems (e.g., Chat2VIS) cannot process reference images or code examples, limiting them to natural language queries.
One-Shot Generation: Approaches like Prompt4Vis lack mechanisms for iterative refinement, failing to accommodate the dynamic nature of analytical workflows.
Rigid Workflows: Systems like nvAgent utilize fixed pipelines that cannot adapt to diverse task types or handle complex multi-modal inputs.

Furthermore, existing LLM-based agent systems face a reliability crisis. They are prone to catastrophic failures, infinite error loops, uncontrolled error propagation, and parameter boundary violations. Traditional rule-based systems offer reliability but lack the flexibility to handle complex, open-ended visualization tasks. There is a critical gap for a system that combines the generative flexibility of Large Language Models (LLMs) with the robustness and reliability of formal logic constraints.

2. Methodology: MultiVis-Agent

To address these challenges, the authors propose MultiVis-Agent, a logic rule-enhanced multi-agent framework designed for reliable, multi-modal, and multi-scenario visualization generation.

2.1 Task Formulation (MultiVis)

The authors formalize the MultiVis task, extending traditional Text-to-Visualization into four distinct scenarios:

Basic Generation (BG): Text query + Database $\rightarrow$ Visualization.
Image-Referenced Generation (IRG): Text query + Database + Reference Image $\rightarrow$ Visualization (matching style/layout).
Code-Referenced Generation (CRG): Text query + Database + Reference Code $\rightarrow$ Visualization (adapting implementation patterns).
Iterative Refinement (IR): Text query + Database + Existing Code $\rightarrow$ Refined Visualization.

2.2 System Architecture

MultiVis-Agent employs a centralized multi-agent architecture orchestrated by a Coordinator Agent. The system decomposes tasks into specialized agents:

Database & Query (DQ) Agent: Handles data exploration and SQL generation using tools like list_tables, get_table, and execute_sql.
Visualization Implementation (Vis) Agent: Generates and modifies executable Python code (specifically Altair) using tools like get_code_example and exec_altair_code.
Validation & Evaluation (VE) Agent: Assesses technical correctness and perceptual effectiveness, providing structured feedback for refinement.

2.3 Core Innovation: Four-Layer Logic Rule Framework

The distinguishing feature of MultiVis-Agent is a four-layer logic rule framework that provides mathematical guarantees for system reliability while preserving LLM flexibility. Unlike traditional rule-based systems that rigidly dictate behavior, these rules act as mathematical constraints guiding LLM reasoning.

The four layers are:

Coordination Rules (CR): Govern the Coordinator Agent. They include deterministic task classification (prioritizing inputs like existing code over images), tool prerequisite validation, and evaluation response decision-making.
Tool Execution Rules (TE): Enforce parameter safety (boundary constraints), code execution environment standardization, and reference material processing.
Error Handling Rules (EH): Provide systematic error recovery, including tool call parsing error classification, code execution error recovery strategies, and image processing validation.
ReAct Control Rules (RC): Manage iteration control with guaranteed termination (e.g., a maximum iteration bound $T_{max}=10$ ) and response format validation.

2.4 Theoretical Guarantees

The framework is underpinned by four formal theorems proving:

Parameter Safety: All tool executions satisfy safety constraints or result in graceful failure.
Bounded Error Recovery: Error recovery processes terminate within a finite number of steps.
Guaranteed Termination: All execution sequences terminate within a maximum of 10 iterations.
System Reliability: The combination of safety, recoverability, and termination guarantees overall system reliability.

3. Key Contributions

Logic Rule-Enhanced Multi-Agent Architecture: A novel framework that achieves task completion rates of 98.68–100% and code execution success rates of 94.56–97.10%, significantly outperforming the same framework without logic rules.
MultiVis Task Formulation: A comprehensive definition of visualization generation covering four scenarios (BG, IRG, CRG, IR) that reflect real-world multi-modal requirements.
MultiVis-Bench: A new benchmark dataset containing over 1,200 cases with multi-modal inputs (text, database, images, code) and executable Python code outputs, designed to evaluate the full lifecycle of visualization generation.
Superior Empirical Performance: Extensive experiments demonstrating substantial improvements in visualization quality and reliability across all scenarios compared to baselines.

4. Experimental Results

The authors evaluated MultiVis-Agent using MultiVis-Bench against baselines including Instructing LLM, LLM Workflow, and nvAgent (adapted for multi-modal inputs).

Overall Performance: MultiVis-Agent (using Gemini-2.0-Flash) achieved a visualization score of 75.63% on challenging Image-Referenced Generation tasks, significantly outperforming baselines (LLM Workflow: 62.79%, Instructing LLM: 57.54%).
Reliability Metrics: The inclusion of logic rules improved task completion rates from ~74–92% to 98.68–100% and code execution success rates from ~63–83% to 94.56–97.10%.
Ablation Studies: Removing the logic rule framework caused significant performance drops (e.g., a 31.70% drop in Image-Referenced Generation), confirming the critical role of formal constraints. Removing the Database Agent caused severe degradation in all generation scenarios, while removing the Validation Agent led to consistent quality drops.
Iterative Refinement: While a streamlined "only GEN" approach performed slightly better in simple refinement tasks (where context is complete), MultiVis-Agent excelled in complex generation scenarios (A-C) requiring database querying and cross-modal understanding.

5. Significance and Claims

The paper claims that MultiVis-Agent establishes a new paradigm for robust agent system design. Its primary significance lies in solving the system robustness crisis inherent in traditional LLM-based agents. By introducing mathematical constraints that guide rather than replace LLM reasoning, the framework achieves a balance between the creativity required for complex multi-modal tasks and the reliability necessary for production deployment.

The authors assert that their approach successfully addresses the limitations of current systems by:

Enabling multi-modal inputs (images, code, text) and iterative refinement.
Providing formal guarantees for error recovery and termination, preventing catastrophic failures and infinite loops.
Offering a comprehensive benchmark (MultiVis-Bench) that moves beyond intermediate representations (like VQL) to evaluate directly executable code.

The work positions MultiVis-Agent as a foundational component for more complex interface synthesis systems, demonstrating that logic-enhanced agents can reliably handle the dynamic and complex nature of real-world data visualization workflows.