ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

1. Problem Statement

Multimodal Large Language Models (MLLMs) are increasingly used to automate chart generation from data tables. While this democratizes data visualization, it introduces a significant security risk: malicious actors can exploit MLLMs to generate misleading charts at scale.

Unlike traditional data poisoning (which alters the underlying data), this attack vector involves design-level manipulation. By injecting "misleaders" (specific visualization techniques that violate design principles) into the chart generation process, an attacker can create charts that are data-consistent (the raw numbers remain correct) but visually deceptive, leading to incorrect interpretations by both human readers and other AI systems. The paper addresses the lack of systematic quantification regarding how effectively MLLMs can be "jailbroken" to produce such deceptive visualizations and the subsequent impact on downstream tasks like Chart Question Answering (ChartQA).

2. Methodology

The authors propose ChartAttack, a novel framework designed to automatically generate misleading charts and evaluate their impact. The framework consists of two primary modules and a supporting dataset.

A. The ChartAttack Framework

The framework operates on chart annotations (JSON files containing data and formatting specifications) rather than raw images, allowing for precise, reproducible manipulation.

1. Demonstration Selection Module:
   • Uses a fine-tuned SBERT model trained with Multiple Negative Ranking Loss (MNR) to retrieve relevant few-shot examples.
   • The retriever matches input chart annotations and questions with training examples that share the same set of misleaders, ensuring the generator learns context-appropriate deception techniques.
2. Misleader-Generator Module:
   • Utilizes code-based, instruction-tuned MLLMs (e.g., Qwen-Coder, DeepSeek-Coder) to modify chart JSON annotations.
   • Process: The model follows a structured reasoning process: (i) select compatible misleaders, (ii) apply minimal modifications to the JSON to implement the technique without altering the underlying data, and (iii) generate a plausible but incorrect answer.
   • Output: A list of modified JSON snippets and the induced misleading answers.

B. The AttackViz Dataset

To support the framework, the authors created AttackViz, a multi-label ChartQA dataset.

• Source: Derived from PlotQA, ChartQA, and ChartX, focusing on bar (horizontal/vertical) and line charts.
• Construction:
  • Rule-based Generation: Initially, 11 specific misleaders (e.g., inverted axes, 3D effects, truncated axes, misrepresentation) were applied using a rule-based Python system to create "ground truth" misleading variants.
  • Filtering: Instances were filtered to ensure the correct answer remained unchanged by the misleader, but the visual representation would lead to a different conclusion.
  • Annotation: Each instance includes the original chart, the modified (misleading) chart, the question, the correct answer, and the specific misleader applied.
• Scale: Covers three datasets (PlotQA, ChartQA, ChartX) with extensive coverage of 11 misleader types.

3. Key Contributions

1. ChartAttack Framework: The first automated framework to systematically apply design-level misleaders to chart annotations to induce targeted misinterpretations, effectively treating chart generation as a jailbreaking problem.
2. AttackViz Dataset: A comprehensive, multi-label dataset containing paired clean and misleading charts with structured annotations, enabling rigorous evaluation of deception in both AI and human contexts.
3. Systematic Evaluation: The first study to quantify the vulnerability of MLLMs to malicious prompting for chart generation, evaluating both in-domain (same distribution) and cross-domain (different datasets) generalization.
4. Human-in-the-Loop Validation: A pilot study demonstrating that LLM-generated misleading charts significantly degrade human chart comprehension, validating the real-world threat.
5. Mitigation Strategy: Demonstration that fine-tuning MLLMs on the AttackViz dataset can improve robustness against misleading charts.

4. Experimental Results

A. MLLM Vulnerability

The authors evaluated 16 open-weight and 3 closed-source MLLMs (including GPT-4o, Gemini-2.5, and Claude-4.6) on the AttackViz test set.

• Performance Degradation:
  • In-domain: Misleading charts reduced MLLM accuracy by an average of 17.2 percentage points (pp).
  • Cross-domain: Accuracy dropped by 11.9 pp on unseen datasets (ChartQA, ChartX).
  • Model Sensitivity: Higher-performing models (e.g., InternVL-3.5, Ovis-2.5) showed larger absolute drops (up to 26.6 pp) compared to smaller models, suggesting that advanced reasoning capabilities do not necessarily confer immunity to visual deception.
• Deception Rates:
  • The "Deception Rate" (instances where a model answers correctly on a clean chart but incorrectly on a misleading one) averaged 11.2% in-domain.
  • Most Effective Misleaders: 3D effects, inappropriate stacked bars, and misrepresentation caused the largest accuracy drops (up to 41.5 pp). Inverted axes and truncated axes were also highly effective.
  • Least Effective: Dual axes and ineffective color schemes had minimal impact.

B. Human Evaluation

A pilot study with 12 participants compared a control group (correct charts) against an experimental group (ChartAttack-generated misleading charts).

• Result: The experimental group's accuracy dropped by 20.2 pp (from 71.2% to 51.0%), closely mirroring the degradation seen in MLLMs.
• Implication: This confirms that LLM-generated misleading charts are not just AI failures but pose a genuine threat to human information processing.

C. Mitigation (Fine-tuning)

The authors fine-tuned Qwen2.5-VL-3B on the AttackViz dataset.

• Result: The fine-tuned model improved test-set performance by 8.4 pp on misleading charts compared to the base model.
• Trade-off: There was a slight decrease (-3.44 pp) in performance on correct charts, indicating a trade-off between robustness to deception and standard chart understanding.

5. Significance and Conclusion

This work exposes a critical, underexplored attack surface in the deployment of MLLMs for data visualization.

• Security Implications: It demonstrates that MLLMs can be easily manipulated to generate "data-faithful but visually deceptive" content at scale, bypassing safety mechanisms that typically focus on text or raw data integrity.
• Broader Impact: The findings highlight the urgent need for robustness in chart generation systems. Current safety guardrails (e.g., simple prompt instructions) were shown to be ineffective against design-level attacks.
• Future Directions: The paper advocates for the integration of robustness-aware training (using datasets like AttackViz) and the development of specialized detection mechanisms for misleading visualizations before they are deployed in real-world reporting and decision-making contexts.

In summary, ChartAttack provides a rigorous methodology and dataset to study the intersection of adversarial prompting and data visualization, proving that both AI and humans are highly susceptible to subtle, algorithmically generated visual deceptions.