💬 NLP

CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

The paper introduces CURV, a curriculum learning framework paired with the CCQA dataset, which enhances chart question answering by training multimodal models to perform intrinsic, multi-step visual grounded reasoning, resulting in significant performance improvements across various benchmarks.

Original authors: Xuehang Guo, Pingyue Zhang, Ruiyi Zhang, Zhenhailong Wang, Hanrui Lyu, Heng Ji, Tong Sun, Qingyun Wang, Manling Li

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xuehang Guo, Pingyue Zhang, Ruiyi Zhang, Zhenhailong Wang, Hanrui Lyu, Heng Ji, Tong Sun, Qingyun Wang, Manling Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a complex puzzle, but instead of just looking at the picture on the box, you have to figure out the picture while you are building it. This is the daily struggle of Multimodal Large Language Models (MLLMs)—super-smart computer programs that can read text and look at images at the same time. Right now, these programs are like brilliant students who can read a math textbook perfectly but get confused when asked to solve a problem on a whiteboard because they can't quite focus on the right numbers. They often mix up what they see with what they think, leading to answers that sound logical but are totally wrong because they misread a chart or missed a tiny detail. Scientists have tried to fix this by giving the computers "reference materials" or asking them to talk through their steps out loud (a method called Chain-of-Thought), but the computers still struggle to connect their thinking directly to the visual evidence in real-time. They need to learn how to look, think, and look again, all in one smooth motion, just like a human does.

Enter CURV, a new training method designed to teach these AI models how to become better visual detectives. Think of CURV as a "video game level system" for learning. Instead of throwing a computer into the hardest boss battle immediately, CURV starts it on "Level 1," where it learns to read simple charts and point to the right numbers. As it gets better, it unlocks "Level 2," where it has to combine two pieces of information, and finally "Level 3," where it has to juggle multiple charts at once. The secret sauce is that at every single step of the way, the model is forced to physically point to the part of the image it is talking about. It's like teaching a child to do math by making them touch the apples on the table before they can say "five."

The researchers behind this paper, CURV, found that this step-by-step, "look-then-think" training works wonders. They created a massive new dataset called CCQA (Curriculum Chart Question Answering) to act as the training ground. This dataset is built like a ladder, with questions getting progressively harder, from simple "what is this number?" to complex "compare these two graphs and tell me the difference." By training on this ladder, the models learned to internalize the skill of grounding their reasoning in what they actually see. The results were impressive: models trained with CURV improved their accuracy by up to 20.50% on their training tests. Even more importantly, they didn't just get good at the practice questions; they got better at real-world charts they had never seen before, improving by up to 12.30%, and even got better at other types of visual puzzles, like math problems, by up to 10.20%.

The paper suggests that the key to this success isn't just giving the AI more data, but changing how it learns. The researchers argue that previous methods failed because they treated seeing and thinking as separate tasks. CURV proves that by weaving them together—making the model constantly shift its visual focus to match its logical steps—it builds a stronger, more reliable brain for understanding charts. However, the authors also note a funny quirk: while forcing the model to "point" at the image during training makes it smarter, forcing it to "point" out loud during the final test sometimes makes it stumble, likely because it gets confused by its own pointing. This suggests that the best approach is to teach the model to look and think together until it becomes a natural habit, so it doesn't need to stop and point anymore to get the right answer. Ultimately, CURV shows that if you teach AI to break big problems into small, visual steps, it can learn to understand the world's data much more like a human does.

Technical Summary: CURV - Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

Problem Definition

Chart Question Answering (CQA) requires Multimodal Large Language Models (MLLMs) to integrate visual comprehension with logical reasoning. However, current MLLMs struggle with accurate visual grounding and coherent reasoning chains. Preliminary analysis reveals that existing models suffer from three fundamental limitations:

Decomposition: Inability to break complex problems into coherent chains of reasoning, often producing inconsistent intermediate steps.
Interleaved Visual Reasoning: Failure to accurately ground individual reasoning steps in visual input (e.g., misreading chart values or attending to incorrect regions).
Composition: Difficulty in integrating logical reasoning with visual grounding across multiple steps, leading to a disconnect between perception, reasoning, and conclusion.

While extrinsic Chain-of-Thought (CoT) prompting and visual cues improve performance, models lack intrinsic visual grounded reasoning capabilities. Without external support, MLLMs often fail to align their reasoning with visual evidence, resulting in errors even when necessary information is present.

Methodology: The CURV Framework

The authors propose CURV, a curriculum learning framework designed to internalize extrinsic CoT prompting and visual guidance into intrinsic capabilities. The core approach reformulates CQA as a multi-step visual grounded reasoning process where each step couples logical reasoning with dynamic visual grounding.

1. Problem Reformulation

Instead of a direct mapping $f_\theta: (I, Q) \to A$ , CURV decomposes the task into a structured progressive chain:
$f_\theta: (I, Q) \to \{(R_1, V_1), (R_2, V_2), \dots, (R_T, V_T)\} \to A$
Where $R_t$ is the $t$ -th reasoning step and $V_t$ is the visual region grounding that step.

2. Two-Stage Curriculum Training

CURV employs a two-stage training paradigm to develop these capabilities:

Stage I: Visual Grounding (RV): The model learns to establish multimodal correspondences between reasoning flows and visual focuses. It predicts the visual focus $V_t$ that grounds the reasoning step $R_t$ onto the image $I$ . The objective is to minimize grounding loss ( $L_V$ ) between predicted and ground-truth visual focuses.
Stage II: Interleaved Visual Reasoning (RVA): Building on Stage I, the model learns to proactively leverage visual grounding as feedback. At each step, the predicted visual focus is applied to the input image to construct a grounded visual state $I_t$ , which is then used as additional input for the next reasoning step. This enables the model to dynamically shift visual attention in alignment with evolving logical reasoning. The training objective balances reasoning supervision, grounding loss, and final answer prediction.

3. Visual Grounding Strategies

To facilitate dynamic focus shifting, the framework implements three grounding strategies:

Applied: Semi-transparent highlighting overlays on relevant regions.
Boxed: Red rectangular borders around focus regions.
Cropped: Zoomed-in sub-regions presented alongside the full chart.

4. Curriculum Design

The framework utilizes a systematic progression of task difficulty:

Reasoning Depth ( $D$ ): Defined by the number of nested logical functions required (e.g., $f_1(f_2(...(x)))$ ).
Visual Complexity: Progresses from single-plot charts to multi-plot compositions.
Curriculum Levels:
- Level 1: Foundational single-operation reasoning ( $D=1$ ).
- Level 2: Multi-operation reasoning on single charts ( $D \ge 2$ ).
- Level 3: Complex reasoning across multiple subplots ( $D \ge 3$ ), involving localization and relational understanding.

Key Contributions

CURV Framework: A curriculum learning framework that develops intrinsic visual reasoning capabilities by progressively shifting visual attention along reasoning chains, transitioning from basic tasks to complex nested reasoning.
CCQA Dataset: A scalable, synthetic Curriculum Chart Question Answering (CCQA) dataset. It features three progressive curriculum levels, supports four generation modes, and includes diverse chart types (bar, histogram, scatter, line, heatmap, pie, radar) across 30 domain categories. The dataset is constructed using template-based generation to ensure accuracy and systematic coverage of reasoning patterns.
Visual Grounding Strategies: The introduction and evaluation of three distinct grounding strategies (Applied, Boxed, Cropped) to control the granularity and representation of visual focus during reasoning.
Empirical Validation: Comprehensive experiments demonstrating that interleaving reasoning with visual grounding provides step-by-step alignment between perception and logic, leading to significant improvements across chart types, complexity levels, and domains.

Experimental Results

Experiments were conducted on CCQA, real-world chart benchmarks (ChartQA, ChartMuseum, CharXiv, ChartQA-Pro), and out-of-domain multimodal reasoning tasks (MathVista, MMMU-Pro).

Performance on CCQA: CURV-tuned models achieved up to 20.92% absolute improvement over baselines in two-stage training. The best configuration (CURV@Applied with Qwen2.5-VL-7B) outperformed its base model by 15.65% and surpassed GPT-4o by 12.22%.
Generalization: Models trained on single-plot charts generalized effectively to multi-plot charts (Level 3), showing improvements of up to 7.10%.
Real-World Benchmarks: CURV demonstrated strong generalizability to real-world chart understanding benchmarks, with improvements of ≥1.20% across four datasets.
Out-of-Domain Reasoning: The framework improved performance on out-of-domain multimodal reasoning tasks by up to 10.20%, validating that the benefits arise from improved visual-structure grounding rather than task-specific adaptation.
Training Paradigms: The approach is compatible with Supervised Fine-Tuning (SFT), Reinforcement Learning (RL), and their combination. Combining SFT and RL yielded the highest gains (up to 17.04%), though SFT alone offered a more favorable performance-efficiency trade-off.

Significance and Claims

The paper claims that CURV establishes a foundation for developing self-contained visual reasoning capabilities in MLLMs. By internalizing the human cognitive process of decomposing tasks, interleaving visual grounding, and composing solutions, CURV moves beyond reliance on extrinsic assistance.

Key findings highlighted by the authors include:

Intrinsic vs. Extrinsic: While explicit visual grounding during inference can sometimes degrade performance due to error accumulation, using visual grounding as a training signal effectively internalizes the ability to focus on relevant image components, improving performance even in direct answer generation modes.
Foundational Learning: Training on foundational levels (Level 1 + 2) yields the most consistent and substantial gains across all difficulty levels, whereas relying solely on high-complexity data (Level 3) can degrade foundational reasoning abilities.
Visual Grounding as a Bridge: Visual grounding serves as an integral intermediate bridge between vision and reasoning, rather than just a terminal prediction objective. Explicit grounding strategies (particularly "Applied" highlighting) proved more effective than implicit methods.

The work suggests that tightly interleaving reasoning with dynamic visual grounding is a critical mechanism for enhancing chart understanding and general multimodal reasoning capabilities.