通过分步视觉对齐训练,让模型更准理解图表逻辑。
CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

- 将图表问答拆解为多步视觉对齐推理,用空间注意力动态聚焦关键区域。
- 在合成数据集上训练后,比基线提升最高20.50%,真实场景提升12.30%。
- 适合需要精准图表理解的科研、金融或教育场景使用。
图表问答(CQA)要求多模态大模型整合视觉理解与逻辑推理,但现有模型在视觉对齐和连贯推理链方面表现不佳。尽管外部思维链提示和视觉线索能显著提升性能,但多模态大模型缺乏内在的视觉对齐推理能力,导致感知不准确且推理脱离视觉证据。为此,我们提出CURV,一种课程学习框架,通过将CQA重构为多步视觉对齐推理,每一步结合逻辑推理与动态视觉聚焦(空间注意力集中)。为辅助学习,我们构建了CCQA,一个三级课程化数据集,支持跨多种图表类型与推理模式的可扩展合成生成。课程从单操作推理逐步过渡到多图表组合任务。实验表明,CURV相比基线最多提升20.50%,在真实世界基准上提升达12.30%,在域外多模态推理任务中提升最高10.20%,验证了内化动态视觉对齐推理对增强图表理解的有效性。代码已公开:https://xhguo7.github.io/CURV/。
原文摘要 · Abstract (English)
Chart question answering (CQA) requires multimodal large language models (MLLMs) to integrate visual comprehension with logical reasoning, yet current models struggle with accurate visual grounding and coherent reasoning chains. While extrinsic chain-of-thought prompting and visual cues significantly improve performance, current MLLMs lack intrinsic visual grounded reasoning capabilities, leading to inaccurate perception and reasoning disconnected from visual evidence. To address these limitations, we propose CURV, a curriculum learning framework that develops intrinsic visual reasoning capabilities by reformulating CQA as multi-step visual grounded reasoning, where each step coordinates logical reasoning with dynamic visual grounding through spatial attention concentration. To assist model learning, we further introduce CCQA, a three-level curriculum dataset with scalable synthetic generation across diverse chart types and reasoning patterns. Our curriculum systematically progresses from basic single-operation reasoning to complex multi-chart compositional tasks. Experiments demonstrate that CURV achieves up to $\uparrow20.50\%$ improvements over baselines and is generalizable to real-world benchmarks (up to $\uparrow12.30\%$) and out-of-domain multimodal reasoning tasks (up to $\uparrow10.20\%$), validating the effectiveness of internalizing visual reasoning with dynamic grounding for enhanced chart understanding capabilities. Code is available at: https://xhguo7.github.io/CURV/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。