让机器像人一样一步步看懂图表,支持无数字场景下的可靠分析。
ChartAgent: A Chart Understanding Framework with Tool Integrated Reasoning
- 通过工具协同推理,分步解析图表元素
- 在缺少关键数字时仍保持高鲁棒性,准确率显著提升
- 适合需要可解释性与可信度的图表分析场景
图表因其信息密度高、直观易读,已成为跨学科数据呈现与交流的主流方式。尽管多模态大模型在自动图表理解上取得进展,但仍高度依赖显式文本标注,当关键数值缺失时性能急剧下降。为此,我们提出ChartAgent,一种基于工具集成推理(TIR)的图表理解框架。受人类认知启发,ChartAgent将复杂分析任务分解为可观测、可复现的步骤。其核心是一个可扩展、模块化的工具库,包含十余种核心工具,如关键元素检测、实例分割和光学字符识别(OCR),支持动态调度以实现对多种图表类型的系统性视觉解析。借助TIR的透明性与可验证性,ChartAgent将中间输出标准化为结构化证据包,提供可追溯、可复现的结论支撑。实验表明,ChartAgent在标注稀疏条件下显著提升鲁棒性,为构建可信、可扩展的图表理解系统提供了可行路径。
原文摘要 · Abstract (English)
With their high information density and intuitive readability, charts have become the de facto medium for data analysis and communication across disciplines. Recent multimodal large language models (MLLMs) have made notable progress in automated chart understanding, yet they remain heavily dependent on explicit textual annotations and the performance degrades markedly when key numerals are absent. To address this limitation, we introduce ChartAgent, a chart understanding framework grounded in Tool-Integrated Reasoning (TIR). Inspired by human cognition, ChartAgent decomposes complex chart analysis into a sequence of observable, replayable steps. Supporting this architecture is an extensible, modular tool library comprising more than a dozen core tools, such as keyelement detection, instance segmentation, and optical character recognition (OCR), which the agent dynamically orchestrates to achieve systematic visual parsing across diverse chart types. Leveraging TIRs transparency and verifiability, ChartAgent moves beyond the black box paradigm by standardizing and consolidating intermediate outputs into a structured Evidence Package, providing traceable and reproducible support for final conclusions. Experiments show that ChartAgent substantially improves robustness under sparse annotation settings, offering a practical path toward trustworthy and extensible systems for chart understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。