构建首个动态图表理解基准，评估模型在交互中分析图表的能力
ChartAct: A Benchmark for Dynamic Chart Understanding
- 设计交互式评测环境，模拟悬停、点击等操作下的图表变化
- 11个模型平均成功率仅60%以下，最强模型达84.5%
- 适合研究交互式数据可视化与多模态智能的学者
图表广泛用于数据分析与决策支持。现有图表理解基准多聚焦静态图表，但真实场景中的图表常为动态且可交互，关键信息需通过悬停、点击、缩放或拖拽等操作才可显现。因此，动态图表理解要求模型能识别可见内容、选择合适交互动作，并对状态变化进行推理。为此，我们提出 ChartAct，一个面向动态图表理解的交互式基准。ChartAct 从 8 个真实网站收集并筛选 673 个动态图表，涵盖 7 种常见图表类型，构建了 1,440 个高质量问答样本。每个样本在两种环境（Dynamic Chart 与 Dashboard Chart）中实例化，以评估不同上下文下的表现。基于 ChartAct，我们系统评估了 11 个先进多模态模型与 GUI 代理。实验结果表明，现有模型在动态图表理解方面仍有明显局限：最强模型 Claude-Opus-4.7 平均成功率达 84.5%，多数模型低于 60%。我们还进行了详细的失败归因与案例分析。ChartAct 为研究真实交互环境中图表理解提供了新基准。代码见 https://github.com/wulin-wulin/OSWorld_Chart
原文摘要 · Abstract (English)
Charts are widely used to present complex data for analysis and decision making. Existing chart understanding benchmarks mainly focus on static charts, but real-world charts are often dynamic and interactive. Key information may only appear after actions such as hovering, clicking, zooming, or dragging. Dynamic chart understanding therefore requires models to identify visible content, choose proper interactions, and reason over changing chart states. To evaluate this ability, we propose ChartAct, an interactive benchmark for dynamic chart understanding. ChartAct collects and filters 673 dynamic charts from 8 real chart websites, covers 7 common chart types, and constructs 1,440 high-quality question-answer samples. Each sample is instantiated in two environments, Dynamic Chart and Dashboard Chart, to evaluate dynamic chart understanding under different contexts. Based on ChartAct, we systematically evaluate 11 advanced multimodal models and GUI agents. Experimental results show that existing models still have clear limitations in dynamic chart understanding. The strongest model, Claude-Opus-4.7, achieves an average success rate of 84.5\%, while most models remain below 60\%. We also conduct detailed failure attribution and case analysis. ChartAct provides a new benchmark for studying chart understanding in real interactive environments. Codes at https://github.com/wulin-wulin/OSWorld_Chart
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。