构建首个覆盖20多种图表的文本生成可视化基准,推动AI理解复杂数据指令
Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text
- 设计包含1985个样本的多模态图表生成评测集,涵盖趋势、相关性等复杂查询
- 发现主流模型在动态数据与对话式查询中准确率不足30%,显著低于人类水平
- 提出跨模态智能体框架,使GPT-4o图表生成通过率从26%提升至42%
自动化数据可视化在简化数据分析、提升决策效率方面至关重要。尽管大语言模型在从自然语言生成可视化方面展现出潜力,但缺乏全面的评估基准限制了其能力验证。本文提出Text2Vis,一个用于评估文本到可视化模型的基准,涵盖20多种图表类型和多样化的数据科学任务,包括趋势分析、相关性识别、异常值检测及预测分析。该数据集包含1,985个样本,每个样本配有数据表、自然语言查询、简短答案、可视化代码和标注图表,查询涉及复杂推理、对话轮次与动态数据获取。我们对11个开源与闭源模型进行测评,揭示显著性能差距,指明关键挑战,并为未来改进提供洞见。为缩小差距,我们提出首个跨模态演员-评论家智能体框架,联合优化文本回答与可视化代码,使GPT-4o的通过率从26%提升至42%。同时引入基于LLM的自动化评估框架,可无须人工标注即可在数千样本上评估答案正确性、代码执行成功率、可视化可读性与图表准确性。Text2Vis已开源于https://github.com/vis-nlp/Text2Vis。
原文摘要 · Abstract (English)
Automated data visualization plays a crucial role in simplifying data interpretation, enhancing decision-making, and improving efficiency. While large language models (LLMs) have shown promise in generating visualizations from natural language, the absence of comprehensive benchmarks limits the rigorous evaluation of their capabilities. We introduce Text2Vis, a benchmark designed to assess text-to-visualization models, covering 20+ chart types and diverse data science queries, including trend analysis, correlation, outlier detection, and predictive analytics. It comprises 1,985 samples, each with a data table, natural language query, short answer, visualization code, and annotated charts. The queries involve complex reasoning, conversational turns, and dynamic data retrieval. We benchmark 11 open-source and closed-source models, revealing significant performance gaps, highlighting key challenges, and offering insights for future advancements. To close this gap, we propose the first cross-modal actor-critic agentic framework that jointly refines the textual answer and visualization code, increasing GPT-4o`s pass rate from 26% to 42% over the direct approach and improving chart quality. We also introduce an automated LLM-based evaluation framework that enables scalable assessment across thousands of samples without human annotation, measuring answer correctness, code execution success, visualization readability, and chart accuracy. We release Text2Vis at https://github.com/vis-nlp/Text2Vis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。