评测大模型生成数据可视化代码的能力，提供真实任务基准。
Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code
- 构建175个真实数据可视化任务的基准数据集
- 大模型在主流库上表现良好，但在Plotly仍有不足
- 支持简洁指令输入，适合交互式数据分析场景
本文提出PandasPlotBench数据集，用于评估大语言模型作为数据可视化助手的有效性。该基准聚焦于根据自然语言指令生成表格数据（如Pandas DataFrame）的可视化代码，涵盖Matplotlib、Seaborn和Plotly三个主流绘图库。数据集包含175个独特任务，实验表明任务简化对绘图能力影响极小，支持用户以简短指令完成复杂可视化。尽管模型在Matplotlib和Seaborn上表现良好，但对Plotly仍存在显著挑战。该基准设计模块化，可扩展性强，有助于推动自动化可视化研究。数据与代码已开源：https://huggingface.co/datasets/JetBrains-Research/PandasPlotBench；https://github.com/JetBrains-Research/PandasPlotBench。
原文摘要 · Abstract (English)
This paper introduces the human-curated PandasPlotBench dataset, designed to evaluate language models' effectiveness as assistants in visual data exploration. Our benchmark focuses on generating code for visualizing tabular data - such as a Pandas DataFrame - based on natural language instructions, complementing current evaluation tools and expanding their scope. The dataset includes 175 unique tasks. Our experiments assess several leading Large Language Models (LLMs) across three visualization libraries: Matplotlib, Seaborn, and Plotly. We show that the shortening of tasks has a minimal effect on plotting capabilities, allowing for the user interface that accommodates concise user input without sacrificing functionality or accuracy. Another of our findings reveals that while LLMs perform well with popular libraries like Matplotlib and Seaborn, challenges persist with Plotly, highlighting areas for improvement. We hope that the modular design of our benchmark will broaden the current studies on generating visualizations. Our dataset and benchmark code are available online: https://huggingface.co/datasets/JetBrains-Research/PandasPlotBench; https://github.com/JetBrains-Research/PandasPlotBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。