让AI理解交互式数据看板并自动生成代码。
Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards
- 通过自主点击、筛选等操作获取反馈，生成可复现的看板代码。
- 构建首个基于Plotly+Dash的180组看板-代码对基准数据集。
- 适合关注交互式可视化生成与模型评估的研究者。
自动数据可视化生成已随多模态大模型快速发展，但现有工作主要聚焦静态图表，忽视了真实数据探索中常用的交互式看板。本文提出Dashboard2Code新任务：要求模型主动探索交互式看板，通过自身操作（如点击、筛选）获取反馈并整合信息，生成可复现目标看板的代码。为支持全面评估，我们构建了DashboardMimic，首个基于Plotly+Dash的基准数据集，包含180组精心设计且人工验证的看板-代码对，覆盖三个难度等级和八种常见真实交互模式。同时提出自动化评估框架，结合代码语义分析与动态交互测试，评估视觉与交互一致性，与人工判断高度一致。在多种开源与闭源多模态模型上的实验表明，即使最强系统在高复杂度看板上仍表现不佳，且开源与闭源模型间存在显著性能差距。
原文摘要 · Abstract (English)
Automatic data visualization generation has advanced rapidly with multi-modal large language models, yet existing efforts largely focus on static charts and overlook the interactive dashboards commonly used for real-world data exploration. We introduce Dashboard2Code, a novel task that requires a model to proactively explore an interactive dashboard, acquire and integrate feedback from its own interactions (e.g., clicking and filtering), and generate code that reproduces the target dashboard. To support comprehensive evaluation, we present DashboardMimic, the first Plotly+Dash benchmark for Dashboard2Code, comprising 180 carefully designed and manually verified dashboard-code pairs spanning three difficulty levels and covering eight common real-world interaction patterns. We further propose an automated evaluation framework tailored to dashboards that combines code semantic analysis with dynamic interaction-based testing to assess visual and interaction consistency, showing strong agreement with human judgments. Experiments across a range of open- and closed-source multi-modal models reveal that even the strongest systems struggle on high-complexity dashboards and that a substantial performance gap remains between open-source and closed-source models on the Dashboard2Code task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。