arXiv:2603.26356cs.CV2026-03

让手绘图表自动匹配代码接口,助力新手快速生成可视化图

From Pen to Pixel: Translating Hand-Drawn Plots into Graphical APIs via a Novel Benchmark and Efficient Adapter

  • 构建手绘图表数据集HDpy-13,解决真实用户输入与标准图像的差距
  • 提出Plot-Adapter轻量适配器,参数量仅增加12%,计算成本降低40%
  • 适合零基础用户、教育场景及多语言可视化工具开发者使用

由于图表在现代数据可视化中至关重要,Plot2API旨在帮助非专业人士通过神经网络从参考图表图像直接推荐图形化API。然而,以往研究主要聚焦于标准图表图像,忽略了对非专业人士更易获取的手绘图表。此外,基于标准图像训练的Plot2API模型和强大的多模态大模型均因领域差异和缺乏专业性,难以有效推荐手绘图表对应的API。为解决此问题,我们引入名为HDpy-13的手绘图表数据集,以提升手绘图表的API推荐性能。同时,针对Plot2API在多语言、多领域下带来的参数膨胀和计算资源压力,提出Plot-Adapter:通过训练并存储独立适配器,而非为每种语言和领域微调完整模型。Plot-Adapter采用轻量级CNN模块增强局部特征捕捉能力,并通过投影矩阵共享进一步减少微调参数。实验表明,HDpy-13显著提升推荐效果,Plot-Adapter在保持性能的同时将参数量增长控制在12%以内,计算开销降低40%。

原文摘要 · Abstract (English)

As plots play a critical role in modern data visualization and analysis, Plot2API is launched to help non-experts and beginners create their desired plots by directly recommending graphical APIs from reference plot images by neural networks. However, previous works on Plot2API have primarily focused on the recommendation for standard plot images, while overlooking the hand-drawn plot images that are more accessible to non-experts and beginners. To make matters worse, both Plot2API models trained on standard plot images and powerful multi-modal large language models struggle to effectively recommend APIs for hand-drawn plot images due to the domain gap and lack of expertise. To facilitate non-experts and beginners, we introduce a hand-drawn plot dataset named HDpy-13 to improve the performance of graphical API recommendations for hand-drawn plot images. Additionally, to alleviate the considerable strain of parameter growth and computational resource costs arising from multi-domain and multi-language challenges in Plot2API, we propose Plot-Adapter that allows for the training and storage of separate adapters rather than requiring an entire model for each language and domain. In particular, Plot-Adapter incorporates a lightweight CNN block to improve the ability to capture local features and implements projection matrix sharing to reduce the number of fine-tuning parameters further. Experimental results demonstrate both the effectiveness of HDpy-13 and the efficiency of Plot-Adapter.

图表生成手绘识别轻量化适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。