用视觉语言模型自动将复杂技术图转为可编辑的SVG文件
VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models
- 用描述生成管道构建6.6万对真实论文图与SVG,覆盖专业图表复杂性
- 引入结构感知评估体系,首次量化判断箭头位置、连接关系等正确性
- 开源模型性能超现有最佳方案30%,接近顶级闭源模型水平
可缩放矢量图形(SVG)在技术绘图和数字设计中至关重要,具有分辨率无关性和语义可编辑性。实践中,原始矢量文件常丢失,仅剩难以修改的位图(如PNG、JPEG),而手动重建成本过高。当前自动化位图转SVG的研究受制于两大瓶颈:现有数据集以图标和装饰性图形为主,缺乏专业图表复杂性;评估依赖像素或嵌入相似性,无法捕捉结构正确性(如连接断裂、箭头错位)。本文通过两项核心贡献填补空白,聚焦于以图表为中心的图像(如模型架构图、流程图、示意图)。训练方面,提出VFIG-Data,目前同类最大数据集,共66,000对,结合真实论文图经描述生成管道转换所得,以及程序生成图提供的无噪声监督(涵盖箭头样式、字体、几何布局)。评估方面,提出VFIG-Bench结构感知评估套件,并配套人工从高引arXiv论文中精选的VFIG-Bench-OOD分布外测试集。评估协议超越像素与嵌入相似性,新增基于评分卡的VLM-Judge得分及成对人类偏好评估的Elo评分。基于此,训练出VFIG系列视觉语言模型,采用由简至繁的SFT微调流程,再结合渲染感知奖励进行强化学习。VFIG达到开源模型最优表现,较最佳开源基线提升超30%,在VFIG-Bench上媲美Claude Sonnet 4.6:Gemini-Judge 78.2% vs. 76.7%,GPT-Judge 87.5% vs. 87.4%。仍略逊于最强闭源模型GPT-5.2和Gemini-3。
原文摘要 · Abstract (English)
Scalable Vector Graphics (SVG) are essential for technical illustration and digital design, offering resolution independence and semantic editability. In practice, original vector files are frequently lost, leaving only rasterized versions (e.g., PNG, JPEG) that resist modification, while manual reconstruction is prohibitively expensive. Progress on automating raster-to-SVG conversion has been bottlenecked by two gaps: existing SVG datasets are dominated by icons and decorative graphics that lack the complexity of professional diagrams, and existing benchmarks rely on pixel- or embedding-level similarity that fails to capture structural correctness (e.g., broken connectivity, misplaced arrows). We close both gaps with paired contributions targeting diagram-centric figures (e.g., model architectures, flowcharts, schematics). For training, we introduce VFIG-Data, the largest figure-to-SVG dataset of its kind at 66K pairs, combining real paper figures converted via a describe-and-generate pipeline with programmatic diagrams that supply noise-free supervision over arrow styles, fonts, and geometry. For evaluation, we introduce VFIG-Bench, a structure-aware evaluation suite, paired with VFIG-Bench-OOD, an out-of-distribution set of figures manually curated from highly cited arXiv papers. Beyond pixel and embedding similarity, our protocol reports rubric-based VLM-Judge scores and Elo ratings from pairwise human preference evaluation. Built on these contributions, VFIG is a VLM family trained with a simple-to-complex SFT curriculum followed by RL with rendering-aware rewards. VFIG achieves state-of-the-art open-source performance, outperforming the best open-source VLM baseline by over 30%, and matches Claude Sonnet 4.6 on VFIG-BENCH: Gemini-Judge 78.2% vs. 76.7% and GPT-Judge 87.5% vs. 87.4%. It remains slightly behind the strongest proprietary models GPT-5.2 and Gemini-3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。