让AI生成SVG时能自我检查,画得更准更可编辑。
IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework
- 用自洽的生成-评审循环,让模型自己评估渲染效果
- 在多个指标上达到当前最好水平,结构更复杂、语义更对齐
- 适合需要高质量可编辑矢量图的设计场景
矢量图形(SVG)因其可缩放性和可编辑性,在数字设计中至关重要。尽管视觉语言模型(VLM)推动了内容生成的发展,现有文本到SVG生成方法仍受限于自回归训练过程无法融入最终渲染图像的视觉感知,严重制约生成质量。为此,我们提出一种内省式SVG生成框架IntroSVG,核心是构建一个在闭环中同时扮演生成器与评审者双重角色的统一VLM。通过监督微调(SFT),模型学会生成SVG并对其渲染结果提供反馈;同时,我们将早期失败案例系统化转化为高质量纠错训练数据,提升模型鲁棒性。随后,利用高容量教师VLM构建偏好数据集,并通过直接偏好优化(DPO)进一步对齐生成策略。推理时,优化后的生成器与评审者协同进行迭代式‘生成-评审-修正’循环,从不完善的中间草图逐步自主提升输出质量。实验表明,该方法在多个关键评估指标上达到领先性能,生成的SVG具有更复杂的结构、更强的语义对齐性以及更高的可编辑性,验证了将显式视觉反馈引入生成环路的有效性。
原文摘要 · Abstract (English)
Scalable Vector Graphics (SVG) are central to digital design due to their inherent scalability and editability. Despite significant advancements in content generation enabled by Visual Language Models (VLMs), existing text-to-SVG generation methods are limited by a core challenge: the autoregressive training process does not incorporate visual perception of the final rendered image, which fundamentally constrains generation quality. To address this limitation, we propose an Introspective SVG Generation Framework (IntroSVG). At its core, the framework instantiates a unified VLM that operates in a closed loop, assuming dual roles of both generator and critic. Specifically, through Supervised Fine-Tuning (SFT), the model learns to draft SVGs and to provide feedback on their rendered outputs; moreover, we systematically convert early-stage failures into high-quality error-correction training data, thereby enhancing model robustness. Subsequently, we leverage a high-capacity teacher VLM to construct a preference dataset and further align the generator's policy through Direct Preference Optimization (DPO). During inference, the optimized generator and critic operate collaboratively in an iterative "generate-review-refine" cycle, starting from imperfect intermediate drafts to autonomously improve output quality. Experimental results demonstrate that our method achieves state-of-the-art performance across several key evaluation metrics, generating SVGs with more complex structures, stronger semantic alignment, and greater editability. These results corroborate the effectiveness of incorporating explicit visual feedback into the generation loop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。