用位图监督训练矢量图生成模型，突破数据瓶颈。
Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision
- 通过位图重建学习矢量形状离散编码，无需直接依赖SVG数据
- 在MNIST和图标字体数据上生成质量超越已有图像监督方法
- 适合需要灵活矢量生成的UI设计与AI绘图应用
可缩放矢量图形（SVG）是网络和设计行业的常用格式。然而，由于其离散且复杂的特性，生成建模研究长期忽视了SVG。我们提出GRIMOIRE，一个基于文本引导的SVG生成模型，包含两个模块：视觉形状量化器（VSQ）通过重建位图来学习将图像映射到离散代码本，自回归变压器（ART）建模形状标记、位置与文本描述的联合分布，实现从自然语言生成矢量图形。与以往需要直接使用SVG数据监督的方法不同，GRIMOIRE仅需位图监督即可学习形状图像块，大幅扩展可用数据规模。我们在MNIST上对闭合填充形状进行拟合，在图标和字体数据上对轮廓笔画进行生成，结果表明其生成质量超过现有图像监督方法，同时具备向量监督方法的灵活性。
原文摘要 · Abstract (English)
Scalable Vector Graphics (SVG) is a popular format on the web and in the design industry. However, despite the great strides made in generative modeling, SVG has remained underexplored due to the discrete and complex nature of such data. We introduce GRIMOIRE, a text-guided SVG generative model that is comprised of two modules: A Visual Shape Quantizer (VSQ) learns to map raster images onto a discrete codebook by reconstructing them as vector shapes, and an Auto-Regressive Transformer (ART) models the joint probability distribution over shape tokens, positions and textual descriptions, allowing us to generate vector graphics from natural language. Unlike existing models that require direct supervision from SVG data, GRIMOIRE learns shape image patches using only raster image supervision which opens up vector generative modeling to significantly more data. We demonstrate the effectiveness of our method by fitting GRIMOIRE for closed filled shapes on the MNIST and for outline strokes on icon and font data, surpassing previous image-supervised methods in generative quality and vector-supervised approach in flexibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。