用图像做桥梁，让AI零样本生成高精度绘图代码
TikZero: Zero-Shot Text-Guided Graphics Program Synthesis
- 用图像作为中间媒介，分离文本理解与绘图生成
- 零样本下可精准生成符合文本描述的TikZ代码
- 适合需要精确可编辑图形的科研与设计场景
从文本描述自动生成图形是一项重要能力。但实现高几何精度和可编辑性需将图形表示为TikZ等编程语言的程序，而带文字标注的程序数据稀缺。相比之下，大量未对齐的图形程序和带注释的位图图像更易获取。本文提出TikZero，通过图像表示作为中介，解耦绘图程序生成与文本理解，实现对图形程序和带图文字数据的独立训练，并在推理时支持零样本文本引导的绘图程序生成。实验表明，该方法显著优于仅依赖对齐数据的基线模型。当结合少量对齐数据作为补充信号时，TikZero性能媲美甚至超越更大规模模型（如GPT-4o）。代码、数据集及部分模型已公开。
原文摘要 · Abstract (English)
Automatically synthesizing figures from text captions is a compelling capability. However, achieving high geometric precision and editability requires representing figures as graphics programs in languages like TikZ, and aligned training data (i.e., graphics programs with captions) remains scarce. Meanwhile, large amounts of unaligned graphics programs and captioned raster images are more readily available. We reconcile these disparate data sources by presenting TikZero, which decouples graphics program generation from text understanding by using image representations as an intermediary bridge. It enables independent training on graphics programs and captioned images and allows for zero-shot text-guided graphics program synthesis during inference. We show that our method substantially outperforms baselines that can only operate with caption-aligned graphics programs. Furthermore, when leveraging caption-aligned graphics programs as a complementary training signal, TikZero matches or exceeds the performance of much larger models, including commercial systems like GPT-4o. Our code, datasets, and select models are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。