arXiv:2608.21659cs.CV2026-08中稿 · SIGGRAPH

用流模型生成自然手绘矢量草图,零样本泛化能力强。

SketchFlow: Zero-Shot Vector Sketch Generation via GMM Prior Flow in CLIP Latent Space

论文配图:SketchFlow: Zero-Shot Vector Sketch Generation via GMM Prior Flow in CLIP Latent Space
图 1 · 摘自论文原文
  • 基于最优传输与流匹配,在CLIP隐空间构建连续先验生成草图。
  • 在QuickDraw外概念上实现高质量零样本草图生成,支持语义插值。
  • 结合一维U-Net与Transformer的解码器,快速生成高保真笔画轨迹。

矢量草图是人类抽象表达最简洁直接的方式之一。然而,由于高质量细粒度图文草图数据稀缺,生成具有类人绘制风格的高质量矢量笔画仍具挑战。现有文本条件生成方法常依赖不稳定且耗时的优化,或难以在零样本场景下泛化至未见类别。为此,我们提出SketchFlow,一种基于最优传输(OT)理论和流匹配的生成框架。通过预训练的CLIP模型绕过繁琐的图像级文本标注,将跨模态对齐建模为直接在CLIP隐空间中的连续映射问题。为弥合离散文本概念与连续草图特征间的模态鸿沟,我们向离散类别嵌入注入噪声,构建连续高斯混合模型(GMM)先验。随后利用最优传输条件流匹配(OT-CFM)模型学习从该先验到目标草图特征分布的确定性向量场。最后,设计融合1D U-Net与Transformer结构的混合扩散解码器,将特征解码为快速且高保真的笔画轨迹。大量实验表明,SketchFlow在视觉质量与类人绘制风格一致性上显著优于现有基线。此外,本框架具备几何保持能力,在超出QuickDraw训练词汇的提示(包括未见概念标签与语义修饰符)上实现出色的局部零样本合成,并支持不同概念间的平滑连续语义插值。源代码已开源:https://github.com/doudin404/SketchFlow。

原文摘要 · Abstract (English)

Vector sketches remain one of the most concise and immediate mediums for abstract human expression. However, generating high-quality vector strokes that exhibit human-like drawing styles remains an open challenge due to the severe scarcity of fine-grained, high-quality text-to-sketch paired data. Existing text-conditioned generation methods often rely on unstable, time-consuming optimization or struggle to generalize to unseen categories in a zero-shot manner. To address these limitations, we present SketchFlow, a novel generative framework rooted in Optimal Transport (OT) theory and flow matching. By leveraging pre-trained CLIP models to bypass labor-intensive image-level text annotations, we formulate cross-modal alignment as a continuous mapping problem directly within the CLIP latent space. To bridge the inevitable modality gap between discrete text concepts and continuous sketch features, we first inject noise into discrete category embeddings to construct a continuous Gaussian Mixture Model (GMM) prior. We then utilize an Optimal Transport Conditional Flow Matching (OT-CFM) model to learn a deterministic vector field mapping from this continuous GMM prior to the target sketch feature distribution. Finally, a Hybrid Diffusion Decoder, fusing 1D U-Net and Transformer architectures, is designed to decode these features into fast and high-fidelity stroke trajectories. Extensive experiments demonstrate that SketchFlow substantially outperforms existing baselines in visual quality and adherence to natural human drawing styles. Furthermore, our geometry-preserving framework demonstrates promising local zero-shot synthesis for prompts beyond the QuickDraw training vocabulary, including unseen concept labels and semantic modifiers, while enabling smooth, continuous semantic interpolation between distinct concepts. Source code is available at: https://github.com/doudin404/SketchFlow.

草图生成流模型零样本扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。