arXiv:2502.06860cs.CVcs.GR2025-02SIGGRAPH被引 3

用视觉语言模型自动补全草图风格,让残缺草图自然延续原作风格。

AutoSketch: VLM-assisted Style-Aware Vector Sketch Completion

  • 用预训练视觉语言模型提取草图风格描述,指导补全过程。
  • 通过扩散先验和可执行代码,实现风格精准匹配。
  • 支持多种草图风格与场景,适合创意设计与交互工具开发。

自动补全描绘复杂场景(如“一位女性在公园与男性交谈”)的不完整草图非常有用。然而,现有草图生成方法通常从零生成,无法在原始风格下完成部分草图。为此,我们提出AutoSketch,一种风格感知的矢量草图补全方法,可适应多样化的草图风格。关键观察是:草图的自然语言风格描述能有效保留风格信息。我们利用预训练视觉语言模型(VLM)将部分草图的风格转化为自然语言描述,并通过新生成的笔画复现这些风格。首先,优化笔画以匹配由VLM提取的风格描述增强后的输入提示,建立与原草图高度一致的扩散先验。随后,使用VLM生成可执行的风格调整代码,使笔画符合目标风格。我们在多种草图风格与提示上对比了现有方法,进行了广泛的消融实验及定性定量评估,结果表明AutoSketch能支持多种草图场景。

原文摘要 · Abstract (English)

The ability to automatically complete a partial sketch that depicts a complex scene, e.g., "a woman chatting with a man in the park", is very useful. However, existing sketch generation methods create sketches from scratch; they do not complete a partial sketch in the style of the original. To address this challenge, we introduce AutoSketch, a styleaware vector sketch completion method that accommodates diverse sketch styles. Our key observation is that the style descriptions of a sketch in natural language preserve the style during automatic sketch completion. Thus, we use a pretrained vision-language model (VLM) to describe the styles of the partial sketches in natural language and replicate these styles using newly generated strokes. We initially optimize the strokes to match an input prompt augmented by style descriptions extracted from the VLM. Such descriptions allow the method to establish a diffusion prior in close alignment with that of the partial sketch. Next, we utilize the VLM to generate an executable style adjustment code that adjusts the strokes to conform to the desired style. We compare our method with existing methods across various sketch styles and prompts, performed extensive ablation studies and qualitative and quantitative evaluations, and demonstrate that AutoSketch can support various sketch scenarios.

草图补全视觉语言模型风格迁移矢量生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。