用文字生成可交互的3D雕刻笔刷，提升创作效率
Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculpting
- 通过得分蒸馏采样引导平面网格变形生成矢量位移图
- 支持多样高质量笔刷生成，适配主流建模软件
- 解决文本引导生成局部结构的语义耦合问题
专业3D资产创作常需多种雕刻笔刷以添加表面细节和几何结构。尽管3D生成技术取得进展，但生成与艺术家工作流兼容的可复用雕刻笔刷仍是开放难题。这类笔刷通常以矢量位移图（VDM）表示，现有模型难以像生成自然图像那样有效生成。本文提出Text2VDM框架，通过分数蒸馏采样（SDS）引导密集平面网格变形，实现文本到VDM笔刷的生成。原始SDS损失针对完整物体生成设计，在笔刷生成中从零开始构建子结构时表现不佳，我们称之为语义耦合。为此，引入提示词权重混合机制，优化目标分布与语义引导。实验表明，Text2VDM能生成多样且高质量的VDM笔刷，用于表面细节与几何结构雕刻。生成的笔刷可无缝集成至主流建模软件，支持网格风格化与实时交互建模等多种应用。
原文摘要 · Abstract (English)
Professional 3D asset creation often requires diverse sculpting brushes to add surface details and geometric structures. Despite recent progress in 3D generation, producing reusable sculpting brushes compatible with artists' workflows remains an open and challenging problem. These sculpting brushes are typically represented as vector displacement maps (VDMs), which existing models cannot easily generate compared to natural images. This paper presents Text2VDM, a novel framework for text-to-VDM brush generation through the deformation of a dense planar mesh guided by score distillation sampling (SDS). The original SDS loss is designed for generating full objects and struggles with generating desirable sub-object structures from scratch in brush generation. We refer to this issue as semantic coupling, which we address by introducing weighted blending of prompt tokens to SDS, resulting in a more accurate target distribution and semantic guidance. Experiments demonstrate that Text2VDM can generate diverse, high-quality VDM brushes for sculpting surface details and geometric structures. Our generated brushes can be seamlessly integrated into mainstream modeling software, enabling various applications such as mesh stylization and real-time interactive modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。