用户手绘草图即可生成3D人体动作,效率提升近一半。
DrawMotion: Generating 3D Human Motions by Freehand Drawing

- 通过自动解析手绘 stickman 草图实现空间动作控制。
- 结合文本与草图条件,生成动作与意图匹配度更高。
- 无需训练即可通过梯度引导优化动作,适合快速创作场景。
文本到动作生成面临用户难以精准用文字描述动作的问题。本文提出 DrawMotion,一种基于扩散模型的多条件框架,可同时接收文本和新颖的手绘草图条件,分别提供语义与空间控制。为准确捕捉用户意图,我们开发算法自动将不同数据格式转换为手绘 stickman 草图;提出多条件融合模块(MCM),嵌入扩散过程,在保持计算效率的同时支持所有条件组合;并利用 MCM 中间特征的连续性,实现无需训练的分类器引导,通过梯度更新使生成动作更贴合用户意图且保留细节。定量实验与用户研究显示,使用手绘草图可减少约46.7%的用户时间。代码、演示及数据已公开于 https://github.com/InvertedForest/DrawMotion。
原文摘要 · Abstract (English)
Text-to-motion generation, which translates textual descriptions into human motions, faces the challenge that users often struggle to precisely convey their intended motions through text alone. To address this issue, this paper introduces DrawMotion, an efficient diffusion-based framework designed for multi-condition scenarios. DrawMotion generates motions based on both a conventional text condition and a novel hand-drawing condition, which provide semantic and spatial control over the generated motions, respectively. Specifically, we tackle the fine-grained motion generation task from three perspectives: 1) freehand drawing condition. To accurately capture users' intended motions without requiring tedious textual input, we develop an algorithm to automatically generate hand-drawn stickman sketches across different dataset formats; 2) multi-condition fusion. We propose a Multi-Condition Module (MCM) that is integrated into the diffusion process, enabling the model to exploit all possible condition combinations while reducing computational complexity compared to conventional approaches; and 3) training-free guidance. Notably, the MCM in DrawMotion ensures that its intermediate features lie in a continuous space, allowing classifier-guidance gradients to update the features and thereby aligning the generated motions with user intentions while preserving fidelity. Quantitative experiments and user studies demonstrate that the freehand drawing approach reduces user time by approximately 46.7% when generating motions aligned with their imagination. The code, demos, and relevant data are publicly available at https://github.com/InvertedForest/DrawMotion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。