用手绘小人图生成3D人体动作,效率提升超一半。
StickMotion: Generating 3D Human Motions by Drawing a Stickman
- 通过手绘小人图实现动作全局与局部控制。
- 相比纯文本生成,手绘可节省51.5%时间。
- 支持动态调整小人位置,动作更自然。
文本到动作生成在从简单文本准确捕捉用户想象的细节动作方面仍具挑战。本文提出StickMotion,一种基于扩散模型的高效多条件网络,可结合传统文本与自研的小人图条件,分别实现对动作的全局与局部控制。针对手绘小人带来的挑战,我们从三方面入手:1)数据生成:开发算法自动适配不同数据集格式生成手绘小人;2)多条件融合:提出融入扩散过程的多条件模块,实现所有条件组合的输出,降低计算复杂度并提升性能;3)动态监督:引入动态监督策略,允许模型在输出序列中微调小人位置,生成更自然的动作。定量实验与用户研究显示,使用手绘小人可节省约51.5%的时间,生成更符合想象的动作。代码、演示与数据将公开,以促进后续研究与验证。
原文摘要 · Abstract (English)
Text-to-motion generation, which translates textual descriptions into human motions, has been challenging in accurately capturing detailed user-imagined motions from simple text inputs. This paper introduces StickMotion, an efficient diffusion-based network designed for multi-condition scenarios, which generates desired motions based on traditional text and our proposed stickman conditions for global and local control of these motions, respectively. We address the challenges introduced by the user-friendly stickman from three perspectives: 1) Data generation. We develop an algorithm to generate hand-drawn stickmen automatically across different dataset formats. 2) Multi-condition fusion. We propose a multi-condition module that integrates into the diffusion process and obtains outputs of all possible condition combinations, reducing computational complexity and enhancing StickMotion's performance compared to conventional approaches with the self-attention module. 3) Dynamic supervision. We empower StickMotion to make minor adjustments to the stickman's position within the output sequences, generating more natural movements through our proposed dynamic supervision strategy. Through quantitative experiments and user studies, sketching stickmen saves users about 51.5% of their time generating motions consistent with their imagination. Our codes, demos, and relevant data will be released to facilitate further research and validation within the scientific community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。