用局部掩码变换器从一段动作生成多样动画,避免过拟合。
MotionDreamer: One-to-Many Motion Synthesis with Localized Generative Masked Transformer
- 基于量化令牌和局部注意力,学习参考动作的内部模式
- 生成动作在真实性和多样性上超越现有GAN与扩散模型方法
- 仅需一个参考动作即可实现编辑、群组动画和节奏对齐生成
生成式掩码变换器在内容生成任务中表现优异,主要得益于其对大规模数据分布的高一致性建模能力。然而在动画领域,大规模数据并不总是可得。将生成式掩码建模应用于单个动捕参考动作生成多样化实例,可能导致过拟合,这一问题尚未被深入研究。本文提出MotionDreamer,一种局部化掩码建模范式,可从任意拓扑结构和时长的动作中学习内部运动模式。通过新颖的分布正则化方法将参考动作嵌入量化令牌,构建稳健且信息丰富的代码本以表示局部运动模式。此外,引入滑动窗口局部注意力机制,使生成动作既自然又多样,且与参考动作高度一致。综合实验表明,MotionDreamer在保真度与多样性上均优于主流的GAN或扩散模型方法。得益于基于量化的方法所具有的稳定性和鲁棒性,MotionDreamer还可有效完成时间编辑、群组动画生成及节拍对齐舞蹈生成等下游任务,仅需单一参考动作。项目页面:https://motiondreamer.github.io/
原文摘要 · Abstract (English)
Generative masked transformers have demonstrated remarkable success across various content generation tasks, primarily due to their ability to effectively model large-scale dataset distributions with high consistency. However, in the animation domain, large datasets are not always available. Applying generative masked modeling to generate diverse instances from a single MoCap reference may lead to overfitting, a challenge that remains unexplored. In this work, we present MotionDreamer, a localized masked modeling paradigm designed to learn internal motion patterns from a given motion with arbitrary topology and duration. By embedding the given motion into quantized tokens with a novel distribution regularization method, MotionDreamer constructs a robust and informative codebook for local motion patterns. Moreover, a sliding window local attention is introduced in our masked transformer, enabling the generation of natural yet diverse animations that closely resemble the reference motion patterns. As demonstrated through comprehensive experiments, MotionDreamer outperforms the state-of-the-art methods that are typically GAN or Diffusion-based in both faithfulness and diversity. Thanks to the consistency and robustness of the quantization-based approach, MotionDreamer can also effectively perform downstream tasks such as temporal motion editing, \textcolor{update}{crowd animation}, and beat-aligned dance generation, all using a single reference motion. Visit our project page: https://motiondreamer.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。