Mogo用单模型实现高质量3D人体动作生成,兼顾长序列与泛化能力。
Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human Motion Generation
- 基于分层残差向量量化与因果变换器,单模型完成高精度动作生成。
- 生成长达260帧(13秒)的连续循环动作,超越现有数据集限制。
- 在分布外生成上表现最佳,适合游戏与多媒体实时应用。
在文本到动作生成领域,当前基于Bert的掩码模型(MoMask、MMM)生成质量优于GPT类自回归模型(T2M-GPT),但缺乏流式输出能力,且在分布外生成上表现较弱。为在保持BERT类模型质量的同时采用GPT类结构,并避免增加额外精修模型以简化数据扩展,本文提出新架构Mogo(Motion Only Generate Once),通过单一Transformer模型生成高质量逼真3D人体动作。Mogo由两部分构成:1)RVQ-VAE,一种分层残差向量量化变分自编码器,可高精度离散化连续动作序列;2)分层因果变换器,以自回归方式生成基础动作序列,同时跨层推断残差。实验表明,Mogo可生成最长达260帧(13秒)的连续循环动作,突破现有HumanML3D数据集196帧(10秒)的长度限制。在HumanML3D测试集上,Mogo获得FID 0.079,优于T2M-GPT(FID=0.116)、AttT2M(FID=0.112)和MMM(FID=0.080)。此外,模型在分布外生成任务中也取得最优定量表现。
原文摘要 · Abstract (English)
In the field of text-to-motion generation, Bert-type Masked Models (MoMask, MMM) currently produce higher-quality outputs compared to GPT-type autoregressive models (T2M-GPT). However, these Bert-type models often lack the streaming output capability required for applications in video game and multimedia environments, a feature inherent to GPT-type models. Additionally, they demonstrate weaker performance in out-of-distribution generation. To surpass the quality of BERT-type models while leveraging a GPT-type structure, without adding extra refinement models that complicate scaling data, we propose a novel architecture, Mogo (Motion Only Generate Once), which generates high-quality lifelike 3D human motions by training a single transformer model. Mogo consists of only two main components: 1) RVQ-VAE, a hierarchical residual vector quantization variational autoencoder, which discretizes continuous motion sequences with high precision; 2) Hierarchical Causal Transformer, responsible for generating the base motion sequences in an autoregressive manner while simultaneously inferring residuals across different layers. Experimental results demonstrate that Mogo can generate continuous and cyclic motion sequences up to 260 frames (13 seconds), surpassing the 196 frames (10 seconds) length limitation of existing datasets like HumanML3D. On the HumanML3D test set, Mogo achieves a FID score of 0.079, outperforming both the GPT-type model T2M-GPT (FID = 0.116), AttT2M (FID = 0.112) and the BERT-type model MMM (FID = 0.080). Furthermore, our model achieves the best quantitative performance in out-of-distribution generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。