用分层自回归生成3D人体动作，10步完成且质量领先。
MoSa: Motion Generation with Scalable Autoregressive Modeling
- 分层预测多尺度动作令牌，实现高效渐进生成。
- 在Motion-X上FID达0.06，推理速度提升27%。
- 支持动作编辑无需微调，适合动画与交互应用。
我们提出MoSa，一种新型分层文本驱动3D人体动作生成框架，通过粗到细的可扩展生成过程增强向量量化引导生成变压器（VQ-GT）范式。MoSa引入多尺度令牌保留策略（MTPS），结合分层残差向量量化变分自编码器（RQ-VAE），在每层量化中采用插值保留粗到细的多尺度令牌。由此，生成变压器支持可扩展自回归（SAR）建模，可同时预测多尺度令牌，而非传统方法仅每步预测一个令牌。因此，MoSa仅需10次推理步骤，与RQ-VAE量化层数一致。为缓解频繁插值带来的重建退化，我们提出轻量但表达力强的卷积-注意力混合VQ-VAE（CAQ-VAE），优化残差块设计并引入注意力机制以更好捕捉全局依赖。大量实验表明，MoSa在生成质量与效率上均达到最先进水平，在Motion-X数据集上FID为0.06（对比MoMask的0.20），推理时间减少27%。此外，MoSa能良好泛化至下游任务如动作编辑，无需额外微调。代码已公开于https://mosa-web.github.io/MoSa-web
原文摘要 · Abstract (English)
We introduce MoSa, a novel hierarchical motion generation framework for text-driven 3D human motion generation that enhances the Vector Quantization-guided Generative Transformers (VQ-GT) paradigm through a coarse-to-fine scalable generation process. In MoSa, we propose a Multi-scale Token Preservation Strategy (MTPS) integrated into a hierarchical residual vector quantization variational autoencoder (RQ-VAE). MTPS employs interpolation at each hierarchical quantization to effectively retain coarse-to-fine multi-scale tokens. With this, the generative transformer supports Scalable Autoregressive (SAR) modeling, which predicts scale tokens, unlike traditional methods that predict only one token at each step. Consequently, MoSa requires only 10 inference steps, matching the number of RQ-VAE quantization layers. To address potential reconstruction degradation from frequent interpolation, we propose CAQ-VAE, a lightweight yet expressive convolution-attention hybrid VQ-VAE. CAQ-VAE enhances residual block design and incorporates attention mechanisms to better capture global dependencies. Extensive experiments show that MoSa achieves state-of-the-art generation quality and efficiency, outperforming prior methods in both fidelity and speed. On the Motion-X dataset, MoSa achieves an FID of 0.06 (versus MoMask's 0.20) while reducing inference time by 27 percent. Moreover, MoSa generalizes well to downstream tasks such as motion editing, requiring no additional fine-tuning. The code is available at https://mosa-web.github.io/MoSa-web
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。