将每个关节独立量化为2D令牌图，实现更精准的动作生成。
MoGenTS: Motion Generation based on Spatial-Temporal Joint Modeling
- 按关节独立量化，构建2D令牌图以保留空间时序结构
- 在HumanML3D上FID降低26.6%，KIT-ML上降低29.9%
- 适合需要精细动作控制的3D角色动画生成场景
离散量化生成动作相比连续回归具有诸多优势，但存在不可避免的近似误差。以往方法通常将全身姿态统一编码为一个码字，不仅难以在单个向量中表示所有关节，还丢失了关节间的空间关系。本文提出将每个关节分别量化为一个向量，实现了：一、简化量化过程，单关节复杂度显著低于全身体态；二、保持关节间空间关系与运动时序模式；三、生成2D令牌图，可直接应用图像领域的多种2D操作。基于此2D量化基础，构建时空联合建模框架，提出2D关节VQVAE、时空2D掩码机制和时空2D注意力，充分挖掘2D令牌间的时空信号。大量实验表明，本方法在多个数据集上显著优于现有方法，在HumanML3D上FID降低26.6%，在KIT-ML上降低29.9%。
原文摘要 · Abstract (English)
Motion generation from discrete quantization offers many advantages over continuous regression, but at the cost of inevitable approximation errors. Previous methods usually quantize the entire body pose into one code, which not only faces the difficulty in encoding all joints within one vector but also loses the spatial relationship between different joints. Differently, in this work we quantize each individual joint into one vector, which i) simplifies the quantization process as the complexity associated with a single joint is markedly lower than that of the entire pose; ii) maintains a spatial-temporal structure that preserves both the spatial relationships among joints and the temporal movement patterns; iii) yields a 2D token map, which enables the application of various 2D operations widely used in 2D images. Grounded in the 2D motion quantization, we build a spatial-temporal modeling framework, where 2D joint VQVAE, temporal-spatial 2D masking technique, and spatial-temporal 2D attention are proposed to take advantage of spatial-temporal signals among the 2D tokens. Extensive experiments demonstrate that our method significantly outperforms previous methods across different datasets, with a 26.6% decrease of FID on HumanML3D and a 29.9% decrease on KIT-ML. Project page: https://aigc3d.github.io/mogents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。