用潜在空间GAN实现快速高质量文本驱动动作生成
LS-GAN: Human Motion Synthesis with Latent-space GANs
- 在潜在空间使用GAN，替代复杂扩散模型
- 在HumanML3D上达FID 0.482，计算量减少91%以上
- 适合需要实时生成的元宇宙、游戏等场景
文本驱动的3D人体动作生成近年来受到广泛关注，应用于游戏、影视制作和虚拟现实等领域。该任务根据文本输入生成对应的动作序列。尽管已有研究采用原始动作数据或潜在空间表示结合扩散模型进行生成，但这些方法通常存在训练和推理时间过长的问题。本文提出一种新框架，利用潜在空间中的生成对抗网络（GAN）实现更高效的训练与推理，性能媲美当前最先进的扩散模型。我们在HumanML3D和HumanAct12数据集上进行了实验，结果表明，一个结构简单的潜在空间GAN在保持高质量输出的同时，相比潜在扩散模型实现了超过91%的浮点运算量（FLOPs）降低，FID达到0.482。本工作为高效且高质量的动作合成开辟了新路径。
原文摘要 · Abstract (English)
Human motion synthesis conditioned on textual input has gained significant attention in recent years due to its potential applications in various domains such as gaming, film production, and virtual reality. Conditioned Motion synthesis takes a text input and outputs a 3D motion corresponding to the text. While previous works have explored motion synthesis using raw motion data and latent space representations with diffusion models, these approaches often suffer from high training and inference times. In this paper, we introduce a novel framework that utilizes Generative Adversarial Networks (GANs) in the latent space to enable faster training and inference while achieving results comparable to those of the state-of-the-art diffusion methods. We perform experiments on the HumanML3D, HumanAct12 benchmarks and demonstrate that a remarkably simple GAN in the latent space achieves a FID of 0.482 with more than 91% in FLOPs reduction compared to latent diffusion model. Our work opens up new possibilities for efficient and high-quality motion synthesis using latent space GANs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。