通过重建监督提升文本到动作生成的准确性和稳定性
Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation
- 引入动作潜在空间作为中间监督,联合训练重建分支
- 在多个数据集上达到当前最优性能,动作生成更自然
- 适合关注动作生成质量与稳定性的研究者
扩散模型因其强大的生成能力和灵活性,在文本驱动的人体动作生成任务中广泛应用。然而,现有动作扩散模型存在两大局限:预训练文本编码器缺乏动作特定信息导致表征差距,以及迭代去噪过程中的误差传播。本文提出重建锚定扩散模型(RAM)以解决这些问题。首先,RAM利用动作潜在空间作为中间监督,联合训练一个动作重建分支,包含自正则化以增强动作空间区分性,以及以动作为中心的潜在对齐,实现从文本到动作潜在空间的精准映射。其次,提出重建误差引导(REG)机制,在测试阶段利用扩散模型的自我修正能力缓解误差传播。每一步去噪中,REG通过重建分支重构前一估计,重现先验误差模式,通过放大当前预测与重构估计之间的残差,突出当前预测的改进。大量实验表明,RAM显著提升性能并达到当前最优水平。代码将公开。
原文摘要 · Abstract (English)
Diffusion models have seen widespread adoption for text-driven human motion generation and related tasks due to their impressive generative capabilities and flexibility. However, current motion diffusion models face two major limitations: a representational gap caused by pre-trained text encoders that lack motion-specific information, and error propagation during the iterative denoising process. This paper introduces Reconstruction-Anchored Diffusion Model (RAM) to address these challenges. First, RAM leverages a motion latent space as intermediate supervision for text-to-motion generation. To this end, RAM co-trains a motion reconstruction branch with two key objective functions: self-regularization to enhance the discrimination of the motion space and motion-centric latent alignment to enable accurate mapping from text to the motion latent space. Second, we propose Reconstructive Error Guidance (REG), a testing-stage guidance mechanism that exploits the motion diffusion model's inherent self-correction ability to mitigate error propagation. At each denoising step, REG uses the motion reconstruction branch to reconstruct the previous estimate, reproducing the prior error patterns. By amplifying the residual between the current prediction and the reconstructed estimate, REG highlights the improvements in the current prediction. Extensive experiments demonstrate that RAM achieves significant improvements and state-of-the-art performance. Our code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。