用Mamba模型提升动作生成的时间对齐精度
Learning Human Motion with Temporally Conditional Mamba
- 将条件信息融入Mamba块的循环动态，实现逐帧对齐
- 在多个任务上显著提升动作真实感和条件一致性
- 适合需要精确时序控制的动作生成场景
基于时间依赖输入信号学习人体运动是一项具有挑战性且意义重大的任务，其目标是生成或估计能准确反映条件输入时间模式的人体动作。现有方法通常依赖交叉注意力机制融合条件与运动信息，但该方法主要捕捉全局交互，难以保持逐步的时间对齐。为解决这一问题，我们提出基于Mamba的时序条件模型（Temporally Conditional Mamba），将条件信息嵌入Mamba块的递归动态中，从而实现更优的时间对齐动作生成。我们在多种人体动作任务上验证了该方法的有效性。大量实验表明，相比当前最优方法，本模型在时间对齐、动作真实性和条件一致性方面均有显著提升。
原文摘要 · Abstract (English)
Learning human motion based on a time-dependent input signal presents a challenging yet impactful task with various applications. The goal of this task is to generate or estimate human movement that consistently reflects the temporal patterns of conditioning inputs. Existing methods typically rely on cross-attention mechanisms to fuse the condition with motion. However, this approach primarily captures global interactions and struggles to maintain step-by-step temporal alignment. To address this limitation, we introduce Temporally Conditional Mamba, a new mamba-based model for human motion generation. Our approach integrates conditional information into the recurrent dynamics of the Mamba block, enabling better temporally aligned motion. To validate the effectiveness of our method, we evaluate it on a variety of human motion tasks. Extensive experiments demonstrate that our model significantly improves temporal alignment, motion realism, and condition consistency over state-of-the-art approaches. Our project page is available at https://zquang2202.github.io/TCM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。