arXiv:2602.01352cs.CV2026-02

解决文本生成动作的长期漂移和语义波动问题

T2M Mamba: Motion Periodicity-Saliency Coupling Approach for Stable Text-Driven Motion Generation

  • 引入周期性与关键帧显著性耦合机制,提升长序列稳定性
  • 在HumanML3D和KIT-ML上实现FID 0.068,指标全面领先
  • 适合需要高鲁棒性的虚拟人动画与机器人交互场景

文本到动作生成可将动作语言描述转化为连贯的3D人体动作序列,在虚拟人动画与人形机器人交互等领域备受关注。现有模型虽在保真度上取得进展,但仍存在两大核心缺陷:(i) 将动作周期性与关键帧显著性视为独立因素,忽略其耦合关系,导致长序列生成出现漂移;(ii) 对语义等价的改写敏感,微小同义词替换会扭曲文本嵌入,经解码器传播后引发动作不稳定或错误。本文提出T2M Mamba,通过:(i) 设计周期性-显著性感知的Mamba架构,利用改进的密度峰值聚类进行关键帧权重估计,并通过快速傅里叶变换加速自相关计算实现周期性检测,以低计算开销捕捉耦合动态;(ii) 构建周期性差异跨模态对齐模块(PDCAM),增强文本与动作嵌入的鲁棒对齐。在HumanML3D和KIT-ML数据集上的大量实验验证了该方法的有效性,达到FID 0.068,且所有其他指标均有稳定提升。

原文摘要 · Abstract (English)

Text-to-motion generation, which converts motion language descriptions into coherent 3D human motion sequences, has attracted increasing attention in fields, such as avatar animation and humanoid robotic interaction. Though existing models have achieved significant fidelity, they still suffer from two core limitations: (i) They treat motion periodicity and keyframe saliency as independent factors, overlooking their coupling and causing generation drift in long sequences. (ii) They are fragile to semantically equivalent paraphrases, where minor synonym substitutions distort textual embeddings, propagating through the decoder and producing unstable or erroneous motions. In this work, we propose T2M Mamba to address these limitations by (i) proposing Periodicity-Saliency Aware Mamba, which utilizes novel algorithms for keyframe weight estimation via enhanced Density Peaks Clustering and motion periodicity estimation via FFT-accelerated autocorrelation to capture coupled dynamics with minimal computational overhead, and (ii) constructing a Periodic Differential Cross-modal Alignment Module (PDCAM) to enhance robust alignment of textual and motion embeddings. Extensive experiments on HumanML3D and KIT-ML datasets have been conducted, confirming the effectiveness of our approach, achieving an FID of 0.068 and consistent gains on all other metrics.

动作生成文本驱动周期性建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。