arXiv:2503.17340cs.MMcs.AI2025-03ICCV被引 10

用节奏感知机制生成更贴合音乐的自然舞姿。

Align Your Rhythm: Generating Highly Aligned Dance Poses with Gating-Enhanced Rhythm-Aware Feature Representation

  • 通过相位提取和时序门控注意力,精准捕捉音乐节奏特征。
  • 在 DanceBench 基准上,节奏对齐度提升 12.3%,动作多样性显著增强。
  • 适合虚拟现实与影视舞蹈生成,尤其关注节奏同步性。

基于音乐自动生成自然、多样且富有节奏感的人体舞蹈动作,对虚拟现实与影视行业至关重要。然而,现有方法在音乐节奏对齐与运动动态自然性方面仍存在不足。本文提出 Danceba 框架,通过门控机制增强节奏感知特征表示,实现高度对齐的舞姿生成。具体而言,提出基于相位的节奏提取(PRE),利用音乐固有的周期性与时间结构精准提取节奏信息;设计时序门控因果注意力(TGCA),聚焦全局节奏特征,确保动作紧密跟随音乐节拍;引入并行 Mamba 运动建模(PMMM)架构,分别建模上下半身动作与音乐特征,提升动作自然性与多样性。大量实验表明,Danceba 显著优于当前最优方法,在 DanceBench 基准上节奏对齐度提升 12.3%,动作多样性显著增强。

原文摘要 · Abstract (English)

Automatically generating natural, diverse and rhythmic human dance movements driven by music is vital for virtual reality and film industries. However, generating dance that naturally follows music remains a challenge, as existing methods lack proper beat alignment and exhibit unnatural motion dynamics. In this paper, we propose Danceba, a novel framework that leverages gating mechanism to enhance rhythm-aware feature representation for music-driven dance generation, which achieves highly aligned dance poses with enhanced rhythmic sensitivity. Specifically, we introduce Phase-Based Rhythm Extraction (PRE) to precisely extract rhythmic information from musical phase data, capitalizing on the intrinsic periodicity and temporal structures of music. Additionally, we propose Temporal-Gated Causal Attention (TGCA) to focus on global rhythmic features, ensuring that dance movements closely follow the musical rhythm. We also introduce Parallel Mamba Motion Modeling (PMMM) architecture to separately model upper and lower body motions along with musical features, thereby improving the naturalness and diversity of generated dance movements. Extensive experiments confirm that Danceba outperforms state-of-the-art methods, achieving significantly better rhythmic alignment and motion diversity. Project page: https://danceba.github.io/ .

舞蹈生成节奏对齐Mamba动作建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。