让舞蹈更符合音乐风格,生成动作更连贯自然。
MEGADance: Mixture-of-Experts Architecture for Genre-Aware 3D Dance Generation
- 分两阶段生成:先压缩动作数据,再按音乐风格精准生成
- 在FineDance和AIST++上达到当前最好效果
- 适合需要控制舞蹈风格的创作者和虚拟演出应用
音乐驱动的3D舞蹈生成近年来受到广泛关注,应用于编舞、虚拟现实和创意内容创作。现有方法虽能从音频生成逼真动作,但常将音乐风格视为次要修饰,而非核心语义驱动力,导致音乐与动作不同步,尤其在复杂节奏转换时破坏风格连续性,影响视觉效果。为此,我们提出MEGADance,一种新型音乐驱动3D舞蹈生成架构。通过将编舞一致性解耦为舞蹈通用性与风格特异性,实现高质量舞蹈生成与强风格可控性。该架构包含两个阶段:(1) 高保真舞蹈量化阶段(HFDQ),利用有限标量量化(FSQ)将舞蹈动作编码为潜在表示,并施加运动学-动力学约束进行重建;(2) 风格感知舞蹈生成阶段(GADG),通过混合专家(MoE)机制与Mamba-Transformer混合骨干网络,将音乐映射至潜在空间。在FineDance和AIST++数据集上的大量实验表明,MEGADance在定性和定量评价上均达到当前最优水平。代码已开源于https://github.com/XulongT/MEGADance。
原文摘要 · Abstract (English)
Music-driven 3D dance generation has attracted increasing attention in recent years, with promising applications in choreography, virtual reality, and creative content creation. Previous research has generated promising realistic dance movement from audio signals. However, traditional methods underutilize genre conditioning, often treating it as auxiliary modifiers rather than core semantic drivers. This oversight compromises music-motion synchronization and disrupts dance genre continuity, particularly during complex rhythmic transitions, thereby leading to visually unsatisfactory effects. To address the challenge, we propose MEGADance, a novel architecture for music-driven 3D dance generation. By decoupling choreographic consistency into dance generality and genre specificity, MEGADance demonstrates significant dance quality and strong genre controllability. It consists of two stages: (1) High-Fidelity Dance Quantization Stage (HFDQ), which encodes dance motions into a latent representation by Finite Scalar Quantization (FSQ) and reconstructs them with kinematic-dynamic constraints, and (2) Genre-Aware Dance Generation Stage (GADG), which maps music into the latent representation by synergistic utilization of Mixture-of-Experts (MoE) mechanism with Mamba-Transformer hybrid backbone. Extensive experiments on the FineDance and AIST++ dataset demonstrate the state-of-the-art performance of MEGADance both qualitatively and quantitatively. Code is available at https://github.com/XulongT/MEGADance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。