用Mamba框架高效生成人类互动动作,速度更快、参数更少。
InterMamba: Efficient Human-Human Interaction Generation with Adaptive Spatio-Temporal Mamba
- 采用双并行状态空间模型,自适应融合时空特征。
- 参数仅6600万,推理速度达0.57秒/次,为基线的46%。
- 适合需要实时反馈的动作生成场景,如虚拟人交互。
人类互动生成在运动合成中受到广泛关注,因其对理解人类社会行为至关重要。然而,现有方法多基于Transformer架构,常面临可扩展性和效率问题。为此,我们提出一种基于Mamba框架的新型高效人类互动生成方法,旨在有效捕捉长序列依赖关系并支持实时反馈。具体而言,设计了一种自适应时空Mamba框架,包含两条并行的SSM分支与自适应机制,以整合运动序列的时空特征。为进一步增强个体运动序列内部及不同序列间的依赖建模能力,开发了自适应时空Mamba模块与跨序列自适应时空Mamba模块,实现高效特征学习。大量实验表明,该方法在两个互动数据集上均达到领先性能,相比基线方法InterGen,精度提升且参数量仅需6600万(仅为InterGen的36%),平均推理速度达0.57秒,为InterGen的46%。
原文摘要 · Abstract (English)
Human-human interaction generation has garnered significant attention in motion synthesis due to its vital role in understanding humans as social beings. However, existing methods typically rely on transformer-based architectures, which often face challenges related to scalability and efficiency. To address these issues, we propose a novel, efficient human-human interaction generation method based on the Mamba framework, designed to meet the demands of effectively capturing long-sequence dependencies while providing real-time feedback. Specifically, we introduce an adaptive spatio-temporal Mamba framework that utilizes two parallel SSM branches with an adaptive mechanism to integrate the spatial and temporal features of motion sequences. To further enhance the model's ability to capture dependencies within individual motion sequences and the interactions between different individual sequences, we develop two key modules: the self-adaptive spatio-temporal Mamba module and the cross-adaptive spatio-temporal Mamba module, enabling efficient feature learning. Extensive experiments demonstrate that our method achieves state-of-the-art results on two interaction datasets with remarkable quality and efficiency. Compared to the baseline method InterGen, our approach not only improves accuracy but also requires a minimal parameter size of just 66M ,only 36% of InterGen's, while achieving an average inference speed of 0.57 seconds, which is 46% of InterGen's execution time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。