用专家模型灵活捕捉多人运动的时空依赖,速度提升3.6倍且参数减少41%。
Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Prediction
- 设计四类时空专家,自适应挖掘复杂运动模式。
- 引入双向时空Mamba结构,实现高效参数共享与计算压缩。
- 在四个数据集上精度超前沿方法,训练速度提升3.6倍。
全面而灵活地捕捉人体运动中的复杂时空依赖关系,对多人运动预测至关重要。现有方法面临两大局限:一是依赖位置编码进行时空建模,导致表示僵化;二是传统注意力机制存在二次时间复杂度,计算开销高。为此,本文提出时空无束缚专家混合模型(ST-MoE),可灵活探索复杂时空依赖,并显著降低计算成本。模型集成四类不同类型的时空专家,分别专注于捕捉空间或时间上的特定依赖关系。为降低多专家融合带来的计算负担,引入双向时空Mamba作为专家模块,通过不同组合共享双向时序与空间信息,实现模型效率与参数经济性。在四个多人运动基准数据集上的大量实验表明,该方法不仅在精度上超越现有最先进水平,还使模型参数减少41.38%,训练速度提升3.6倍。代码已公开于https://github.com/alanyz106/ST-MoE。
原文摘要 · Abstract (English)
Comprehensively and flexibly capturing the complex spatio-temporal dependencies of human motion is critical for multi-person motion prediction. Existing methods grapple with two primary limitations: i) Inflexible spatiotemporal representation due to reliance on positional encodings for capturing spatiotemporal information. ii) High computational costs stemming from the quadratic time complexity of conventional attention mechanisms. To overcome these limitations, we propose the Spatiotemporal-Untrammelled Mixture of Experts (ST-MoE), which flexibly explores complex spatio-temporal dependencies in human motion and significantly reduces computational cost. To adaptively mine complex spatio-temporal patterns from human motion, our model incorporates four distinct types of spatiotemporal experts, each specializing in capturing different spatial or temporal dependencies. To reduce the potential computational overhead while integrating multiple experts, we introduce bidirectional spatiotemporal Mamba as experts, each sharing bidirectional temporal and spatial Mamba in distinct combinations to achieve model efficiency and parameter economy. Extensive experiments on four multi-person benchmark datasets demonstrate that our approach not only outperforms state-of-art in accuracy but also reduces model parameter by 41.38% and achieves a 3.6x speedup in training. The code is available at https://github.com/alanyz106/ST-MoE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。