用统一图结构联合建模时空特征,提升多人运动预测的连贯性。
UnityGraph: Unified Learning of Spatio-temporal features for Multi-person Motion Prediction
- 构建超图结构,将时空特征视为整体统一建模。
- 在多个数据集上达到当前最优性能,验证方法有效性。
- 适合需要高连贯性多人动作预测的场景,如智能监控、体育分析。
多人运动预测是具有重要现实应用的新兴领域。现有最先进方法通常采用双路网络分别建模空间与时间特征,但二者兼容性不确定,导致时空特征融合困难,违背人类运动的时空一致性和耦合性。为此,本文提出新型图结构UnityGraph,将时空特征作为一个整体进行建模,增强模型的内在一致性与耦合性。具体而言,UnityGraph是一种基于超图的网络,将观测到的运动视为图节点,并通过超边连接这些节点以探索时空特征。该视角统一处理时空动态,将多人运动预测重构为单一图上的问题。基于此超图的动态消息传递机制,模型可同时学习两类关系,生成反映节点间相关性的目标消息。在多个数据集上的大量实验表明,本方法实现了当前最优性能,证实了其有效性和创新设计。
原文摘要 · Abstract (English)
Multi-person motion prediction is a complex and emerging field with significant real-world applications. Current state-of-the-art methods typically adopt dual-path networks to separately modeling spatial features and temporal features. However, the uncertain compatibility of the two networks brings a challenge for spatio-temporal features fusion and violate the spatio-temporal coherence and coupling of human motions by nature. To address this issue, we propose a novel graph structure, UnityGraph, which treats spatio-temporal features as a whole, enhancing model coherence and coupling.spatio-temporal features as a whole, enhancing model coherence and coupling. Specifically, UnityGraph is a hypervariate graph based network. The flexibility of the hypergraph allows us to consider the observed motions as graph nodes. We then leverage hyperedges to bridge these nodes for exploring spatio-temporal features. This perspective considers spatio-temporal dynamics unitedly and reformulates multi-person motion prediction into a problem on a single graph. Leveraging the dynamic message passing based on this hypergraph, our model dynamically learns from both types of relations to generate targeted messages that reflect the relevance among nodes. Extensive experiments on several datasets demonstrates that our method achieves state-of-the-art performance, confirming its effectiveness and innovative design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。