建模多人互动中的相互关系，提升2D姿态转3D的准确性。
MuPPet: Multi-person 2D-to-3D Pose Lifting
- 用人物编码和动态注意力显式建模多人间关系
- 在多人交互数据集上显著超越现有方法
- 适合需要精准社交行为分析的场景
多人社交互动本质上依赖于群体内所有个体之间的协调与关系，因此多人定位与身体姿态估计对于理解这些社交动态至关重要。一种有前景的方法是2D到3D姿态提升，该方法通过利用2D姿态估计的重大进展，生成包含丰富空间细节的3D人体姿态。然而，现有2D到3D姿态提升方法常忽视人与人之间的关系，或无法处理不同规模的群体，限制了其在多人群场景下的有效性。本文提出MuPPet，一种新型多人群2D到3D姿态提升框架，显式建模人与人之间的相关性。为利用这种人际依赖，我们的方法引入人物编码（Person Encoding）以结构化个体表示，排列增强（Permutation Augmentation）以提升训练多样性，并动态多人注意力（Dynamic Multi-Person Attention）以自适应建模个体间的关联。在群体互动数据集上的大量实验表明，MuPPet显著优于当前最先进的单人及多人2D到3D姿态提升方法，并在遮挡场景中表现出更强鲁棒性。研究结果凸显了建模人际相关性的重要性，为准确且具有社交意识的3D姿态估计铺平了道路。代码已开源：https://github.com/Thomas-Markhorst/MuPPet
原文摘要 · Abstract (English)
Multi-person social interactions are inherently built on coherence and relationships among all individuals within the group, making multi-person localization and body pose estimation essential to understanding these social dynamics. One promising approach is 2D-to-3D pose lifting which provides a 3D human pose consisting of rich spatial details by building on the significant advances in 2D pose estimation. However, the existing 2D-to-3D pose lifting methods often neglect inter-person relationships or cannot handle varying group sizes, limiting their effectiveness in multi-person settings. We propose MuPPet, a novel multi-person 2D-to-3D pose lifting framework that explicitly models inter-person correlations. To leverage these inter-person dependencies, our approach introduces Person Encoding to structure individual representations, Permutation Augmentation to enhance training diversity, and Dynamic Multi-Person Attention to adaptively model correlations between individuals. Extensive experiments on group interaction datasets demonstrate MuPPet significantly outperforms state-of-the-art single- and multi-person 2D-to-3D pose lifting methods, and improves robustness in occlusion scenarios. Our findings highlight the importance of modeling inter-person correlations, paving the way for accurate and socially-aware 3D pose estimation. Our code is available at: https://github.com/Thomas-Markhorst/MuPPet
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。