测试九种3D人体姿态模型在遮挡下的表现,发现所有模型都严重退化,尤其是远端关节。
Benchmarking 3D Human Pose Estimation Models under Occlusions
- 用真实检测器噪声模拟遮挡,评估9种主流模型的鲁棒性
- 所有模型在遮挡下性能显著下降,扩散模型反而最差
- 远端关节(手腕、脚踝)普遍脆弱,适合实际应用开发者参考
人体姿态估计(HPE)旨在从视觉数据中检测和定位人体关键点。在3D HPE中,遮挡(身体部分不可见)是准确重建姿态的重大挑战。本文针对现实遮挡条件下3D HPE模型的鲁棒性开展基准测试,涵盖常见遮挡组合。我们评估了九种先进的2D到3D HPE模型,包括卷积、Transformer、图神经网络和扩散模型架构,使用BlendMimic3D合成数据集(含真实2D/3D标注和遮挡标签)。所有模型均在Human3.6M上训练,未重新训练直接测试以评估泛化能力。我们提出一种基于真实检测器行为添加噪声的遮挡模拟协议,并进行全局与逐关节敏感性分析。结果表明,所有模型在遮挡下均出现明显性能下降,扩散模型尽管具有随机性仍表现最差。逐关节分析揭示,远端关节(如手腕、脚)在各类模型中普遍存在脆弱性。本研究揭示了当前3D HPE模型在真实场景中处理遮挡的关键局限,并为提升实际鲁棒性提供依据。
原文摘要 · Abstract (English)
Human Pose Estimation (HPE) involves detecting and localizing keypoints on the human body from visual data. In 3D HPE, occlusions, where parts of the body are not visible in the image, pose a significant challenge for accurate pose reconstruction. This paper presents a benchmark on the robustness of 3D HPE models under realistic occlusion conditions, involving combinations of occluded keypoints commonly observed in real-world scenarios. We evaluate nine state-of-the-art 2D-to-3D HPE models, spanning convolutional, transformer-based, graph-based, and diffusion-based architectures, using the BlendMimic3D dataset, a synthetic dataset with ground-truth 2D/3D annotations and occlusion labels. All models were originally trained on Human3.6M and tested here without retraining to assess their generalization. We introduce a protocol that simulates occlusion by adding noise into 2D keypoints based on real detector behavior, and conduct both global and per-joint sensitivity analyses. Our findings reveal that all models exhibit notable performance degradation under occlusion, with diffusion-based models underperforming despite their stochastic nature. Additionally, a per-joint occlusion analysis identifies consistent vulnerability in distal joints (e.g., wrists, feet) across models. Overall, this work highlights critical limitations of current 3D HPE models in handling occlusions, and provides insights for improving real-world robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。