解决跨数据集姿态估计中的骨骼差异问题,提升模型泛化能力。
PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation
- 用无参数关键点原型在统一空间中融合不同骨骼类型
- 通过跨类型自监督机制实现无需额外标注的训练
- 适用于人体、动物及手部/身体形状估计,迁移效果好
我们研究姿态估计中的多数据集训练(MDT),其中骨骼异质性带来独特挑战,现有方法尚未有效解决。传统任务如回归和分类通常依赖数据合并或多头监督,但姿态估计因骨骼类型多样且跨数据集监督有限,难以整合。为此,我们提出PoseBH框架,通过两项关键技术应对关键点异质性和监督不足:首先,引入非参数关键点原型,在统一嵌入空间中学习,实现跨骨骼类型的无缝集成;其次,设计跨类型自监督机制,将关键点预测与原型对齐,无需教师-学生模型或额外增强即可提供监督。PoseBH显著提升在全身体和动物姿态数据集(包括COCO-WholeBody、AP-10K、APT-36K)上的泛化性能,同时保持在标准人体姿态基准(COCO、MPII、AIC)上的表现。此外,所学关键点嵌入可有效迁移至手部形状估计(InterHand2.6M)和人体形状估计(3DPW)。代码已公开于:https://github.com/uyoung-jeong/PoseBH。
原文摘要 · Abstract (English)
We study multi-dataset training (MDT) for pose estimation, where skeletal heterogeneity presents a unique challenge that existing methods have yet to address. In traditional domains, \eg regression and classification, MDT typically relies on dataset merging or multi-head supervision. However, the diversity of skeleton types and limited cross-dataset supervision complicate integration in pose estimation. To address these challenges, we introduce PoseBH, a new MDT framework that tackles keypoint heterogeneity and limited supervision through two key techniques. First, we propose nonparametric keypoint prototypes that learn within a unified embedding space, enabling seamless integration across skeleton types. Second, we develop a cross-type self-supervision mechanism that aligns keypoint predictions with keypoint embedding prototypes, providing supervision without relying on teacher-student models or additional augmentations. PoseBH substantially improves generalization across whole-body and animal pose datasets, including COCO-WholeBody, AP-10K, and APT-36K, while preserving performance on standard human pose benchmarks (COCO, MPII, and AIC). Furthermore, our learned keypoint embeddings transfer effectively to hand shape estimation (InterHand2.6M) and human body shape estimation (3DPW). The code for PoseBH is available at: https://github.com/uyoung-jeong/PoseBH.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。