用深度图指导图像关键点检测，自监督下更准更稳。
Depth-Guided Self-Supervised Human Keypoint Detection via Cross-Modal Distillation
- 通过深度图与图片跨模态蒸馏，引导模型学习关键点。
- 在Human3.6M上误差降低47.15%，其他数据集也显著提升。
- 适合做无标注人体姿态估计的研究者参考。
现有无监督关键点检测方法通常对图像施加人工形变（如遮挡部分图像），并以重建原图作为学习目标，但缺乏深度信息，常将关键点误检在背景上。为此，我们提出Distill-DKP，一种新颖的跨模态知识蒸馏框架，利用深度图与RGB图像在自监督设置下进行关键点检测。训练时，从基于深度图的教师模型中提取嵌入层知识，指导基于图像的学生模型；推理仅使用学生模型。实验表明，Distill-DKP在Human3.6M上将均方误差降低47.15%，在Taichi上平均误差减少5.67%，在DeepFashion上关键点准确率提升1.3%。详细消融实验验证了网络不同层间知识蒸馏的有效性。
原文摘要 · Abstract (English)
Existing unsupervised keypoint detection methods apply artificial deformations to images such as masking a significant portion of images and using reconstruction of original image as a learning objective to detect keypoints. However, this approach lacks depth information in the image and often detects keypoints on the background. To address this, we propose Distill-DKP, a novel cross-modal knowledge distillation framework that leverages depth maps and RGB images for keypoint detection in a self-supervised setting. During training, Distill-DKP extracts embedding-level knowledge from a depth-based teacher model to guide an image-based student model with inference restricted to the student. Experiments show that Distill-DKP significantly outperforms previous unsupervised methods by reducing mean L2 error by 47.15% on Human3.6M, mean average error by 5.67% on Taichi, and improving keypoints accuracy by 1.3% on DeepFashion dataset. Detailed ablation studies demonstrate the sensitivity of knowledge distillation across different layers of the network. Project Page: https://23wm13.github.io/distill-dkp/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。