不依赖纹理的鸟类3D关节点估计算法，支持多鸟跟踪
Towards Texture- And Shape-Independent 3D Keypoint Estimation in Birds
- 用轮廓分割替代纹理识别，实现无纹理依赖的2D关键点定位
- 在鸽子上达到与原方法相当的3D姿态估计精度
- 可直接用于其他四种鸟类，无需额外调参
本文提出一种不依赖纹理的3D关节点估计算法，用于多只鸽子的3D姿态估计与跟踪。基于现有的3D-MuPPET框架，该方法通过分割生成个体轮廓，利用轮廓估计2D关键点，再通过三角测量获得3D姿态。身份匹配在首帧完成，后续帧中在2D空间跟踪。所提方法在鸽子上的性能与原始依赖纹理的3D-MuPPET相当。此外，我们未对模型进行微调，直接应用于四种其他鸟类，初步结果令人鼓舞。表明该方法具备良好的泛化能力，为更鲁棒的无纹理姿态估计框架提供了坚实基础。
原文摘要 · Abstract (English)
In this paper, we present a texture-independent approach to estimate and track 3D joint positions of multiple pigeons. For this purpose, we build upon the existing 3D-MuPPET framework, which estimates and tracks the 3D poses of up to 10 pigeons using a multi-view camera setup. We extend this framework by using a segmentation method that generates silhouettes of the individuals, which are then used to estimate 2D keypoints. Following 3D-MuPPET, these 2D keypoints are triangulated to infer 3D poses, and identities are matched in the first frame and tracked in 2D across subsequent frames. Our proposed texture-independent approach achieves comparable accuracy to the original texture-dependent 3D-MuPPET framework. Additionally, we explore our approach's applicability to other bird species. To do that, we infer the 2D joint positions of four bird species without additional fine-tuning the model trained on pigeons and obtain preliminary promising results. Thus, we think that our approach serves as a solid foundation and inspires the development of more robust and accurate texture-independent pose estimation frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。