用轻量模型微调实现动物姿态自动标注，省时高效。
Animal Pose Labeling Using General-Purpose Point Trackers
- 在测试时优化模型，用少量标注帧微调通用追踪器
- 在多个数据集上达到当前最好效果，标注成本低
- 适合需快速标注动物行为的研究者使用
从视频中自动估计动物姿态对研究动物行为至关重要。现有方法可靠性不足，因其训练数据集未能充分覆盖所有动物行为，而全面数据集的收集因动物形态差异大而极具挑战。本文提出一种动物姿态标注流程，采用测试时优化策略：给定视频后，在少量标注帧上微调预训练通用点追踪器中的轻量外观嵌入；这些标注可来自人工标注或现成的姿态检测器。微调后的模型用于其余帧的自动标注。该方法在合理标注成本下达到当前最优性能，为动物行为自动化量化提供了有力工具。
原文摘要 · Abstract (English)
Automatically estimating animal poses from videos is important for studying animal behaviors. Existing methods do not perform reliably since they are trained on datasets that are not comprehensive enough to capture all necessary animal behaviors. However, it is very challenging to collect such datasets due to the large variations in animal morphology. In this paper, we propose an animal pose labeling pipeline that follows a different strategy, i.e. test time optimization. Given a video, we fine-tune a lightweight appearance embedding inside a pre-trained general-purpose point tracker on a sparse set of annotated frames. These annotations can be obtained from human labelers or off-the-shelf pose detectors. The fine-tuned model is then applied to the rest of the frames for automatic labeling. Our method achieves state-of-the-art performance at a reasonable annotation cost. We believe our pipeline offers a valuable tool for the automatic quantification of animal behavior. Visit our project webpage at https://zhuoyang-pan.github.io/animal-labeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。