用扩散模型+语言提示,让动物姿态估计更准更鲁棒。
DiffPose-Animal: A Language-Conditioned Diffusion Framework for Animal Pose Estimation
- 将姿态估计转为扩散去噪过程,结合语言模型提供物种级语义引导。
- 在多个数据集上超越传统方法,尤其在遮挡和标注稀疏时表现优异。
- 适合生态监测、畜牧管理等需要跨物种泛化的场景。
动物姿态估计是计算机视觉中的基础任务,在生态监测、行为分析和智能畜牧管理中日益重要。与人体姿态估计相比,动物姿态估计更具挑战性,源于物种间形态差异大、身体结构复杂以及标注数据有限。本文提出 DiffPose-Animal,一种基于扩散模型的自顶向下动物姿态估计框架。不同于传统热图回归方法,DiffPose-Animal 将姿态估计重构为生成式扩散框架下的去噪过程。为增强关键点生成中的语义引导,我们利用大语言模型(LLMs)根据物种特异性提示提取全局解剖先验和局部关键点语义,并通过交叉注意力模块将这些文本先验编码并与图像特征融合,为去噪过程提供生物合理的约束。此外,设计了基于扩散的关键点解码器,逐步优化姿态预测,提升对遮挡和标注稀疏的鲁棒性。在多个公开动物姿态数据集上的大量实验表明,该方法在多样物种、杂乱背景和关键点不完整等挑战场景下均展现出优异的有效性和泛化能力。
原文摘要 · Abstract (English)
Animal pose estimation is a fundamental task in computer vision, with growing importance in ecological monitoring, behavioral analysis, and intelligent livestock management. Compared to human pose estimation, animal pose estimation is more challenging due to high interspecies morphological diversity, complex body structures, and limited annotated data. In this work, we introduce DiffPose-Animal, a novel diffusion-based framework for top-down animal pose estimation. Unlike traditional heatmap regression methods, DiffPose-Animal reformulates pose estimation as a denoising process under the generative framework of diffusion models. To enhance semantic guidance during keypoint generation, we leverage large language models (LLMs) to extract both global anatomical priors and local keypoint-wise semantics based on species-specific prompts. These textual priors are encoded and fused with image features via cross-attention modules to provide biologically meaningful constraints throughout the denoising process. Additionally, a diffusion-based keypoint decoder is designed to progressively refine pose predictions, improving robustness to occlusion and annotation sparsity. Extensive experiments on public animal pose datasets demonstrate the effectiveness and generalization capability of our method, especially under challenging scenarios with diverse species, cluttered backgrounds, and incomplete keypoints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。