arXiv:2502.18214cs.CV2025-02IJCV被引 8

提出关键点交互变换器,提升通用哺乳动物姿态估计的泛化能力。

Learning Structure-Supporting Dependencies via Keypoint Interactive Transformer for General Mammal Pose Estimation

  • 通过关键点聚类生成实例级提示,融合外观上下文与结构关系
  • 设计无空间分割的交互式变换器,自适应调整不同关键点权重
  • 在多个物种上实现更鲁棒的姿态估计,适合跨物种行为分析

通用哺乳动物姿态估计是计算机视觉中的重要且具有挑战性的任务,对理解真实场景中哺乳动物行为至关重要。然而现有研究仍处于初级阶段,仅针对少数特定物种开展。从特定物种到通用哺乳动物姿态估计的核心难点在于不同物种间巨大的外观和姿态差异。我们认为,在给定外观上下文、实例级先验以及关键点之间的结构关系可作为互补证据。为此,我们提出关键点交互变换器(KIT),用于学习通用哺乳动物姿态估计中的实例级结构支持依赖关系。KITPose由两个耦合组件构成:第一部分提取关键点特征并生成身体部位提示,使用专门设计的泛化热图回归损失(GHRL)进行监督;不引入外部视觉或文本提示,而是通过关键点聚类生成身体部位偏置,与图像上下文对齐以生成对应实例级提示。第二部分提出一种新型交互式变换器,以特征切片为输入令牌,无需进行空间分割。此外,为增强模型能力,设计了自适应权重策略,解决不同关键点间的不平衡问题。

原文摘要 · Abstract (English)

General mammal pose estimation is an important and challenging task in computer vision, which is essential for understanding mammal behaviour in real-world applications. However, existing studies are at their preliminary research stage, which focus on addressing the problem for only a few specific mammal species. In principle, from specific to general mammal pose estimation, the biggest issue is how to address the huge appearance and pose variances for different species. We argue that given appearance context, instance-level prior and the structural relation among keypoints can serve as complementary evidence. To this end, we propose a Keypoint Interactive Transformer (KIT) to learn instance-level structure-supporting dependencies for general mammal pose estimation. Specifically, our KITPose consists of two coupled components. The first component is to extract keypoint features and generate body part prompts. The features are supervised by a dedicated generalised heatmap regression loss (GHRL). Instead of introducing external visual/text prompts, we devise keypoints clustering to generate body part biases, aligning them with image context to generate corresponding instance-level prompts. Second, we propose a novel interactive transformer that takes feature slices as input tokens without performing spatial splitting. In addition, to enhance the capability of the KIT model, we design an adaptive weight strategy to address the imbalance issue among different keypoints.

姿态估计关键点交互通用哺乳动物变换器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。