arXiv:2511.13208cs.CV2025-11AAAI被引 4

端到端视频多人姿态估计,用注意力机制解决跨帧人物关联难题。

End-to-End Multi-Person Pose Estimation with Pose-Aware Video Transformer

论文配图:End-to-End Multi-Person Pose Estimation with Pose-Aware Video Transformer
图 1 · 摘自论文原文
  • 全端到端设计,无需检测和非极大值抑制等启发式操作。
  • 在PoseTrack2017上比之前端到端方法提升6.0 mAP,逼近两阶段最优水平。
  • 适合需要高效高精度视频姿态估计的场景,如动作识别与人机交互。

现有视频多人姿态估计方法通常采用两阶段流程:先在每帧中检测个体,再进行时序建模。该设计依赖检测、感兴趣区域裁剪和非极大值抑制等启发式操作,限制了准确率与效率。本文提出一种完全端到端的视频多人群体2D姿态估计框架,有效消除这些启发式操作。核心挑战在于复杂重叠轨迹下的跨帧人物关联。为此,我们引入新的姿势感知视频变换网络(PAVE-Net),包含空间编码器建模帧内关系,以及时空姿态解码器捕捉跨帧全局依赖。提出姿势感知注意力机制,使每个姿态查询能选择性聚合同一人物在连续帧中的特征。同时显式建模关键点间的时空依赖关系以提升精度。本方法是首个端到端的多帧2D人体姿态估计方案。大量实验表明,PAVE-Net显著优于以往基于图像的端到端方法,在PoseTrack2017上提升6.0 mAP,且达到与顶尖两阶段视频方法相当的精度,同时效率大幅提高。

原文摘要 · Abstract (English)

Existing multi-person video pose estimation methods typically adopt a two-stage pipeline: detecting individuals in each frame, followed by temporal modeling for single person pose estimation. This design relies on heuristic operations such as detection, RoI cropping, and non-maximum suppression (NMS), limiting both accuracy and efficiency. In this paper, we present a fully end-to-end framework for multi-person 2D pose estimation in videos, effectively eliminating heuristic operations. A key challenge is to associate individuals across frames under complex and overlapping temporal trajectories. To address this, we introduce a novel Pose-Aware Video transformEr Network (PAVE-Net), which features a spatial encoder to model intra-frame relations and a spatiotemporal pose decoder to capture global dependencies across frames. To achieve accurate temporal association, we propose a pose-aware attention mechanism that enables each pose query to selectively aggregate features corresponding to the same individual across consecutive frames. Additionally, we explicitly model spatiotemporal dependencies among pose keypoints to improve accuracy. Notably, our approach is the first end-to-end method for multi-frame 2D human pose estimation. Extensive experiments show that PAVE-Net substantially outperforms prior image-based end-to-end methods, achieving a 6.0 mAP improvement on PoseTrack2017, and delivers accuracy competitive with state-of-the-art two-stage video based approaches, while offering significant gains in efficiency. Project page: https://github.com/zgspose/PAVENet.

姿态估计视频分析注意力机制端到端

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。