arXiv:2509.01095cs.CV2025-09

端到端视频多人姿态估计,提升准确率与推理速度。

An End-to-End Framework for Video Multi-Person Pose Estimation

  • 用三个时空变换器统一处理空间与时间信息。
  • 在Posetrack上精度超越多数两阶段模型,提速300%。
  • 通过实例一致性机制实现跨帧追踪,减少匹配误差。

基于视频的人体姿态估计旨在解决静态图像模型难以应对的运动模糊、失焦和遮挡等问题。现有方法多为两阶段:先检测每帧中的人体实例,再用时序模型进行单人姿态估计。该方法分离了空间与时间维度,无法捕捉全局时空上下文,且依赖独立检测器和复杂的后处理(如RoI裁剪与NMS),降低了视频场景下的推理效率。为此,我们提出VEPE(Video End-to-End Pose Estimation)框架,实现视频中多人姿态估计的端到端建模。该框架引入三种关键的时空变换器组件:时空姿态编码器(STPE)、时空可变形记忆编码器(STDME)和时空姿态解码器(STPD),有效利用时序上下文优化姿态估计。此外,为缓解跨帧姿态查询匹配中的不一致问题,提出实例一致性机制,增强跨帧实例查询的一致性与差异性,实现实例追踪功能,从而精准引导姿态查询完成跨帧匹配。在Posetrack数据集上的大量实验表明,本方法优于多数两阶段模型,并将推理效率提升300%。

原文摘要 · Abstract (English)

Video-based human pose estimation models aim to address scenarios that cannot be effectively solved by static image models such as motion blur, out-of-focus and occlusion. Most existing approaches consist of two stages: detecting human instances in each image frame and then using a temporal model for single-person pose estimation. This approach separates the spatial and temporal dimensions and cannot capture the global spatio-temporal context between spatial instances for end-to-end optimization. In addition, it relies on separate detectors and complex post-processing such as RoI cropping and NMS, which reduces the inference efficiency of the video scene. To address the above problems, we propose VEPE (Video End-to-End Pose Estimation), a simple and flexible framework for end-to-end pose estimation in video. The framework utilizes three crucial spatio-temporal Transformer components: the Spatio-Temporal Pose Encoder (STPE), the Spatio-Temporal Deformable Memory Encoder (STDME), and the Spatio-Temporal Pose Decoder (STPD). These components are designed to effectively utilize temporal context for optimizing human body pose estimation. Furthermore, to reduce the mismatch problem during the cross-frame pose query matching process, we propose an instance consistency mechanism, which aims to enhance the consistency and discrepancy of the cross-frame instance query and realize the instance tracking function, which in turn accurately guides the pose query to perform cross-frame matching. Extensive experiments on the Posetrack dataset show that our approach outperforms most two-stage models and improves inference efficiency by 300%.

视频姿态估计端到端Transformer实例追踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。