arXiv:2501.08446cs.CV2025-01被引 3

Poseidon通过动态加权帧和多尺度融合,提升视频姿态估计精度。

Poseidon: A ViT-based Architecture for Multi-Frame Pose Estimation with Adaptive Frame Weighting and Multi-Scale Feature Fusion

  • 引入自适应帧加权机制,动态聚焦关键帧
  • 在PoseTrack21/18上分别达88.3/87.8的mAP,刷新纪录
  • 适合需要高时序一致性的实时动作分析场景

人体姿态估计是计算机视觉中的核心任务,旨在图像与视频中检测并定位人体关节。尽管单帧姿态估计已取得显著进展,但难以捕捉复杂连续动作的时序动态。本文提出Poseidon,一种基于ViT的多帧姿态估计架构,在ViTPose基础上融入时序信息以提升准确性和鲁棒性。其核心创新包括:(1) 自适应帧加权(AFW)机制,动态优先处理相关帧,确保模型聚焦最有效信息;(2) 多尺度特征融合(MSFF)模块,聚合不同骨干网络层特征,兼顾细节与高层语义;(3) 跨注意力模块,促进中心帧与上下文帧间的信息交互,增强时序一致性。该架构在复杂视频场景中表现优异,具备良好可扩展性与计算效率,适用于真实应用。在PoseTrack21和PoseTrack18数据集上分别取得88.3和87.8的mAP,优于现有方法。

原文摘要 · Abstract (English)

Human pose estimation, a vital task in computer vision, involves detecting and localising human joints in images and videos. While single-frame pose estimation has seen significant progress, it often fails to capture the temporal dynamics for understanding complex, continuous movements. We propose Poseidon, a novel multi-frame pose estimation architecture that extends the ViTPose model by integrating temporal information for enhanced accuracy and robustness to address these limitations. Poseidon introduces key innovations: (1) an Adaptive Frame Weighting (AFW) mechanism that dynamically prioritises frames based on their relevance, ensuring that the model focuses on the most informative data; (2) a Multi-Scale Feature Fusion (MSFF) module that aggregates features from different backbone layers to capture both fine-grained details and high-level semantics; and (3) a Cross-Attention module for effective information exchange between central and contextual frames, enhancing the model's temporal coherence. The proposed architecture improves performance in complex video scenarios and offers scalability and computational efficiency suitable for real-world applications. Our approach achieves state-of-the-art performance on the PoseTrack21 and PoseTrack18 datasets, achieving mAP scores of 88.3 and 87.8, respectively, outperforming existing methods.

姿态估计视频理解ViT多帧融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。