针对体育视频中动作复杂、瞬时性强的问题，提出融合运动解剖信息的3D姿态估计新方法。
KASportsFormer: Kinematic Anatomy Enhanced Transformer for 3D Human Pose Estimation on Short Sports Scene Video
- 引入骨骼提取与肢体融合模块，以解剖结构增强运动特征表示
- 在SportsPose和WorldPose数据集上分别达到58.0mm和34.3mm的MPJPE误差
- 特别适合处理短时瞬时动作，如投篮等关键体育场景
近期基于Transformer的方法在真实世界3D人体姿态估计任务中表现优异。然而，在运动更复杂的体育场景中，由于存在运动模糊、遮挡和域偏移等问题，现有方法性能下降。此外，体育比赛中关键动作常在极短时间内完成（如投篮），对瞬时动作的关注能力成为分析重点，但当前方法对此仍显不足。为此，本文提出KASportsFormer，一种面向体育场景的新型3D姿态估计框架，通过引入运动解剖学信息的特征表示与融合模块，利用骨骼提取器（BoneExt）和肢体融合器（LimbFus）提取并多模态编码固有运动信息，提升对短视频中体育姿态的理解能力。在SportsPose和WorldPose两个代表性体育数据集上的实验表明，该方法分别取得58.0mm和34.3mm的MPJPE误差，达到当前最优水平。
原文摘要 · Abstract (English)
Recent transformer based approaches have demonstrated impressive performance in solving real-world 3D human pose estimation problems. Albeit these approaches achieve fruitful results on benchmark datasets, they tend to fall short of sports scenarios where human movements are more complicated than daily life actions, as being hindered by motion blur, occlusions, and domain shifts. Moreover, due to the fact that critical motions in a sports game often finish in moments of time (e.g., shooting), the ability to focus on momentary actions is becoming a crucial factor in sports analysis, where current methods appear to struggle with instantaneous scenarios. To overcome these limitations, we introduce KASportsFormer, a novel transformer based 3D pose estimation framework for sports that incorporates a kinematic anatomy-informed feature representation and integration module. In which the inherent kinematic motion information is extracted with the Bone Extractor (BoneExt) and Limb Fuser (LimbFus) modules and encoded in a multimodal manner. This improved the capability of comprehending sports poses in short videos. We evaluate our method through two representative sports scene datasets: SportsPose and WorldPose. Experimental results show that our proposed method achieves state-of-the-art results with MPJPE errors of 58.0mm and 34.3mm, respectively. Our code and models are available at: https://github.com/jw0r1n/KASportsFormer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。