融合骨骼信息提升滑冰摔倒识别准确率，最高提升40%。
Gate-Shift-Pose: Enhancing Action Recognition in Sports with Skeleton Information
- 用骨骼关键点热图与图像早期融合，增强动作捕捉能力。
- 在FR-FS数据集上，使用ResNet50时准确率达98.08%，比纯图像提升40%。
- 适合需要高精度动作识别的体育分析场景，尤其关注复杂运动模式。
本文提出Gate-Shift-Pose，是Gate-Shift-Fuse网络的改进版本，用于滑冰运动员摔倒分类任务，通过结合RGB图像与骨骼姿态数据提升性能。设计了两种融合策略：早期融合在输入阶段将RGB帧与骨骼关键点高斯热图合并；晚期融合采用多流架构，通过注意力机制整合RGB与姿态特征。在FR-FS数据集上的实验表明，该方法显著优于仅使用RGB的基线模型，使用ResNet18时准确率提升达40%，使用ResNet50时提升20%。早期融合在ResNet50上表现最佳，准确率达到98.08%，充分利用了模型对多模态信息的整合能力；而晚期融合更适合轻量级主干网络如ResNet18。结果验证了多模态架构在体育动作识别中的潜力，凸显骨骼姿态信息对捕捉复杂运动模式的关键作用。
原文摘要 · Abstract (English)
This paper introduces Gate-Shift-Pose, an enhanced version of Gate-Shift-Fuse networks, designed for athlete fall classification in figure skating by integrating skeleton pose data alongside RGB frames. We evaluate two fusion strategies: early-fusion, which combines RGB frames with Gaussian heatmaps of pose keypoints at the input stage, and late-fusion, which employs a multi-stream architecture with attention mechanisms to combine RGB and pose features. Experiments on the FR-FS dataset demonstrate that Gate-Shift-Pose significantly outperforms the RGB-only baseline, improving accuracy by up to 40% with ResNet18 and 20% with ResNet50. Early-fusion achieves the highest accuracy (98.08%) with ResNet50, leveraging the model's capacity for effective multimodal integration, while late-fusion is better suited for lighter backbones like ResNet18. These results highlight the potential of multimodal architectures for sports action recognition and the critical role of skeleton pose information in capturing complex motion patterns. Visit the project page at https://edowhite.github.io/Gate-Shift-Pose
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。