用时序移位模块+集成学习，在20类动作识别中达到满分表现。
Action Recognition Using Temporal Shift Module and Ensemble Learning
- 基于时序移位模块捕捉视频动态，融合多模态输入。
- 通过迁移学习与微调，在20类动作上实现100%准确率。
- 集成不同模态输出，显著提升模型鲁棒性与性能。
本文提出针对ICPR 2024多模态视觉模式识别研讨会中的多模态动作识别挑战赛的一流解决方案。该挑战使用包含20个动作类别的多样化数据集，来自多种模态源。方法基于时序移位模块（TSM），高效捕捉视频中的时序动态，并整合多种输入类型。策略包括利用预训练模型进行迁移学习，随后在竞赛特定数据集上进行精细微调以优化20个动作类别的识别性能。精心选择骨干网络，在计算效率与识别精度间取得平衡，并采用集成学习技术融合不同模态的输出，显著提升整体表现。最终方案在测试集上实现100%的顶级准确率，证明了该方法在20类动作识别上的有效性。代码已公开于https://github.com/ffyyytt/TSM-MMVPR。
原文摘要 · Abstract (English)
This paper presents the first-rank solution for the Multi-Modal Action Recognition Challenge, part of the Multi-Modal Visual Pattern Recognition Workshop at the \acl{ICPR} 2024. The competition aimed to recognize human actions using a diverse dataset of 20 action classes, collected from multi-modal sources. The proposed approach is built upon the \acl{TSM}, a technique aimed at efficiently capturing temporal dynamics in video data, incorporating multiple data input types. Our strategy included transfer learning to leverage pre-trained models, followed by meticulous fine-tuning on the challenge's specific dataset to optimize performance for the 20 action classes. We carefully selected a backbone network to balance computational efficiency and recognition accuracy and further refined the model using an ensemble technique that integrates outputs from different modalities. This ensemble approach proved crucial in boosting the overall performance. Our solution achieved a perfect top-1 accuracy on the test set, demonstrating the effectiveness of the proposed approach in recognizing human actions across 20 classes. Our code is available online https://github.com/ffyyytt/TSM-MMVPR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。