用小数据集对比发现，视觉Transformer比CNN更懂人体动作。
Pose Matters: Evaluating Vision Transformers and CNNs for Human Action Recognition on Small COCO Subsets
- 用三类动作数据测试，ViT准确率达90%，远超CNN的35%。
- 统计检验确认差异显著（F=61.37, p<0.001）。
- 可视化显示ViT关注肢体姿态，而CNN常误认背景纹理。
本研究在COCO图像语料库的三类动作子集上评估人体动作识别，对比从全连接网络到Transformer架构的多种模型。二分类视觉Transformer（ViT）达到90%的平均测试准确率，显著优于卷积网络（约35%）和基于CLIP的模型（约62-64%）。一元方差分析（F = 61.37，p < 0.001）证实差异具有统计显著性。通过SHAP解释器与LeGrad热图进行定性分析表明，ViT能准确定位与动作相关的身体部位（如行走或跑步时的下肢），而简单前馈模型则常聚焦于背景纹理，导致错误。这些结果强调了变压器表征的数据效率及可解释性技术在诊断类别特异性失败中的重要性。
原文摘要 · Abstract (English)
This study explores human action recognition using a three-class subset of the COCO image corpus, benchmarking models from simple fully connected networks to transformer architectures. The binary Vision Transformer (ViT) achieved 90% mean test accuracy, significantly exceeding multiclass classifiers such as convolutional networks (approximately 35%) and CLIP-based models (approximately 62-64%). A one-way ANOVA (F = 61.37, p < 0.001) confirmed these differences are statistically significant. Qualitative analysis with SHAP explainer and LeGrad heatmaps indicated that the ViT localizes pose-specific regions (e.g., lower limbs for walking or running), while simpler feed-forward models often focus on background textures, explaining their errors. These findings emphasize the data efficiency of transformer representations and the importance of explainability techniques in diagnosing class-specific failures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。