对比视觉语言模型在动作识别中的表现，发现其效果有限。
Are Visual-Language Models Effective in Action Recognition? A Comparative Study
- 用多个数据集测试CLIP等模型的零样本迁移能力
- 在细粒度动作识别任务上表现不如专用模型
- 适合关注跨模态迁移潜力的研究者
当前视觉-语言基础模型（如CLIP）在众多下游任务中表现出显著性能提升。然而，这些模型在更复杂的细粒度动作识别任务中是否仍能带来显著改进，仍是未解之谜。为回答该问题并明确未来野外人类行为分析的研究方向，本文对当前最先进的视觉基础模型进行了大规模比较研究，系统评估其在零样本与帧级动作识别任务上的迁移能力。实验覆盖近期多个细粒度、以人为中心的动作识别数据集（如Toyota Smarthome、Penn Action、UAV-Human、TSU、Charades），涵盖动作分类与分割任务。
原文摘要 · Abstract (English)
Current vision-language foundation models, such as CLIP, have recently shown significant improvement in performance across various downstream tasks. However, whether such foundation models significantly improve more complex fine-grained action recognition tasks is still an open question. To answer this question and better find out the future research direction on human behavior analysis in-the-wild, this paper provides a large-scale study and insight on current state-of-the-art vision foundation models by comparing their transfer ability onto zero-shot and frame-wise action recognition tasks. Extensive experiments are conducted on recent fine-grained, human-centric action recognition datasets (e.g., Toyota Smarthome, Penn Action, UAV-Human, TSU, Charades) including action classification and segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。