不训练直接用视觉语言模型检测未见动作，速度快且准确。
Training-Free Zero-Shot Temporal Action Detection with Vision-Language Models
- 直接利用现成视觉语言模型，无需微调或标注
- 在THUMOS14和ActivityNet上超越现有无监督方法，速度提升13倍
- 适合追求快速部署的零样本动作检测场景
现有零样本时序动作检测（ZSTAD）方法多依赖有监督或无监督训练策略，但易受领域偏移影响且计算成本高。本文提出无需训练的零样本时序动作检测方法（FreeZAD），直接使用预训练视觉语言模型对未见动作进行分类与定位，无需额外微调。通过设计对数衰减加权内外对比得分（LogOIC）和基于频率的动作置信度校准，减少对显式时序建模和伪标签质量的依赖。此外，引入测试时自适应策略（TTA）与原型中心采样（PCS），提升模型在实际场景中的表现。在THUMOS14和ActivityNet-1.3数据集上的实验表明，该方法性能优于当前最优无监督方法，且运行时间仅为后者的1/13。结合TTA后，进一步缩小与全监督方法的差距。
原文摘要 · Abstract (English)
Existing zero-shot temporal action detection (ZSTAD) methods predominantly use fully supervised or unsupervised strategies to recognize unseen activities. However, these training-based methods are prone to domain shifts and require high computational costs, which hinder their practical applicability in real-world scenarios. In this paper, unlike previous works, we propose a training-Free Zero-shot temporal Action Detection (FreeZAD) method, leveraging existing vision-language (ViL) models to directly classify and localize unseen activities within untrimmed videos without any additional fine-tuning or adaptation. We mitigate the need for explicit temporal modeling and reliance on pseudo-label quality by designing the LOGarithmic decay weighted Outer-Inner-Contrastive Score (LogOIC) and frequency-based Actionness Calibration. Furthermore, we introduce a test-time adaptation (TTA) strategy using Prototype-Centric Sampling (PCS) to expand FreeZAD, enabling ViL models to adapt more effectively for ZSTAD. Extensive experiments on the THUMOS14 and ActivityNet-1.3 datasets demonstrate that our training-free method outperforms state-of-the-art unsupervised methods while requiring only 1/13 of the runtime. When equipped with TTA, the enhanced method further narrows the gap with fully supervised methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。