不微调模型，用任务向量融合实现更鲁棒的开放词汇动作识别。
Robust Zero-Shot Generalization for Open-Vocabulary Action Recognition via Task Arithmetic
- 通过任务算术合并多个数据集微调后的模型向量。
- 在分布外场景下，零样本泛化性能优于预训练基线模型。
- 适合需快速部署、避免数据隐私风险的应用场景。
开放词汇动作识别（OVAR）利用视觉语言表示实现对新动作的识别，突破了传统封闭集方法的局限。然而，在真实场景中实现鲁棒性能通常需要针对目标域进行微调，这既成本高昂，又引发隐私与合规问题。本文提出一种替代范式，无需目标域训练，而是重新组合现有数据集和模型的知识。通过模型合并与任务算术，从多个公开OVAR数据集上微调的模型中提取并融合任务向量。实验表明，在分布外设置下，融合模型相比预训练基线实现了更优的零样本泛化性能。代码已开源：https://github.com/omaymaMoussadek/robust-ovar。
原文摘要 · Abstract (English)
Open Vocabulary Action Recognition (OVAR) enables the recognition of novel actions by leveraging vision-language representations, overcoming the limitations of traditional closed-set approaches. However, achieving robust performance in real-world scenarios typically requires domain-specific fine-tuning, which is often costly and raises privacy and regulatory concerns. In this work, we propose an alternative paradigm that bypasses target-domain training and recombines knowledge from existing datasets and models. Leveraging model merging and task arithmetic, we extract and combine task vectors from models fine-tuned on diverse public OVAR datasets. We show that, in out-of-distribution settings, the resulting merged model achieves superior zero-shot generalization to the pre-trained base model. Code is available at https://github.com/omaymaMoussadek/robust-ovar
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。