arXiv:2604.22595cs.CV2026-04

针对弱光等视觉挑战,提出高效视觉提示框架提升少样本动作识别性能

EV-CLIP: Efficient Visual Prompt Adaptation for CLIP in Few-shot Action Recognition under Visual Challenges

论文配图:EV-CLIP: Efficient Visual Prompt Adaptation for CLIP in Few-shot Action Recognition under Visual Challenges
图 1 · 摘自论文原文
  • 设计掩码与上下文双重视觉提示,增强模型对关键区域的关注和轻量时序建模
  • 在5个基准数据集上超越现有参数高效方法,且效率不随主干网络规模变化
  • 适合资源受限场景部署,尤其适用于复杂视角和低光照环境下的动作识别

CLIP通过自然语言监督在视觉领域展现出强大泛化能力,即使在视频动作识别中也表现优异。然而,现有适配方法多聚焦于时序建模,忽视了空间感知的重要性。在真实场景中,如低光环境或第一人称视角会严重损害空间理解,而空间理解是有效时序推理的前提。为此,我们提出高效视觉提示框架EV-CLIP,用于跨多样场景与视角的少样本视频动作识别。EV-CLIP引入两种视觉提示:掩码提示通过重加权像素引导模型关注动作相关区域;上下文提示通过压缩帧级特征为紧凑表示实现轻量时序建模。我们构建了五个基准数据集并分析域偏移,量化不同视觉与语义因素对动作识别的影响。实验表明,EV-CLIP在整体性能上优于现有参数高效方法,且其效率不依赖主干网络规模,非常适合资源受限的真实场景部署。代码已开源。

原文摘要 · Abstract (English)

CLIP has demonstrated strong generalization in visual domains through natural language supervision, even for video action recognition. However, most existing approaches that adapt CLIP for action recognition have primarily focused on temporal modeling, often overlooking spatial perception. In real-world scenarios, visual challenges such as low-light environments or egocentric viewpoints can severely impair spatial understanding, an essential precursor for effective temporal reasoning. To address this limitation, we propose Efficient Visual Prompting for CLIP (EV-CLIP), an efficient adaptation framework designed for few-shot video action recognition across diverse scenes and viewpoints. EV-CLIP introduces two visual prompts: mask prompts, which guide the model's attention to action-relevant regions by reweighting pixels, and context prompts, which perform lightweight temporal modeling by compressing frame-wise features into a compact representation. For a comprehensive evaluation, we curate five benchmark datasets and analyze domain shifts to quantify the influence of diverse visual and semantic factors on action recognition. Experimental results demonstrate that EV-CLIP outperforms existing parameter-efficient methods in overall performance. Moreover, its efficiency remains independent of the backbone scale, making it well-suited for deployment in real-world, resource-constrained scenarios. The code is available at https://github.com/AI-CV-Lab/EV-CLIP.

少样本识别视觉提示动作识别CLIP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。