arXiv:2504.05956cs.CVcs.AI2025-04CVPR被引 13

无需对齐时间单元,用固定模式令牌实现高效少样本动作识别

Temporal Alignment-Free Video Matching for Few-shot Action Recognition

  • 用固定数量的模式令牌全局捕捉视频特征,摆脱帧或片段依赖
  • 通过令牌级对比计算相似度,匹配效率显著提升
  • 自适应去除类别共性信息,提升新类别间区分能力

少样本动作识别(FSAR)旨在仅用少量标注视频实例训练模型。其核心挑战在于处理不同叙事轨迹带来的精确视频匹配问题。现有基于帧或元组的对齐方法依赖预定义且长度相关的对齐单元,限制了对不同长度和速度动作的灵活性。本文提出一种新型无时间对齐匹配方法(TEAM),消除动作表征中的时间单元需求及匹配时的暴力对齐过程。TEAM将每段视频表示为一组固定的模式令牌,捕捉视频内全局判别性线索,不受动作长度与速度影响,具备高度灵活性。此外,TEAM天然高效,通过令牌级对比而非成对时间对齐来衡量视频相似度。我们还设计了一种自适应机制,识别并移除跨类别的共性信息,强化新类别间的边界区分。大量实验验证了该方法的有效性。代码已开源。

原文摘要 · Abstract (English)

Few-Shot Action Recognition (FSAR) aims to train a model with only a few labeled video instances. A key challenge in FSAR is handling divergent narrative trajectories for precise video matching. While the frame- and tuple-level alignment approaches have been promising, their methods heavily rely on pre-defined and length-dependent alignment units (e.g., frames or tuples), which limits flexibility for actions of varying lengths and speeds. In this work, we introduce a novel TEmporal Alignment-free Matching (TEAM) approach, which eliminates the need for temporal units in action representation and brute-force alignment during matching. Specifically, TEAM represents each video with a fixed set of pattern tokens that capture globally discriminative clues within the video instance regardless of action length or speed, ensuring its flexibility. Furthermore, TEAM is inherently efficient, using token-wise comparisons to measure similarity between videos, unlike existing methods that rely on pairwise comparisons for temporal alignment. Additionally, we propose an adaptation process that identifies and removes common information across classes, establishing clear boundaries even between novel categories. Extensive experiments demonstrate the effectiveness of TEAM. Codes are available at github.com/leesb7426/TEAM.

少样本学习视频匹配动作识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。