arXiv:2512.11340cs.CV2025-12AAAI

用新度量提升少样本动作识别准确率,解决匹配不精准问题

Task-Specific Distance Correlation Matching for Few-Shot Action Recognition

  • 引入任务特异性距离相关性匹配,捕捉帧间非线性关系
  • 在5个基准上超越现有最佳方法,显著提升识别精度
  • 适合研究少样本学习与视频理解的学者参考

少样本动作识别(FSAR)近年来通过集合匹配和大规模预训练模型的高效适配取得了显著进展。然而仍存在两大局限:其一,现有集合匹配度量多依赖余弦相似度衡量帧间线性关联,仅使用实例级信息,难以捕捉非线性模式及任务特定线索;其二,为高效适配CLIP至FSAR,近期工作采用跳接融合层(侧层)微调,大幅降低内存开销,但侧层在数据有限条件下难优化。为此,本文提出TS-FSAR框架,包含三部分:(1) 视觉梯形侧网络(LSN),实现高效CLIP微调;(2) 任务特异性距离相关性匹配(TS-DCM)度量,利用α-距离相关性建模线性与非线性帧间依赖,并引入任务原型实现任务感知匹配;(3) 带有自适应CLIP的引导LSN(GLAC)模块,通过冻结的适配后CLIP正则化LSN,提升有限监督下α-距离相关性估计的稳定性。在五个常用基准上的大量实验表明,所提方法优于现有最先进水平。

原文摘要 · Abstract (English)

Few-shot action recognition (FSAR) has recently made notable progress through set matching and efficient adaptation of large-scale pre-trained models. However, two key limitations persist. First, existing set matching metrics typically rely on cosine similarity to measure inter-frame linear dependencies and then perform matching with only instance-level information, thus failing to capture more complex patterns such as nonlinear relationships and overlooking task-specific cues. Second, for efficient adaptation of CLIP to FSAR, recent work performing fine-tuning via skip-fusion layers (which we refer to as side layers) has significantly reduced memory cost. However, the newly introduced side layers are often difficult to optimize under limited data conditions. To address these limitations, we propose TS-FSAR, a framework comprising three components: (1) a visual Ladder Side Network (LSN) for efficient CLIP fine-tuning; (2) a metric called Task-Specific Distance Correlation Matching (TS-DCM), which uses $α$-distance correlation to model both linear and nonlinear inter-frame dependencies and leverages a task prototype to enable task-specific matching; and (3) a Guiding LSN with Adapted CLIP (GLAC) module, which regularizes LSN using the adapted frozen CLIP to improve training for better $α$-distance correlation estimation under limited supervision. Extensive experiments on five widely-used benchmarks demonstrate that our TS-FSAR yields superior performance compared to prior state-of-the-arts.

少样本学习动作识别视觉匹配CLIP适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。